python3: singledispatch in class, how to dispatch self type

You cannot use functools.singledispatch on methods at all, not as a decorator at least. Python 3.8 adds a new option, just for methods: functools.singledispatchmethod().

It doesn't matter that Vector isn't defined here yet; the first argument to any method is always going to be self, while you'd use single dispatch for the second argument here.

Because decorators apply to the function objects before the class object is created, you could just as well register your 'methods' as functions instead, outside of the class body, so you have access to the Vector name:

class Vector(object):

    @functools.singledispatch
    def __mul__(self, other):
        return NotImplemented

@Vector.__mul__.register(int)
@Vector.__mul__.register(Vector)                
def _(self, other):
    result = Vector(len(self))           # start with vector of zeros
    for j in range(len(self)):
        result[j] = self[j]*other
    return result

For non-supported types, you need to return the NotImplemented singleton, not raise an exception. This way Python will try the inverse operation too.

However, since the dispatch is going to key on the wrong argument (self) here anyway, you'll have to come up with your own single dispatch mechanism.

If you really want to use @functools.singledispatch you'd have to delegate to a regular function, with the arguments inversed:

@functools.singledispatch
def _vector_mul(other, self):
    return NotImplemented

class Vector(object):
    def __mul__(self, other):
        return _vector_mul(other, self)


@_vector_mul.register(int)
def _vector_int_mul(other, self):
    result = Vector(len(self))
    for j in range(len(self)):
        result[j] = self[j] * other
    return result

As for your updates using __init__mul__: v * 3 is not translated to v.__mul__(3). It is instead translated to type(v).__mul__(v, 3), see Special method lookup in the Python datamodel reference. This always bypasses any methods set directly on the instance.

Here type(v) is Vector; Python looks up the function, it won't use a bound method here. Again, because functools.singledispatch dispatches on the first argument, always, you cannot use single dispatch directly on the methods of Vector, because that first argument is always going to be a Vector instance.

In other words, Python will not use the methods you set on self in __init__mul__; special methods are never looked up on the instance, see Special method lookup in the datamodel documentation.

The functools.singledispatchmethod() option that Python 3.8 adds uses a class as the decorator which implements the descriptor protocol, just like methods do. This lets it then handle dispatch before binding (so before self would be prepended to the argument list) and then bind the registered function that the singledispatch dispatcher returns. The source code for this implementation is fully compatible with older Python versions, so you could use that instead:

from functools import singledispatch, update_wrapper

# Python 3.8 singledispatchmethod, backported
class singledispatchmethod:
    """Single-dispatch generic method descriptor.

    Supports wrapping existing descriptors and handles non-descriptor
    callables as instance methods.
    """

    def __init__(self, func):
        if not callable(func) and not hasattr(func, "__get__"):
            raise TypeError(f"{func!r} is not callable or a descriptor")

        self.dispatcher = singledispatch(func)
        self.func = func

    def register(self, cls, method=None):
        """generic_method.register(cls, func) -> func

        Registers a new implementation for the given *cls* on a *generic_method*.
        """
        return self.dispatcher.register(cls, func=method)

    def __get__(self, obj, cls):
        def _method(*args, **kwargs):
            method = self.dispatcher.dispatch(args[0].__class__)
            return method.__get__(obj, cls)(*args, **kwargs)

        _method.__isabstractmethod__ = self.__isabstractmethod__
        _method.register = self.register
        update_wrapper(_method, self.func)
        return _method

    @property
    def __isabstractmethod__(self):
        return getattr(self.func, '__isabstractmethod__', False)

and apply that to your Vector() class. You still have to register your Vector implementation for the single dispatch after the class has been created, because only then can you register a dispatch for the class:

class Vector(object):
    def __init__(self, d):
        self._coords = [0] * d

    def __setitem__(self, key, value):
        self._coords[key] = value

    def __getitem__(self, item):
        return self._coords[item]

    def __len__(self):
        return len(self._coords)

    def __repr__(self):
        return f"Vector({self._coords!r})"

    def __str__(self):
        return str(self._coords)

    @singledispatchmethod
    def __mul__(self, other):
        return NotImplemented

    @__mul__.register
    def _int_mul(self, other: int):
        result = Vector(len(self))
        for j in range(len(self)):
            result[j] = self[j] * other
        return result

@Vector.__mul__.register
def _vector_mul(self, other: Vector):
    return sum(sc * oc for sc, oc in zip(self._coords, other._coords))

You could of course also create a subclass first and dispatch based on that, since dispatch works for subclasses too:

class _Vector(object):
    def __init__(self, d):
        self._coords = [0] * d

class Vector(_Vector):
    def __setitem__(self, key, value):
        self._coords[key] = value

    def __getitem__(self, item):
        return self._coords[item]

    def __len__(self):
        return len(self._coords)

    def __repr__(self):
        return f"{type(self).__name__}({self._coords!r})"

    def __str__(self):
        return str(self._coords)

    @singledispatchmethod
    def __mul__(self, other):
        return NotImplemented

    @__mul__.register
    def _int_mul(self, other: int):
        result = Vector(len(self))
        for j in range(len(self)):
            result[j] = self[j] * other
        return result

    @__mul__.register
    def _vector_mul(self, other: _Vector):
        return sum(sc * oc for sc, oc in zip(self._coords, other._coords))

This is a little ugly, as you need to defer binding the implementation of Vector/Vector multiplication until after Vector is actually defined. But the idea is that the single-dispatch function needs the first argument to be of arbitrary type, so Vector.__mul__ will call that function with self as the second argument.

import functools

class Vector:

    def __mul__(self, other):
        # Python has already dispatched Vector() * object() here, so
        # swap the arguments so that our single-dispatch works. Note
        # that in general if a*b != b*a, then the _mul_by_other
        # implementations need to compensate.
        return Vector._mul_by_other(other, self)

    @functools.singledispatch
    def _mul_by_other(x, y):
        raise NotImplementedError("Can't multiply vector by {}".format(type(x)))

    @_mul_by_other.register(int)
    def _(x, y):
        print("Multiply vector by int")

@Vector._mul_by_other.register(Vector)
def _(x, y):
    print("Multiply vector by another vector")

x = Vector()
y = Vector()
x * 3
x * y
try:
    x * "foo"
except NotImplementedError:
    print("Caught attempt to multiply by string")