Is torsion a property of a manifold or of a connection?

The torsion is indeed defined from the connection, independently of the manifold and metric. By synedoche, people sometimes refer to the structure composed of the manifold, the differential structure, the metric and the connection, $(M, \mathfrak{A}, g, \nabla)$, as "the manifold", even though those are somewhat independent objects.

Given a reasonable manifold, it is always possible to find a torsionless connection on it, since every (metrizable) manifold admits a metric tensor, and every manifold with a metric admits a Levi-Civita connection. As a general rule, the difference between two connections is

\begin{equation} \nabla_a \omega_b = \tilde{\nabla}_a \omega_b - {C^c}_{ab} \omega_c \end{equation}

If we have a connection with a torsion tensor ${T^c}_{ab}$, we can in particular define $C$ to be the torsion tensor, so that any connection with torsion can give rise to a torsionless connection.

Similarly, if you have a torsion-free connection, and you add a tensor field ${C^c}_{ab}$ that is not symmetric in $a$ and $b$ (you can always do this by picking the zero tensor plus a non-zero antisymmetric tensor in a small neighbourhood), then this will give rise to a connection with torsion.