Covariant derivative of a tangent vector

I'm going to propose an approach to justify the formula in the OP employing the idea of pullback bundles and pullback connections.

Introduction to the Setup

Let $\Sigma : W\subset \mathbb{R}^2\to M$ be the embedding of the worldsheet on spacetime. To connect with more usual notation, if $x^\mu$ is a coordinate chart on some open subset of $M$ then $X^\mu = x^\mu \circ \Sigma$ are the coordinates of the worldsheet.

Now you have a metric $g$ on $M$. Given that we can always pullback this metric to $W$ by the embedding $\Sigma$. This is your pullback metric $$\gamma = \Sigma^\ast g.$$

Now you want to understand differentiation of $t^\mu_A$. First you should ask what this is as an intrinsic object. The first point is that these are functions $t^\mu_A(\xi)$ in the worldsheet. Hence you would like to view it as a section of some bundle over $W$. Since it has two indices it must correspond to some tensor product bundle. The covariant index part is easy: it corresponds to the cotangent bundle $T^\ast W$. The contravariant index should correspond to the tangent bundle $TM$, but now notice that $t^\mu_A$ is meant to be a vector field just over the image $\Sigma(W)$, so the appropriate bundle is the pullback bundle $\Sigma^\ast (TM)$.

Brief review of the Pullback Bundle

The pullback bundle is the appropriate construction to talk about "vector fields over some embedded submanifold". Focusing in your case, it is defined to be $$\Sigma^\ast (TM)=\{(\xi,v)\in W\times TM : v\in T_{\Sigma(\xi)}M\},\quad \pi(\xi,v)=\xi,$$

where $\pi$ is the bundle projection. It is the space of all vectors in $M$ which lie on points of the embedded worldsheet $\Sigma(W)$. A section $\Sigma^\ast(TM)$ is meant to be a map $S : W\to \Sigma^\ast(TM)$ such that $S(\xi)=(\xi,{\cal S}(\xi))$ where ${\cal S} : W\to TM$ with the property that ${\cal S}(\xi)\in T_{\Sigma(\xi)}M$.

One special kind of section is obtained by taking a vector field in $M$, say $X : M\to TM$ and restricting it to $\Sigma(W)$, thereby definining the section $$(\Sigma^\ast X)(\xi)=(\xi,X(\Sigma(\xi))).$$

This is called a pullback section and it arises when ${\cal S}$ is a composition $X\circ \Sigma$ of a vector field with an embedding.

Now if $E_a$ is a local frame in $M$ in a neighborhood of $\Sigma(W)$, since ${\cal S}(\xi)\in T_{\Sigma(\xi)}M$ we can always expand $${\cal S}(\xi)={\cal S}^a(\xi)E_a(\Sigma(\xi))$$

and therefore a section $S : W\to \Sigma^\ast(TM)$ is always expanded in a basis of pullback sections $$S(\xi)={\cal S}^a(\xi) \Sigma^\ast E_a(\xi)$$

The connection on $\Sigma^\ast(TM)\otimes T^\ast W$

All this construction allows that a connection $\nabla$ on $TM$ naturally induce a connection $\Sigma^\ast \nabla$ on $\Sigma^\ast(TM)$. For that one defines its action on pullback sections as $$(\Sigma^\ast\nabla)_{Z}(\Sigma^\ast X)=\Sigma^\ast\bigg(\nabla_{\Sigma_\ast Z} X\bigg),\quad Z\in \Gamma(TW), X\in \Gamma(TM).$$

This is how the connection $\nabla$ on the spacetime manifold will act upon the contravariant index of $t^\mu_A$. Pay attention because you'll see that this pushforward $\Sigma_\ast Z$ is what will make the additional $t^\nu_B$ appear in your middle term.

We now have two bundles over $W$: the pullback bundle $\Sigma^\ast (TM)$ of spacetime vectors over the worldsheet, with the pullback connection $(\Sigma^\ast \nabla)$ and the cotangent bundle $T^\ast W$ with the metric induced connection $D$. We can form their tensor product $\Sigma^\ast(TM)\otimes T^\ast W$ and endow it with a connection $(\Sigma^\ast \nabla)\otimes D$ defined to act on tensor products of sections:

$$(\Sigma^\ast \nabla\otimes D)_Z(f\otimes g)=(\Sigma^\ast \nabla_Z f)\otimes g+ f\otimes (D_Z g).$$

Since tensor products form a basis this fully defines the connection $\Sigma^\ast \nabla \otimes D$.

Deriving the Result

Finally we are ready to derive the result. You can show by the chain rule that $t^\mu_A$ are the components of a section of $\Sigma^\ast(TM)\otimes T^\ast W$. Therefore consider $${\frak t}=t^\mu_A (\Sigma^\ast \partial_\mu)\otimes d\xi^A.$$

We wish to evaluate $(\Sigma^\ast \nabla\otimes D)_Z$ of this expression. This will be:

$$(\Sigma^\ast \nabla\otimes D)_Z\mathfrak{t} = \bigg[(\Sigma^\ast \nabla\otimes D)_Zt^\mu_A\bigg](\Sigma^\ast \partial_\mu)\otimes d\xi^A+t^\mu_A\bigg[(\Sigma^\ast \nabla\otimes D)_Z(\Sigma^\ast \partial_\mu)\otimes d\xi^A\bigg]$$

The first term has a covariant derivative of a real-valued function. This is just the application of $Z$ on the functions $t^\mu_A$. On the second term we employ the definition

$$(\Sigma^\ast \nabla\otimes D)_Z\mathfrak{t} = (Zt^\mu_A)(\Sigma^\ast \partial_\mu)\otimes d\xi^A+t^\mu_A\bigg[(\Sigma^\ast \nabla)_Z(\Sigma^\ast \partial_\mu)\otimes d\xi^A+(\Sigma^\ast \partial_\mu)\otimes D_Zd\xi^A\bigg]$$

Since the second term is a the covariant derivative of a pullback section using the definition we find

$$(\Sigma^\ast \nabla\otimes D)_Z\mathfrak{t} = (Zt^\mu_A)(\Sigma^\ast \partial_\mu)\otimes d\xi^A+t^\mu_A\bigg[\Sigma^\ast (\nabla_{\Sigma_\ast Z}\partial_\mu)\otimes d\xi^A+(\Sigma^\ast \partial_\mu)\otimes D_Zd\xi^A\bigg]$$

The last term is very easy to evaluate. It is just $D_Z d\xi^A = -Z^C\gamma^A_{CB}d\xi^B$ by definition of the connection coefficients. The second term, however, demands us to evaluate $\Sigma_\ast Z$. This is a pushforward, which we know can be evaluated as $$\Sigma_\ast Z = \dfrac{\partial (x^\nu\circ \Sigma)}{\partial \xi^B}Z^B \partial_\nu = t^\nu_B Z^B \partial_\nu.$$

Putting it all together and relabeling indices to factor the basis vectors it yields

$$(\Sigma^\ast \nabla\otimes D)_Z\mathfrak{t} = \bigg[Z^B\partial_B t^\mu_A+t^\alpha_A t^\nu_B Z^B \Gamma_{\nu\alpha}^{\mu}-t^\mu_B Z^C\gamma^B_{CA}\bigg](\Sigma^\ast \partial_\mu)\otimes d\xi^A$$

In the particular case in which $Z = \partial/\partial \xi^B$ the components of this derivative is your result.

Having put the label $B$ on the covariant derivative $D_{B}$ there is no reason why such a derivative should be sensitive to the $\mu$ label. In other words you can differentiate each of the $D$ (two-component worldsheet) vectors $t_{A}^{\mu}$, but the space-time label $\mu$ will be sterile to the action of $D_{B}$.

It is a little like when you make a worldsheet reparameterisation on the fields $X^{\mu}(\tau, \sigma)$. Each of the $D$ fields (one for each value of $\mu$) will transform as a diffeomorphism scalar and its index $\mu$ plays no role on the transformation.

In particular the first and last terms of your proposed covariant derivative work fine from the perspective of the worldsheet but the second one is out of place (where is its derivative $t_{A}^{\mu, \nu}$ to team up with the connection term?).