Tension in a massless string being pulled at its ends with unequal forces

The arrangement you describe is impossible. The tension of the string will be 70N. Whatever was trying to restrain the end of the string with a force of 60N will be subject to a force of 70N by the string. As a result it will accelerate subject to a net force of 10N. The reaction on the string will be 70N.

I think you are on the right track. Instead of looking at a net force on the string, which would not be possible as @Marco Ocram points out, consider that the net force is acting on a system consisting of the string and masses connected to each end. Then the system, including the string, can be considered accelerating with no net force on the string itself.

As an example, the diagram below is a system of two masses and a string, in this case without a pulley. The top diagram shows two external forces of 70 N and 60 N acting on the masses. The bottom diagram shows the equivalent system with a net force of 10 N acting on the system.

The acceleration of the system is given by

$$a=\frac {10}{M_{1}+M_{2}}$$

Since both masses have the same acceleration, the net force $F_1$ acting on $M_1$ is by Newton's second law

$$F_{1}=\frac {10M_{1}}{M_{1}+M_{2}}$$

Since the only net external force acting on $M_1$ is the tension in the string, this also equals the tension on the string, or

$$T=\frac {10M_{1}}{M_{1}+M_{2}}$$

The net force acting on $M_2$ is by Newton' second law

$$F_{2}=\frac {10M_{2}}{M_{1}+M_{2}}$$

Now we know that the string is experiencing the same acceleration $a$ at the two masses. Per Newton's second law on the string you have, where $F_{net}$ is the net force on the string,

$$F_{net}=M_{string}a$$.

If both $F_{net}$ and $M_{string}$ are zero, you can have any non-zero value of $a$ without violating Newton's second law.

Some observations:

If $M_1$ = 0, $T$ = 0, as expected.

If $M_2$ = 0, $T$ = 10N, as expected.

Hope this helps.

Consider a variation on the Atwood machine where instead of hanging straight down, the two masses $m_1$ and $m_2$ each sit on a separate frictionless plane at an angle of $\theta_1$ and $\theta_2$ to the vertical. The planes are joined along a ridge and the string connecting the two masses runs over the ridge, again without friction. The simple Atwood machine corresponds to the case $\theta_1=\theta_2=0$.

We set the values of $m_1$ and $m_2$ so that

$m_1g \cos \theta_1 = 70 N\\m_2g \cos \theta_2 = 60 N$

so there is a force of $70$ N on one end of the string and a force of $60$ N on the other end. The accelerations of $m_1$ and $m_2$ must be equal in magnitude and opposite in direction, so we have

$\frac{70-T}{m_1}=\frac{T-60}{m_2}$

where $T$ is the tension in the string. This gives us

$T= \frac{60m_1+70m_2}{m_1+m_2}$

If we let $k=\frac{\cos \theta_2}{\cos \theta_1}$ we can rewrite $T$ in terms of $k$:

$T=\frac{4200(1+k)}{60+70k}$

By varying $\theta_1$ and $\theta_2$ we can give $k$ any value we like between $0$ and $\infty$, and so $T$ can take any value between $60$ N and $70$ N.