How should feedback be given for "silly mistakes" on an exam
First, notice that students will typically be able to tell which of their mistakes were silly and which more significant. So for silly mistakes you don't need to write feedback other than to circle where they went wrong, and indicate the number of points taken off (or granted).
For a more serious mistake you can write more detailed feedback. Sometimes feedback here isn't necessary, either, other than to write, "see solution."
Second, you can figure out what types of errors and what frequency of errors you want to bring the grade down from A to B, from A to C, etc., and decide how many points to remove based on that.
More importantly: what's a well written exam like? Example: I'm asking them to solve this eigenvalue-eigenvector problem. If the student has a vague idea what is being sought, I want him to get one point. If he sets things up well but doesn't know what to do next, he gets two points. If he follows through well but for whatever reason didn't quite get to the perfectly correct punch line, he'll get 3 points. Perfect, complete answer: 4 points. (This is just an example of the point distribution. You might end up with a different scoring structure.) In short, the scoring should be integral to the exam design from the beginning.
Most importantly: as a TA, you should be getting special guidance from the professor for grading a midterm exam.
This is mainly at the teacher's choice, in practice. Some deduct, say, zero or 5% of the grade for a silly arithmetic mistake (which I recommend); some deduct much more --- to the limit of those who look at the final result only and deduct 100% if it doesn't match the official solution (which I don't recommend).
It may be difficult to give an objective evaluation because in some cases a mistake at the beginning can turn an exercise into a completely different one, much easier or harder. If there is more than one person grading, you should discuss it with the rest of the group.
As a mathematician, an interesting point to consider is that not all silly mistakes are equal. In some cases, a student should realize they have made a mistake with some sanity checks at the end. Examples: if you have shown that a certain event has probability -3.72 to happen, it is clear that there is something wrong. If your symmetric matrix has non-real eigenvalues, you should notice it (if you were taught that it is impossible). More subtly, the eigenvalues you have computed may fail the trace test (sum of eigenvalues = sum of diagonal entries of the matrix); a smart student would make that check at the end as well.
In my view, submitting an answer with mistakes that fail obvious sanity checks deserves a more substantial deduction: maybe student A and student B both flipped a sign, but while student A got a perfectly plausible solution, student B really should have realized that. It's part of their job to check that the solution they find is reasonable. Especially in this age when computers do most of the work in practice, noticing when a computed solution is patently wrong is arguably more important than computing it in the first place.
I've used a different color ink to distinguish feedback which does not contribute to the grade this time but which might in the future. One of the general problems with an educational system which uses grades punitively as "feedback" is that completely non-punitive feedback is almost always ignored.