How does one refute this ultrafinitist argument?

I am agnostic about most questions of the philosophy of mathematics, but in this case, I think Nelson's argument is incoherent: a primitive shepherd who doesn't have any abstract concept of number can count sheep being herded into a fold by making marks on a stick and count the sheep out again by crossing the marks off (and this is how our human notions of cardinal numbers probably developed). There is no circularity involved.

To talk meaningfully about notations like $2 \uparrow\uparrow\uparrow 6$, you must have admitted a notion of definition by recursion that (as an ultrafinitist) Nelson can't accept. It seems to be me to be incoherent for an ultrafinitist to say anything more than "I can't accept that a primitive shepherd could ever have enough sticks".

An arithmetic theory (like Peano's) doesn't define what we understand by natural numbers, it is just an axiomatic theory over a formal deductive system, the circularity only appears when we try to associate each natural number to a specific term of the formal language the theory is based on, because to comunicate that association we should say something like "the term associated to the number TWO is the result of applying TWO times the successor function to the closed term denoted by the zero symbol". Here is where it is necessary to distinguish clearly which propositions belong to the theory and which ones belong to the meta-theory. When you say "refute" you don't mean a formal refutation of Nelson's argument, since it isn't a formal argument, so there is not clear and objective way to determine if his argument is valid (we are not talking about logical validity)