Why don't we use base 6 or 11?
The idea of a "base", and even the idea implicit in it of a consistent positional numbering system, is a relatively modern one. For that matter, even "base 10", in the sense of decimal numerals — the Hindu–Arabic numeral system we all use today — is relatively modern; witness the fact that much of Europe didn't begin to use it until well into the second millenium. The idea that we can count in any base is even newer.
Your argument might make sense if humanity started with the idea in mind of using a base-$b$ representation for some $b$, and then looked to their fingers to decide what the base $b$ should be. This of course is not what happened, nor even is it imaginable in any culture. The concrete precedes the abstract.
Instead, what we see historically is counting on one's fingers, and thus counting by fives or by tens, not counting in base 10. If you're counting on two hands, when you reach $10$ you literally run out of fingers to count, and that is where you have to leave a mental (or physical) note to yourself that you're done with one round of counting, and begin counting anew. (Similarly $5$, if using one hand.)
We can see traces of this "counting by $5$" or "counting by $10$" in early systems like Roman numerals: note that $8$ is represented as "VIII", denoting one count of five (done with one hand's worth of counting), and starting again, reaching up to $3$ in the process. Similarly, the representations "XX" and "XXX" show that they were being thought of as "two tens" and "three tens", rather than as "in base $10$, three in the tens place and zero in the units place" — the idea of base $10$ is not actually present. Thus "XXXVIII" literally denotes the process of counting: "three tens (three double-hands), a five (one hand) and three fingers". (There are even traces of counting by $20$s; consider the English "score" and French number names).
It was only after centuries of already counting by $10$s, conducting transactions with $10$-based words for numbers and so on, that the base-$10$ representation arose, as a representation system for the same numbers that everyone was already accustomed to thinking of in tens, and (very slowly) spread across the world.
Let's consider your base 6 proposal. You motivate $10_6$ to be the first number that you can't count on one hand. To be consistent, $20_6$ should be the first number that you can't count on two hands. But it isn't! Instead:
- $10_6$ is $1$ more than the number you can count on $1$ hand
- $20_6$ is $2$ more than the number you can count on $2$ hands
- $30_6$ is $3$ more than the number you can count on $3$ hands
What a strange pattern. Wouldn't it be better if $d$ more than the number you can count on $d$ hands were written $dd$? Well, that's what we get in base 5:
- $11_5$ is $1$ more than the number you can count on $1$ hand
- $22_5$ is $2$ more than the number you can count on $2$ hands
- $33_5$ is $3$ more than the number you can count on $3$ hands
And there's an even nicer pattern for numbers of the form $d0$:
- $10_5$ is the number you can count on $1$ hand
- $20_5$ is the number you can count on $2$ hands
- $30_5$ is the number you can count on $3$ hands
The same arguments apply to 10 versus 11.
Well, you're clearly aware that different cultures have used different bases, and indeed developed sophisticated mathematics using them. Further, most of the mathematical advances in the West over the last 500 years (say) would seem to have nothing to do with what base we use in ordinary life.
So I'm not quite sure where the "why" question comes in. I agree that claims that base 10 is "natural", or is more efficient than some other base, seem dubious. So what are we seeking to explain here, exactly?
If it's a simple causal explanation that's wanted -- "How did it come to be that we use base 10" -- then the answer is, I'm willing to bet, that it's an accident of history.
In other words: isn't this a bit like asking why we have 26 letters in the alphabet?