What does echelon mean?

As has already been suggested, it has its origins in military vocabulary.

The Wiki article on echelon formation contains many pictures of stuff in echelon formation, and you can immediately see why one might say that the rows of a row-echelon matrix are in "echelon formation." I would have liked to include the wiki images, but for some reason they would not load. I found another one instead:

From Online Etymology Dictionary:

echelon (n.)

1796, echellon, "step-like arrangement of troops," from French échelon "level, echelon," literally "rung of a ladder," from Old French eschelon, from eschiele "ladder," from Late Latin scala "stair, slope," from Latin scalae (plural) "ladder, steps," from PIE $\ast$skand- "to spring, leap" (see scan (v.)). Sense of "level, subdivision" is from World War I.

Steven Schwartzman's The Words of Mathematics: An Etymological Dictionary of Mathematical Terms (MAA, 1994) also attributes the root of the term to the French word échelon and its meaning of "rung of a ladder" (link from Google Books).

The earliest appearance of "echelon form" as a linear algebra term that I can find is in Birkhoff and Mac Lane (1953), A Survey of Modern Algebra (link from Google Books). Yet, Nathan Jacobson's Lectures in Abstract Algebra: II. Linear Algebra (1953) does not contain this term. I guess the term was introduced to linear algebra more or less around the 1950s.

It simply means the rows are ordered in a unique hierarchy by the positions of their leading ones (rows of all zeros being lumped at the bottom).