use GNU sort to sort by a single key / prevent unwanted sorting of other keys

You need a stable sort. From man sort:

-s, --stable
       stabilize sort by disabling last-resort comparison

viz.:

$ sort -sk 1,1 <data.txt
1 Don't
1 Sort
1 Me
2 C 
2 B
2 A

Note that you probably also want a -n or --numeric-sort if your key is numeric (for example, you may get unexpected results when comparing 10 to 2 with the default - lexical - sort order). In which case it's just a matter of doing:

sort -sn <data.txt

No need to extract the first field as the numeric interpretation of the whole line will be the same as the one of the first field.

For (non-GNU) sort implementations that lack a -s option, you can always do:

<data.txt awk '{print NR "\t" $0}' | sort -n -k 2,2 -k 1,1 | cut -f 2-

That is, prepend the line number to make it the second sort key, and strip it afterwards.