How to make a better `Sort` in my case

How about this one?:

SortBy[#, FileBaseName] &@fileNames

How about this?:

#[[Ordering[StringTake[FileBaseName@# <> "0", 2] & /@ #]]] &@fileNames

there is a nasty way of doing this:

orig = (Riffle[ToCharacterCode[fileNames], (Characters /@ fileNames)]) // 
Partition[#, 2] & // Thread[#] & /@ # & // #[[All, {21, 22}]] &

(*{{{54, "6"}, {50, "2"}}, {{49, "1"}, {46, "."}}, {{50, "2"}, {46,"."}},
{{51, "3"}, {46, "."}}, {{52, "4"}, {46, "."}}, {{53,"5"}, {46, "."}},
{{54, "6"}, {46, "."}}, {{55, "7"}, {46,"."}}, {{52, "4"}, {50, "2"}},
{{53, "5"}, {50, "2"}}} *)

sorted = Flatten[Position[orig, #] &/@ Sort@orig]

(* {2, 3, 4, 5, 9, 6, 10, 7, 1, 8} *)

fileNames[[sorted]]

(* {"F:\\mathematica\\liu3\\1.jpg", "F:\\mathematica\\liu3\\2.jpg",
"F:\\mathematica\\liu3\\3.jpg", "F:\\mathematica\\liu3\\4.jpg",
"F:\\mathematica\\liu3\\42.jpg", "F:\\mathematica\\liu3\\5.jpg",
"F:\\mathematica\\liu3\\52.jpg", "F:\\mathematica\\liu3\\6.jpg",
"F:\\mathematica\\liu3\\62.jpg", "F:\\mathematica\\liu3\\7.jpg"} *)