How to order my dataframe lexicographicaly

One option is to use mixedorder() from the gtools package.

library(gtools)
a[mixedorder(a$c),]
#   a  b  c
# 1 1  1  1
# 2 2  2  2
# 7 7  4  3
# 3 3  3 10
# 4 4 10 11
# 5 5 12  X
# 6 6 21  Y

Sticking in base you could make a function yourself:

a = data.frame(a=c(1,2,3,4,5,6,7),b=c(1,2,3,10,12,21,4),c=c(1,2,10,11,"X","Y",3))

SORTER_DEVICE <- function(x) {
    c(sort(as.numeric(na.omit(gsub("[a-zA-Z]", NA, x)))),
        sort(na.omit(gsub("[0-9]", NA, x))))
}
data.frame(apply(a, 2, SORTER_DEVICE))