How to deal with characters like ":" or "?" that make invalid filenames?

These characters ? and : are not valid on a FAT32 filesystem, so if that is where you need to copy your files you will need to rename them.

From the command line you can use command-line tools such as rename (sometimes known as prename) to replace these characters with _ or even to remove them:

rename 's/[?<>\\:*|\"]/_/g'    # Change invalid characters to _
rename 's/[?<>\\:*|\"]//g'     # Remove invalid characters

I am not familiar with thunar so I do not know if there is a way to perform this substitution/replacement operation directly.

I have just found Linux copy to fat32 filesystem: invalid argument which suggests adding this into the pax command (another tool to copy files), so that you can keep your full filenames on your local disk but convert the filenames during the copy to your USB device:

pax -rw -s '/[?<>\\:*|\"]/_/gp' *.mp3 /media/usb_device

If the complete filenames are really important to you, I would suggest that you reformat the USB stick to use a Linux-native filesystem such as ext4. (There are Windows drivers available for the extN family of filesystems if that's necessary.)

It is definitely allowed to have ? in a filename.

From the POSIX standard:

Filename

A sequence of bytes consisting of 1 to {NAME_MAX} bytes used to name a file. The bytes composing the name shall not contain the <NUL> or <slash> characters. In the context of a pathname, each filename shall be followed by a <slash> or a <NUL> character; elsewhere, a filename followed by a <NUL> character forms a string (but not necessarily a character string). The filenames dot and dot-dot have special meaning. A filename is sometimes referred to as a "pathname component". See also Pathname.

So the only characters a filename may not contain is \0 (signifying an end-of-string) and / (which is a path separator). Also, the filenames . and .. are special and may not be used by files that you create.

Some characters are special to the shell and needs to be escaped or properly quoted for the shell to not treat them as globbing patterns. The question mark is one of those characters, as it usually stands for "any one character" in shell globbing patterns.

My guess is that Thunar is choosing not to handle these filenames as the characters : and ? (and <, >, ", |, \, and *) are not allowed on Windows, and it wants to force you to use filenames that are portable between Windows and Unix. I don't know the reasoning behind this, but it could have something to do with making Thunar work on filesystems shared with Windows machines.

EDIT: After the question was updated with further information, it is clear that the files will need to conform to Windows naming rules as the filesystem is indeed a Windows filesystem.

@roaima's answer shows how to rename files using the Linux rename utility. This is a another solution that will work on systems without rename, a bash function, fatmv that renames files containing illegal Windows characters by changing them to underscores (_):

function fatmv {
    name="$( basename "$1" )"
    destdir="$2"

    fat_name="$( tr ':?<>"|\\*' '_' <<<"$name" )"

    if [[ -n "$destdir" ]]; then
      fat_name="$destdir"/"$fat_name"
    fi

    echo mv -f -- "$1" "$fat_name"
}

If a second argument is used, this is treated as a destination directory the which the renamed file should be moved:

$ fatmv '<illegal>:filename?' /mount/fat_disk/
mv -f -- <illegal>:filename? /mount/fat_disk//_illegal__filename_

Test this as is, and then remove the echo to make it actually do something.

The ? is a special character that has significance in bash, You can work with it by escaping it with \

ie:

touch test\?

will make the file:

test?

and you can move it with

mv test\? /new/loction/test\?

Edit: The same for any special character, such as .

You also need it for file names that contain spaces

this is a file

would have to be moved with:

mv this\ is\ a\ file <location>