Linux Both Case Sensitive AND Case Insensitive and Always Inconvenient?

You can enable case-insensitive file name completion in bash by adding the following line to $HOME/.inputrc:

set completion-ignore-case On

Regarding the file names: This depends on the file system. On Linux file systems, there should be no problem. On other file systems, the behavior can be controlled with options to the mount command. See the manual page mount(8) for more information on the available options to mount.

NTFS¹ and VFAT are not case-sensitive, they are just case-preserving. That means if you create a file named FileName.txt, the file system will preserve the mixed case name, but you can access the file with whatever case combination of the same letters, like FILENAME.TXT, filename.txt or fileNAME.txt. This explains you cannot have two files with the same spelling with only a variation of upper/lower case in the same directory.

SMB exported file system have to implement this behavior not to confuse Windows clients.

ZFS can be configured to behave that way with the casesensitivity=mixed property.

¹ _{Technically, NTFS is case sensitive but the OSes mounting file systems of this type are almost always configured to hide this underlying feature and only preserve the case. Windows can however enable case sensitivity with modifying this register key HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\dword:ObCaseInsensitive and Linux can mount these file systems with various behaviors depending on the ignore_case and windows_names mount options.}

Is this a property of the filesystem or something else?

pa-ubuntu-11388$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 10.04.3 LTS
Release:        10.04
Codename:       lucid
pa-ubuntu-11388$ ls -l
pa-ubuntu-11388$ touch filename.txt
pa-ubuntu-11388$ ls -l
-rw-r--r-- 1 dshawley dev 0 May 31 15:17 filename.txt
pa-ubuntu-11388$ cp ./filename.txt ./FileName.txt
pa-ubuntu-11388$ ls -l
-rw-r--r-- 1 dshawley dev 0 May 31 15:17 FileName.txt
-rw-r--r-- 1 dshawley dev 0 May 31 15:17 filename.txt

It works fine for me.