How do I use filesystem functions in PHP, using UTF-8 strings?
PHP 7.1 supports UTF-8 filenames on Windows disregarding the OEM codepage.
Just urlencode
the string desired as a filename. All characters returned from urlencode
are valid in filenames (NTFS/HFS/UNIX), then you can just urldecode
the filenames back to UTF-8 (or whatever encoding they were in).
Caveats (all apply to the solutions below as well):
- After url-encoding, the filename must be less that 255 characters (probably bytes).
- UTF-8 has multiple representations for many characters (using combining characters). If you don't normalize your UTF-8, you may have trouble searching with
glob
or reopening an individual file. - You can't rely on
scandir
or similar functions for alpha-sorting. You musturldecode
the filenames then use a sorting algorithm aware of UTF-8 (and collations).
Worse Solutions
The following are less attractive solutions, more complicated and with more caveats.
On Windows, the PHP filesystem wrapper expects and returns ISO-8859-1 strings for file/directory names. This gives you two choices:
Use UTF-8 freely in your filenames, but understand that non-ASCII characters will appear incorrect outside PHP. A non-ASCII UTF-8 char will be stored as multiple single ISO-8859-1 characters. E.g.
ó
will be appear asó
in Windows Explorer.Limit your file/directory names to characters representable in ISO-8859-1. In practice, you'll pass your UTF-8 strings through
utf8_decode
before using them in filesystem functions, and pass the entriesscandir
gives you throughutf8_encode
to get the original filenames in UTF-8.
Caveats galore!
- If any byte passed to a filesystem function matches an invalid Windows filesystem character in ISO-8859-1, you're out of luck.
- Windows may use an encoding other than ISO-8859-1 in non-English locales. I'd guess it will usually be one of ISO-8859-#, but this means you'll need to use
mb_convert_encoding
instead ofutf8_decode
.
This nightmare is why you should probably just transliterate to create filenames.
The problem is that Windows uses utf-16 for filesystem strings, whereas Linux and others use different character sets, but often utf-8. You provided a utf-8 string, but this is interpreted as another 8-bit character set encoding in Windows, maybe Latin-1, and then the non-ascii character, which is encoded with 2 bytes in utf-8, is handled as if it was 2 characters in Windows.
A normal solution is to keep your source code 100% in ascii, and to have strings somewhere else.
Under Unix and Linux (and possibly under OS X too), the current file system encoding is given by the LC_CTYPE
locale parameter (see function setlocale()
). For example, it may evaluate to something like en_US.UTF-8
that means the encoding is UTF-8. Then file names and their paths can be created with fopen()
or retrieved by dir()
with this encoding.
Under Windows, PHP operates as a "non-Unicode aware program", then file names are converted back and forth from the UTF-16 used by the file system (Windows 2000 and later) to the selected "code page". The control panel "Regional and Language Options", tab panel "Formats" sets the code page retrieved by the LC_CTYPE
option, while the "Administrative -> Language for non-Unicode Programs" sets the translation code page for file names. In western countries the LC_CTYPE
parameter evaluates to something like language_country.1252
where 1252 is the code page, also known as "Windows-1252 encoding" which is similar (but not exactly equal) to ISO-8859-1. In Japan the 932 code page is usually set instead, and so on for other countries. Under PHP you may create files whose name can be expressed with the current code page. Vice-versa, file names and paths retrieved from the file system are converted from UTF-16 to bytes using the "best-fit" current code page.
This mapping is approximated, so some characters might be mangled in an unpredictable way. For example, Caffé Brillì.txt
would be returned by dir()
as the PHP string Caff\xE9 Brill\xEC.txt
as expected if the current code page is 1252, while it would return the approximate Caffe Brilli.txt
on a Japanese system because accented vowels are missing from the 932 code page and then replaced with their "best-fit" non-accented vowels. Characters that cannot be translated at all are retrieved as ?
(question mark). In general, under Windows there is no safe way to detect such artifacts.
More details are available in my reply to the PHP bug no. 47096.