RStudio not picking the encoding I'm telling it to use when reading a file
This problem is caused by the wrong locale being set, whether inside RStudio or command-line R:
If the problem only happens in RStudio not command-line R, go to RStudio->Preferences:General, tell us what 'Default text encoding:'is set to, click 'Change' and try Windows-1252, UTF-8 or ISO8859-1('latin1') (or else 'Ask' if you always want to be prompted). Screenshot attached at bottom. Let us know which one worked!
If the problem also happens in command-line R, do the following:
Do locale -m on your Mac and tell us whether it supports CP1252 or else ISO8859-1 ('latin1')? Dump the list of supported locales if you need to. (You might as well tell us your version of MacOS while you're at it.)
For both of those locales, try to change to that locale:
# first try Windows CP1252, although that's almost surely not supported on Mac:
Sys.setlocale("LC_ALL", "pt_PT.1252") # Make sure not to omit the `"LC_ALL",` first argument, it will fail.
Sys.setlocale("LC_ALL", "pt_PT.CP1252") # the name might need to be 'CP1252'
# next try IS08859-1(/'latin1'), this works for me:
Sys.setlocale("LC_ALL", "pt_PT.ISO8859-1")
# Try "pt_PT.UTF-8" too...
# in your program, make sure the Sys.setlocale worked, sprinkle this assertion in your code before attempting to read.csv:
stopifnot(Sys.getlocale('LC_CTYPE') == "pt_PT.ISO8859-1")
That should work.
Strictly the Sys.setlocale() command should go in your ~/.Rprofile for startup, not inside your R session or source-code.
However Sys.setlocale() can fail, so just be aware of that. Also, assert Sys.getlocale() inside your setup code early and often, as I do. (really, read.csv should figure out if the encoding it uses is compatible with the locale, and warn or error if not).
Let us know which fix worked! I'm trying to document this more generally so we can figure out the correct enhance.
- Screenshot of RStudio Preferences Change default text encoding menu:
It works fine for me.
Did you try to change/reset locale?
in my case it works with
Sys.setlocale(category = "LC_ALL", locale = "Portuguese_Portugal.1252")
d <- read.table(text=readClipboard(), header=TRUE, sep = ';')
head(d)
1 25/04/2014 22:29:30 2012 1 ELEIÇÃO MUNICIPAL 2012 PB 20419 20419 ITAPORANGA 33 13 VEREADOR 17157
2 25/04/2014 22:29:30 2012 1 ELEIÇÃO MUNICIPAL 2012 PB 20770 20770 MALTA 51 11 PREFEITO 4677
3 25/04/2014 22:29:30 2012 1 ELEIÇÃO MUNICIPAL 2012 PB 21091 21091 OLHO D'ÁGUA 32 13 VEREADOR 6653
4 25/04/2014 22:29:30 2012 1 ELEIÇÃO MUNICIPAL 2012 PB 21113 21113 OLIVEDOS 23 13 VEREADOR 3243
...
I had the same problem with Portuguese locale in r (MAC OS 10.12.3)
I've tried as per thread above and no one worked. Then I found this webpage: https://docs.moodle.org/dev/Table_of_locales
and just tried Sys.setlocale(category = "LC_ALL", locale = "pt_PT.UTF-8") and it works.