CSV in Python adding an extra carriage return, on Windows
Python 3:
The official csv
documentation recommends open
ing the file with newline=''
on all platforms to disable universal newlines translation:
with open('output.csv', 'w', newline='', encoding='utf-8') as f:
writer = csv.writer(f)
...
The CSV writer terminates each line with the lineterminator
of the dialect, which is '\r\n'
for the default excel
dialect on all platforms because that's what RFC 4180 recommends.
Python 2:
On Windows, always open your files in binary mode ("rb"
or "wb"
), before passing them to csv.reader
or csv.writer
.
Although the file is a text file, CSV is regarded a binary format by the libraries involved, with \r\n
separating records. If that separator is written in text mode, the Python runtime replaces the \n
with \r\n
, hence the \r\r\n
observed in the file.
See this previous answer.
While @john-machin gives a good answer, it's not always the best approach. For example, it doesn't work on Python 3 unless you encode all of your inputs to the CSV writer. Also, it doesn't address the issue if the script wants to use sys.stdout as the stream.
I suggest instead setting the 'lineterminator' attribute when creating the writer:
import csv
import sys
doc = csv.writer(sys.stdout, lineterminator='\n')
doc.writerow('abc')
doc.writerow(range(3))
That example will work on Python 2 and Python 3 and won't produce the unwanted newline characters. Note, however, that it may produce undesirable newlines (omitting the LF character on Unix operating systems).
In most cases, however, I believe that behavior is preferable and more natural than treating all CSV as a binary format. I provide this answer as an alternative for your consideration.