Pandas - Writing an excel file containing unicode - IllegalCharacterError

Use this to remove any error that you might be getting. You can save to excel post this.

df = df.applymap(lambda x: x.encode('unicode_escape').
                 decode('utf-8') if isinstance(x, str) else x)

The same problem happened to me. I solved it as follows:

First, install python package xlsxwriter:

pip install xlsxwriter

Second, replace the default engine 'openpyxl' with 'xlsxwriter':

df.to_excel("test.xlsx", engine='xlsxwriter')

Not a Unicode issue as such... \x16 (or in Unicode strings \u0016 refers to the same character) is ASCII control code 22 (SYN). Pandas says it's invalid to have control codes (other than tab and newlines) in an Excel file, and though I don't know much about Excel files it would certainly be impossible to include them in an XML 1.0 file, which is what's inside a xlsx.

So most likely there is no way to include arbitrary character sequences (with control codes) in an Excel. You should filter them out before writing, or if you really need to preserve the original data use some form of ad hoc encoding recognised only by your application.