Pandas read csv file with float values results in weird rounding and decimal digits

Pandas uses a dedicated dec 2 bin converter that compromises accuracy in preference to speed.

Passing float_precision='round_trip' to read_csv fixes this.

Check out this page for more detail on this.

After processing your data, if you want to save it back in a csv file, you can pass
float_format = "%.nf" to the corresponding method.

A full exemple:

import pandas as pd

df_in  = pd.read_csv(source_file, float_precision='round_trip')
df_out = ... # some processing of df_in
df_out.to_csv(target_file, float_format="%.3f") # for 3 decimal places

I realise this is an old question, but maybe this will help someone else:

I had a similar problem, but couldn't quite use the same solution. Unfortunately the float_precision option only exists when using the C engine and not with the python engine. So if you have to use the python engine for some other reason (for example because the C engine can't deal with regex literals as deliminators), this little "trick" worked for me:

In the pd.read_csv arguments, define dtype='str' and then convert your dataframe to whatever dtype you want, e.g. df = df.astype('float64') .

Bit of a hack, but it seems to work. If anyone has any suggestions on how to solve this in a better way, let me know.