Why is an extra field necessary when creating point shapefile from csv files in Python?

When you split headerLine by the commas, the last entry "LON" contains a newline character:

print headerLine.split(",")
['DATE', 'LAT', 'LON\n']

This is why adding a new column after LON causes the script to work, since the newline character is applied to the TEST field.

Try using STRIP to remove the newline character:

print headerLine.strip().split(",")
['DATE', 'LAT', 'LON']

Here is an alternative workflow using geoprocessing tools that may be easier:

  1. Use Make XY Event Layer (Data Management) to create a temporary point layer from your CSV file

  2. Use Copy Features (Data Management) or Feature Class To Feature Class (Conversion) to make a permanent feature class or shapefile from the temporary point layer.