Spark: what options can be passed with DataFrame.saveAsTable or DataFrameWriter.options?

The reason you don't see options documented anywhere is that they are format-specific and developers can keep creating custom write formats with a new set of options.

However, for few supported formats I have listed the options as mentioned in the spark code itself:

  • CSVOptions
  • JDBCOptions
  • JSONOptions
  • ParquetOptions
  • TextOptions
  • OrcOptions
  • AvroOptions

Take a look at https://github.com/delta-io/delta/blob/master/src/main/scala/org/apache/spark/sql/delta/DeltaOptions.scala the class "DeltaOptions'

Currently, supported options are:

  • replaceWhere
  • mergeSchema
  • overwriteSchema
  • maxFilesPerTrigger
  • excludeRegex
  • ignoreFileDeletion
  • ignoreChanges
  • ignoreDeletes
  • optimizeWrite
  • dataChange
  • queryName
  • checkpointLocation
  • path
  • timestampAsOf
  • versionAsOf