Data between quotes and field separator

  • API V2 https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load.quote

  • bq command --quote: Quote character to use to enclose records. Default is ". To indicate no quote character at all, use an empty string.


From the CSV RFC4180 page: "If double-quotes are used to enclose fields, then a double-quote appearing inside a field must be escaped by preceding it with another double quote."

You probably want to do this:

This | is | test only | to check quotes
second | line | "with quotes" | no text
third line | with | " ""start quote"" and " | a word after quotes

More about our CSV input format here.


Using --quote worked perfectly.

bq load 
--source_format CSV --quote "" 
--field_delimiter \t 
--max_bad_records 10 
-E UTF-8   
destination table
Source files

Try this as an alternative:

  • Load the MySQL backup files into a Cloud SQL instance.
  • Read the data in BigQuery straight out of MySQL.

Longer how-to:

  • https://medium.com/google-cloud/loading-mysql-backup-files-into-bigquery-straight-from-cloud-sql-d40a98281229

Tags:

Google Bigquery

Recent Posts