When does BigQuery flush the streaming output buffer

Answer is "it depends" and mostly based on size of data you stream to buffer - but it also based on algorithmic tuning on BQ side. As of now - there is no definite time you can somehow calculate before data will flush. And there is no mechanism to invoke flush of buffer manually.


So apparently BigQuery now allows update on older partitions of partitioned tables with streaming buffer now. But not on the streaming buffer itself.

For example :

update 
  `dataset.table_name` 
set column = 'value' 
where _PARTITIONTIME = '2018-05-01' 

Works beautifully.

But

update 
  `dataset.table_name` 
set column = 'value' 
where _PARTITIONTIME is null 

Doesn't work and fails with the below error:

UPDATE or DELETE statement over table dataset.table_name would affect rows in the streaming buffer, which is not supported