2018-09-20 (GCS 1.9.7, BQ 0.13.7)
Changelog
Cloud Storage connector:
-
Add an option to provide credentials directly in Hadoop Configuration, without having to place a file on every node, or associating service accounts with GCE VMs.
fs.gs.auth.service.account.private.key.id fs.gs.auth.service.account.private.key
-
Add an option to specify max bytes rewritten per rewrite request when
fs.gs.copy.with.rewrite.enable
is set totrue
:fs.gs.rewrite.max.bytes.per.call (default: 536870912)
Even though GCS does not require this parameter for rewrite requests, rewrite requests are flaky without it.
BigQuery connector:
- POM updates for GCS connector 1.9.7.