Welcome Guest! Log in
Stambia versions 2.x, 3.x, S17, S18, S19 and S20 are reaching End of Support January, 15th, 2024. Please consider upgrading to the supported Semarchy xDI versions. See Global Policy Support and the Semarchy Documentation.

The Stambia User Community is moving to Semarchy! All the applicable resources have already been moved or are currently being moved to their new location. Read more…


This article describes the principal changes of Google BigQuery Addon.

Note:

Stambia DI is a flexible and agile solution. It can be quickly adapted to your needs.

If you have any question, any feature request or any issue, do not hesitate to contact us.

 

This article is dedicated to Stambia DI S17, S18 and S19 addon.

If you are using Stambia DI 2020 (S20) or higher, please refer to this article.

 

addons.google.bigquery_1.1.0_20200309_102230

Prerequisites:
  • Google Cloud Platform Addon 1.1.0 or higher
  • Templates BigQuery 2020-03-09 or higher

 

Credentials and Project ID

A new attribute to drag and drop a Google Cloud Platform Credentials Metadata node on the BigQuery Metadata has been added, to replace the usage of 'credentialsFile' and 'projectId' properties which are now deprecated.

Note that for backward compatibility, when properties are defined they have priority on the new credentials link.

To use the new mechanism, you must remove the properties and then define the link.

 

Refer to this article for more information.

 

Temporary Bucket

A new attribute to drag and drop a Google Cloud Storage Bucket Metadata node has been added, to replace the default storage bucket attribute which is now deprecated.

Note that for backward compatibility, when the old attribute is defined, it has priority on the new storage link.

Note that old attribute has been moved under deprecated tab.

 

Refer to this article for more information.

 

Fix issue with Bigint

This version fixes an issue which was causing an SQLFeatureNotSupportedException in mapping using BigInt as source.

 

Fix issue with credentials error message

This version fixes an issue which was causing unclear and misleading error message when the credentials were the defined or incorrectly.

 

addons.google.bigquery_1.0.4.201911081030

Prerequisites:
  • Google Cloud Platform Addon 1.0.4 or higher

 

Update of Connector

Google BigQuery Connector has been updated to work with the new Google Cloud Platform Connector.

All Google APIs libraries have been updated to latest ones to benefit of the last improvements, fixes and optimizations.

They are now all shipped in the main Google Cloud Platform Connector which centralize all the dependencies.

Google BigQuery Connector now therefore requires the Google Cloud Platform Connector to be installed.

 

Fix encoding issue

Encoding issues have been fixed in this version.

All read and write operations are now properly performed using UTF-8 encoding.

 

addons.google.bigquery_1.0.3.201907191042

More Statistics are now retrieved when performing operations on Google BigQuery

Number of processed rows are now computed and retrieved when inserting data into BigQuery through the Load and Integration Templates when using the direct method.

Number of processed rows are now computed and retrieved on the step which is streaming data into Google Storage, and on the one which is extracting source data into a local file.

 

Performance improvement on source data export

Performances have been improved on steps extracting source data into a local file before sending it to Google Storage and finally load it into Google BigQuery.

Extraction of source data should now be significantly faster when working large sets of data.

 

Fixed issue with timestamp datatype

This version fixes an issue which was causing unexpected errors when loading value from or to double datatype.

 

Fixed issue with source smallint datatype

This version fixes an issue which was causing unexpected errors when loading value from smallint datatypes to Google BigQuery.

 

addons.google.bigquery_1.0.2.201906141610

Fixed issue with null values

This version fixes an issue which was causing unexpected errors when loading "null" values.

 

Fixed issue with double datatype

This version fixes an issue which was causing unexpected errors when loading value from or to double datatype.

 

addons.google.bigquery_1.0.1.201809191348

This version changes the mode used to perform SQL queries from Legacy SQL to Standard SQL, which is now the default mode used by the connector.

 

The Google BigQuery Templates have also been updated to support this change.

Please update your Templates to templates.google.bigquery.2018-09-19 or higher.

 

Articles

Suggest a new Article!