fixed

Template - INTEGRATION BigQuery - Reject Recycling is unexpectedly ignored when working with partitioned tables.

added

Template - Load GCS File to BigQuery - support file_name property field

fixed

Template - Integration BigQuery - add a cast when selecting data from source table to avoid datatype issues

fixed

Template - Integration BigQuery - some actions are unexpectedly not executed when both "Target Partition Filter" and "Target Partition Pruning" parameters are enabled

added

Template - Integration BigQuery - when deletion strategy is set to "key", create the preintegration table systematically for performances purposes

fixed

Template - Integration BigQuery - null values in columns being part of update or delete keys were not taken into account

fixed

Template - Integration BigQuery - null values were not taken into account on the action "Recycle of previous rejected data"

fixed

Template - LOAD GCS File to BigQuery - when loading data into BigQuery the templates is not using the correct work schema

fixed

BigQuery - table and column names were unexpectedly truncated to 30 characters

fixed

Template - Integration BigQuery - Flag of rows to update step should also be executed in insert only mode

added

TOOL - BigQuery Table Operations - add "truncate target table" parameter

added

TOOL - BigQuery Table Operation - use SQL syntax for Create table statement instead of scripting

added

Template - INTEGRATION BigQuery - remove option 'Useless Update Detection Method' which is not used anymore

added

Template - INTEGRATION BigQuery - When using a merge, a comparison is made on 'Non target' fields to minimize and optimize the number of updated records

added

Template - INTEGRATION BigQuery - Change partition boundaries calculation method so it executes once in a Mapping

added

Template - INTEGRATION BigQuery - New parameters "Enable Target Partition Pruning" and "Target Partition Filter"

added

Template - INTEGRATION BigQuery - New parameter "Deletion Strategy"

added

Template - INTEGRATION BigQuery - New parameter "Truncate Target Table"

added

Template - INTEGRATION BigQuery - Expressions executed on target where not supported with merge incremental method

added

Template - INTEGRATION BigQuery - Create as select is now used to create integration table

fixed

Template - INTEGRATION BigQuery - Merge did not work with doUpdate only mode

fixed

Template - INTEGRATION BigQuery - Fix insertion of columns that are supposed to be evaluated on target when being in append mode

added

Addition of two new data types - BIGNUMERIC and BIGDECIMAL

fixed

Google Cloud Storage and Google BigQuery wizards - When launching the wizard on an existing Metadata, the first step of the wizard did not automatically select the account defined in Metadata

fixed

Template - INTEGRATION BigQuery - handle nullity in update key when performing updates or comparison with target table

added

Template - Reject BigQuery - when using "Drop Reject Table" parameter, the drop and re-creation of the reject table is now performed in one query using REPLACE or IF NOT EXISTS clauses for performances and optimizations purposes

fixed

Template - Integration BigQuery - sometimes the preintegration table is unexpectedly created and populated even if it is not used later in the Template

fixed

Template - Reject BigQuery - when both "Drop Reject Tables" and "Delete Previous Rejects" parameters are enabled at the same time, the delete of previous rejects should not be performed because it is not necessary, as the table is recreated empty

fixed

Template - Integration BigQuery - incorrect query might be generated in some situations when using deleteInsert integration strategy

added

EMF compare utility - Component has been updated to support EMF Compare comparison utility

fixed

Google BigQuery Templates - when loading data into BigQuery, some Templates were not using the correct working schema

fixed

Google BigQuery - When loading DATE (or TIME) type from Bigquery into some databases (such as PostgreSQL) an error such as the following could unexpectedly be thrown "l1_c_time is of type time without time zone but expression is of type character varying"

fixed

Google BigQuery - Randomly, the "EngineExceptionI: A class was not found in the engine" error was displayed even when the Module was complete, because there was an unexpected jar in the Module which caused conflicts, and which has now been removed

fixed

Template - "INTEGRATION Bigquery" and "Load Rdbms to BigQuery" - SQL_STAT_INSERT and SQL_STAT_UPDATE statistics were not computed and not displayed

fixed

Template - Load Rdbms to Bigquery - an error such as "No such field: L1" was thrown when there was multiple time this Load Template for the same target datastore (parallelism was not handled properly)

added

Template - LOAD GCS File to BigQuery - Add support for file_name property field

fixed

Template - Load Rdbms to Bigquery - CDC_CHANGE_TYPE column is unexpectedly not loaded

added

Metadata Variables - Added support of Metadata Variable in BigQuery

added

Template - INTEGRATION BigQuery (Hierarchical) BETA - New Template to write into repeated / nested fields

added

The isRepeated attribute was not reversed in the metadata, to use repeated fields.

fixed

BigQuery Metadata - Java Type used for NUMERIC columns was incorrect

fixed

Templates - "LOAD GCS File to BigQuery" and "LOAD GCS JSON to BigQuery" Templates were throwing an error when using credentials string - WrappedException: Wrapped java.lang.NumberFormatException: For input string: "{"

fixed

Templates - "Load GCS File to BigQuery" and "Load GCS Json to BigQuery" Templates - an exception was thrown when using credentials "string" authentication method (Null Pointer in Method Invocation BSF info)

fixed

Template - INTEGRATION BigQuery - SQL_STAT_INSERT and SQL_STAT_UPDATE statistics were not displayed

fixed

When reversing a BigQuery table with multiple records some of the records (not all of them) was reversed on the wrong level.

added

Ability to define partitions in Metadata, and to optimize read requests in Mappings based on them

added

BigQuery - Support using constant values in GROUP BY clauses by automatically using aliases

added

Support a new authentication mode named "applicationDefault" which searches for authentication credentials in the current environment automatically through default Google's mechanism

added

Templates updated - New Parameters 'Unlock Cdc Table' and 'Lock Cdc Table' to configure the behaviour of CDC tables locking

added

Templates updated - New parameter 'Cdc Subscriber' on Templates on which it was not handled yet

added

Templates updated - support having CDC sources on Templates which were not supporting it (such as staging templates)

fixed

Templates updated - The 'Cdc Subscriber' parameter was ignored in some Templates when querying the source data

fixed

Templates updated - The 'Cdc Subscriber' parameter was ignored in some Templates on Lock / Unlock CDC steps

added

Google BigQuery Component - update module definition to propose more accurate driver and urls