Welcome Guest! Log in


As explained in the presentation article, Stambia can work with Google Cloud Platform to perform operations on technologies such as Google BigQuery or Google Cloud Storage.

To accomplish this, you'll first have to install the dedicated Components and prepare your environment.

You can find below all the explanations and instructions to install and set up everything to get ready to work with Google Cloud Platform (GCP).

Prerequisites:
  • Stambia DI Designer S20 or higher
  • Stambia DI Runtime S20 or higher
  • Java 1.8

This article is dedicated to Stambia DI S20 and higher.

Please refer to this article if you are using Stambia DI S17, S18, S19

 

Component installation

Installation

You must install Google Cloud Platform and Google BigQuery Components to be able to work with those technologies in Stambia DI.

You can find Components download section at this location.

You can find Components installation procedure at this location.

 

Metadata Definition

When Stambia DI Google Cloud Platform Connector is installed and configured, you can start creating your first Google Cloud Platform Metadata.

This Metadata is the main Metadata that will be used to centralize the credentials that will be used in other Metadata such as Google Storage or Google BigQuery.

It represents a Google Cloud Project and the information to connect to this project.

 

Metadata creation

Create the Metadata, as usual, by selecting the technology in the Metadata Creation Wizard:

md creation

Click next, choose a name and click on finish.

 

Metadata configuration

First information to define in Metadata is the Google Cloud Project Identifier, which is the unique identifier of the project you want to work on.

Check with your administrator or lead to Google Cloud Platform's console page in your browser to find it, and then define it in Metadata.

md definition project id

 

Next step is to define credentials to connect and perform operations on the project.

For this, create an Account node under the root node. 

add account

 

And then define a "Credentials Mode" : "ApplicationDefault" or "LocalFile" or "String".

In all cases, the credential represents the JSON private key file containing the access token, which can be generate from Google Cloud Platform's by administrators.

 

Example with application default

application default
 

 

Example with local file

localFile

 

Example with string

string

 

Here is some information about these attributes:

Name

Description

ApplicationDefault

When using this mode, the credentials will try to be retrieved automatically by Google APIs.

They will first be searched from an environment variable named GOOGLE_APPLICATION_CREDENTIALS.

If the environment variable does not exist, it will search in the current Google engine.


Have a look at Google Authentication as service account article for further information.

LocalFile

Absolute file path to the JSON private key file corresponding to the account to connect with.

Note that it must be accessible by the Runtime on the file system.

String

JSon Access token string.

This corresponds to the content of the JSON private key file corresponding to the account to connect with.

To fill this attribute, open your JSON private key file, copy its content and paste it in the attribute.

 

The advantage is that, instead of using a file path as with the other attribute, you specify the credentials directly in Metadata, so you do not have to store a local file somewhere that can be accessed by the Runtime.

The downside is that this attribute is stored as plain text inside the Metadata.

We advise to use it for development purposes, and to consider using the local file option instead for production environments.

 

Here is an example of JSON credential file:

{
  "type": "service_account",
  "project_id": "mygoogleproject",
  "private_key_id": "myprivatekey",
  "private_key": "myprivatekeyvalue",
  "client_email": "myaccountemail",
  "client_id": "myclientid",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://accounts.google.com/o/oauth2/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "mycerturl"
}

 

Demonstration Project

Demonstration projects presenting common and advanced usage of the Google Cloud Platform and Google BigQuery in Stambia DI are available.

You can import them it in your workspace and then have a look at the Mappings and Processes examples.

It is a good start to familiarize with its usage and to see how the Metadata are configured, for instance.

 

From Stambia DI 2020 and higher, the sample projects are shipped withing the Components, and can be imported directly from the Designer as explained in Components getting started guide.

 

 

You have no rights to post comments

Articles

Suggest a new Article!