As explained in the presentation article, Stambia can work with Google Cloud Platform to perform operations on technologies such as Google BigQuery or Google Cloud Storage.
To accomplish this, you'll first have to install the dedicated Components and prepare your environment.
You can find below all the explanations and instructions to install and set up everything to get ready to work with Google Cloud Platform (GCP).
Prerequisites:
- Stambia DI Designer S20 or higher
- Stambia DI Runtime S20 or higher
- Java 1.8
This article is dedicated to Stambia DI S20 and higher.
Please refer to this article if you are using Stambia DI S17, S18, S19
Component installation
Installation
You must install Google Cloud Platform and Google BigQuery Components to be able to work with those technologies in Stambia DI.
You can find Components download section at this location.
You can find Components installation procedure at this location.
Metadata Definition
When Stambia DI Google Cloud Platform Connector is installed and configured, you can start creating your first Google Cloud Platform Metadata.
This Metadata is the main Metadata that will be used to centralize the credentials that will be used in other Metadata such as Google Storage or Google BigQuery.
It represents a Google Cloud Project and the information to connect to this project.
Metadata creation
Create the Metadata, as usual, by selecting the technology in the Metadata Creation Wizard:
Click next, choose a name and click on finish.
Metadata configuration
First information to define in Metadata is the Google Cloud Project Identifier, which is the unique identifier of the project you want to work on.
Check with your administrator or lead to Google Cloud Platform's console page in your browser to find it, and then define it in Metadata.
Next step is to define credentials to connect and perform operations on the project.
For this, create an Account node under the root node.
And then define a "Credentials Mode" : "ApplicationDefault" or "LocalFile" or "String".
In all cases, the credential represents the JSON private key file containing the access token, which can be generate from Google Cloud Platform's by administrators.
Example with application default
Example with local file
Example with string
Here is some information about these attributes:
| Name |
Description |
| ApplicationDefault |
When using this mode, the credentials will try to be retrieved automatically by Google APIs. They will first be searched from an environment variable named GOOGLE_APPLICATION_CREDENTIALS. If the environment variable does not exist, it will search in the current Google engine.
|
| LocalFile |
Absolute file path to the JSON private key file corresponding to the account to connect with. Note that it must be accessible by the Runtime on the file system. |
| String |
JSon Access token string. This corresponds to the content of the JSON private key file corresponding to the account to connect with. To fill this attribute, open your JSON private key file, copy its content and paste it in the attribute.
The advantage is that, instead of using a file path as with the other attribute, you specify the credentials directly in Metadata, so you do not have to store a local file somewhere that can be accessed by the Runtime. The downside is that this attribute is stored as plain text inside the Metadata. We advise to use it for development purposes, and to consider using the local file option instead for production environments. |
Here is an example of JSON credential file:
{
"type": "service_account",
"project_id": "mygoogleproject",
"private_key_id": "myprivatekey",
"private_key": "myprivatekeyvalue",
"client_email": "myaccountemail",
"client_id": "myclientid",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "mycerturl"
}
Demonstration Project
Demonstration projects presenting common and advanced usage of the Google Cloud Platform and Google BigQuery in Stambia DI are available.
You can import them it in your workspace and then have a look at the Mappings and Processes examples.
It is a good start to familiarize with its usage and to see how the Metadata are configured, for instance.
From Stambia DI 2020 and higher, the sample projects are shipped withing the Components, and can be imported directly from the Designer as explained in Components getting started guide.