Welcome Guest! Log in


In this article

You can find below the major changes and improvements of Stambia DI Designer 2020 (S20.0.x) version.

This article only highlights the most important changes.

If you need further information, please consult the full changelog and/or reference documentation.

 

Prerequisites

Java version

Java required version has changed in this major version.

Java 8 is now the minimum required to use this version.

Higher versions such as Java 11 are also supported now.

 

Migration

Stambia DI 2020 (S20) major version changes some important part of the software which require your attention.

There is a dedicated migration article that you must read if you want to upgrade from previous versions.

You can find this article here: migration guide

 

Analytics

If you are using Stambia DI Production Analytics, note that the following versions are required to work with Stambia DI Stambia DI Designer 2020 (S20.0.x) and Stambia DI Runtime 2020 (S20.0.x).

  • Stambia DI Production Analytics 3.1.1 or higher
  • Stambia DI Production Analytics 2.3.3 or higher

 

What's new on Stambia DI Designer 2020 (S20.0.0)

Packaging / Platform improvements

Eclipse Platform update

 

As you may know, Stambia DI Designer is based on the Eclipse platform for its graphical interface.

The Eclipse version from which Stambia DI Designer is based has been updated to a recent version to benefit of all the performances, improvements, and fixes which has been done since the previous one which we were using.

This offers the ability to work with Java versions higher than Java 8 such as Java 11, 12, 13, ... and therefore also changes the minimum version required to use Stambia DI Designer, which is now Java 8.

This also offers performances and usability improvements when using Stambia DI Designer on Mac OS which had some graphical glitches and issues on previous versions.

 

Components packaging

 

We revised the packaging of Components (previously called Connectors.)

They now include everything related to the corresponding technology.

Installing a Component will now install automatically everything required to work with this technology:

  • Technology and Product files
  • Templates
  • Sample Projects
  • Modules
  • Documentation

 

You want to install Elasticsearch Component to use Elasticsearch in Stambia DI?

Through a few clicks, you will install everything required from one archive file, and you will be ready to go.

On previous versions you had to install all those items separately: they are now installed all at once through the new install procedure.

For instance, Templates are now installed automatically in Stambia DI Designer's internal resources, you don't need to import them anymore!

 

Installation of a component in Stambia DI 2020 (S20.0.0), from "Install New Software" menu

component installation 01

 

Then simply select Component's archive and install it

component installation 02

 

Everything is now installed, including all the previous indicated items, such as Templates and sample projects to start playing with examples right away!

You can find more information on the migration guide if you are coming from a previous version.

You can also find more information about this in the new Component installation procedure.

Do not hesitate to take a look at those articles, there are important differences about installation procedure which we performed to simplify it!

 

Additional notes about Components

The main idea behind this new feature is to facilitate installation and update as you will now install everything at once for a given Component.

One of the major points to have in mind is that you don't have to import anymore any Templates in your workspace.

They are included in Components, including standard and generic Templates.

You can still of course modify them if needed by importing them from Stambia DI Designer's internal resources as explained in getting started article.

 

Welcome page / samples projects

 

Stambia DI Designer's Welcome page has been revised!

welcome2

The new welcome page now displays and gives access to:

  • Overview: access to documentation
  • Tutorials: access to tutorials
  • Samples: access to available sample projects
  • What's New: access to an overview of the new features

 

One of the most interesting point of this new welcome page is the access to sample projects.

This is a brand-new feature which allows to import sample projects containing examples about various subjects.

Those sample projects are provided both from default installation and Components installation.

You have a default list of sample projects which is filled automatically with additional ones when installing additional Components.

 

Example of sample projects list:

sample projects

To open the Welcome Page again after it has been closed, go to Help > Welcome menu

 

Modules

 

We introduced a new feature and mechanism in this version: Modules.

What is a Module?

When you need to communicate with a technology such as SQL Databases, NoSQL Databases, Cloud Platforms, and more... you will now need to use what we call a Module.

What is a Module? A Module is a set of all what is necessary to communicate with a technology.

Which means in the Java world all the libraries files which are required for the communication with the given technology, such as JDBC drivers, API libraries, libraries dependencies, ...

You will manage all of those through Modules.

Modules are uniquely identified through their name and can be reused in multiple Metadata.

 

When should I use a Module?

Using a Module to communicate with a technology is mandatory for most of them, as most require some libraries for communication.

The Module to be used must be defined most of the time directly in your Metadata.

For instance, in your PostgreSQL database Metadata, you’ll need to select a Module for communicating with PostgreSQL.

The Designer will help you to select and create Modules when necessary through dedicated wizards and helpers.

 

Here is a quick overview of the Module Wizard

modules wizard overview 2

 

Additional notes about Modules

The main idea behind this new feature is to:

  • Facilitate installation as you will now be guided to install additional libraries such as JDBC Drivers
  • Allow to compartmentalize libraries to be able to have different version of libraries (such as different version of a JDBC Driver in the same installation)
  • No more confusion about what to put in Stambia DI Runtime lib folder, now everything is managed through Modules.
Refer to "Getting started with Modules" for further information.

 

Metadata improvements

Metadata creation filter

 

When you create a Metadata, there is a new option to only see available technology in current installation.

This new option is automatically enabled when creating a new Metadata, and you can uncheck it to see other technologies that are not currently installed.

As a reminder, this usually means that you have not installed the corresponding Component in you Stambia DI Designer.

Some Components are shipped on default installation and others must be installed afterwards.

 

Metadata modele

 

Metadata editor filter

 

On Metadata editor, you can now filter what type of nodes should be displayed.

For this simply click on the filter icon which appears on the left of the text filter box.

 

Everything is displayed when opening a Metadata

filtre01

 

When unchecking a node, it will be filtered from the editor

filtre02

This helps to display only specific nodes when you are searching for a given type of node for instance.

 

Metadata search

 

New utility to search for Metadata in current Workspace.

You can search by Metadata category and by Metadata name.

You can, with this new search tool, find in a few clicks all Elasticsearch Metadata of your workspace, for instance!

 

Accessing the search tool

metadata search tool

 

Searching by Metadata category

metadata search tool by category

 

Searching by Metadata name

metadata search tool by name

 

You can then double click on a found Metadata to open it, or click on the link to find it back in Project Explorer

metadata search tool usage

Metadata Link evaluation

 

Metadata Link evaluation has been improved to support evaluating from different manner.

It is now possible to retrieve values from Metadata Link.

It is supported only on Metadata Link from a Generic Element Metadata attribute for now.

metadata link native evaluation

 

In this example, we create an attribute to define the path of an XML File

 

Configuration attributes overview

 

Metadata Configuration user interface has been improved to provide an overview of the value that will be overriden, and the ability to remove the overriden value in one click.

Now, when you are in Metadata editor and you are modifying a Configuration, the attributes are greyed out with an hyperlink to override them:

metadata configuration

 

Mapping improvements

Mapping stages can be created from a menu

 

In a Mapping, when you drag and drop a set of columns from a datastore, you have now a menu which allows you use those selected columns to create a filter, or to create a stage.

This is a new alternative way to create stages on the fly.

You have the possibility to define your favorite staging area Metadata nodes in preferences, which will be displayed in this menu.

 

Creating a Mapping stage through context menu

mapping stage

 

If you have defined some preferred staging area nodes in your preferences, they will appear in the menu

mapping stage preferred

 

Under Window / Preferences / Stambia / Mapping menu, you can define your preferred staging area nodes that will then appear in the context menu in Mapping.

staging area preferences

 

Mapping update key provider

 

On a Mapping, you can now choose from where the updated key of a datastore should be computed.

On previous versions, the update key of a datastore was initialized when adding the datastore in Mapping and then handled directly in the Mapping by the user if needed to perform some changes.

The updated key was therefore a manual definition in Mapping.

 

Changes in this version

You can now choose which provider to use for the update key.

You have the choice between "Manual Definition" which is the default value, and which corresponds to the actual behavior of using what is defined in Mapping.

And you have now also the choice to specify that the update key should be provided from a defined Metadata Key.

 

mapping update key provider

 

When using a Metadata key provider, such as a primary key, the update key of the datastore will reflect the primary key of the Metadata.

If the given key is updated in Metadata, the Mapping will reflect the changes automatically.

When a datastore is using a specific update key provider, the key is colored in purple, as you can see in previous screenshot, and the user cannot change it manually.

 

Mapping Extra Nodes

 

FILE, JSON, and XML Metadata have the ability to define additional fields to retrieve for instance while reading files the file name, path, the node iteration, and more...

Those fields used to be defined directly in Metadata on previous versions.

When you needed those extra fields, it was required to add them manually in your Metadata before being able to use them in Mapping.

From now, you'll be able to add them dynamically in your Mapping without having to define them in Metadata.

 

Example on File

extra nodes example on file

 

Example on XML

extra nodes example on xml

 

Depending on the node chosen, Stambia directly offers you the list of available fields applicable to this type of node.

Note that you can of course still define those nodes in your Metadata if you prefer, it can be more efficient and practical depending on if you are using them a lot for instance, in such case it can be more practical to define them directly in Metadata.

Limitation: with JSON and XML, the Extra Nodes added dynamically in Mapping can be used only on expressions executed on "Source" execution location. Extra fields are not yet supported on staging area execution location. If you want to use them on such expressions, you can define them in Metadata as before.

 

Mapping Diagrams included in model

 

Mappings were originally physically separated in two files.

The Mapping file with extension ".map" which contains all the Mapping rules and definition.

The Mapping diagram file extension '.map_diagram" which includes the layout of the Mapping, such as how the user has positioned the different boxes graphically and the user notes.

 

Changes in this version

From now everything is stored inside the Mapping file itself.

There is now only one file with extension ".map" which contains everything.

This change has been done to ease the management of versioning systems to have only one file to handle for a given Mapping, so that the Mapping and its layout are always synchronized and that there is only one file to commit and manage.

 

mapping diagram included

 

You have nothing particular to do about this point.

On existing Mappings coming from previous versions which still have a separated diagram, the diagram will be included inside the Mapping automatically the next time the Mapping is saved.

The folder "indy.diagram/map" will therefore gradually be emptied as you re-save your Mappings on the new version.

 

Process improvements

Process Palette Tools

 

All applicable Templates and Tools such as Replicators are now displayed directly in Process Palette, under the 'Tools' section.

When installing new Components in Stambia DI Designer, the corresponding tools, if any, will appear in this list automatically.

Moreover, when using them, you will now have Metadata Link Helpers to know when they need Metadata!

palette

 

As an example, you can now use the Replicator in a few clicks

process palette

 

Process Diagrams included in model

 

Process were originally physically separated in two files.

The Process file with extension ".proc" which contains all the Process rules and operations.

The Process diagram file extension '.proc_diagram" which includes the layout of the Process, such as how the user has positioned the different boxes graphically and the user notes.

 

Changes in this version

From now everything is stored inside the Process file itself.

There is now only one file with extension ".proc" which contains everything.

This change has been done to ease the management of versioning systems to have only one file to handle for a given Process, so that the Process and its layout are always synchronized and that there is only one file to commit and manage.

process diagram included

You have nothing particular to do about this point.

On existing Processes coming from previous versions which still have a separated diagram, the diagram will be included inside the Process automatically the next time the Process is saved.

The folder "indy.diagram/proc" will therefore gradually be emptied as you re-save your Processes on the new version.

 

Miscellaneous improvements

Syntax coloring and helpers for Metadata Links and Eclipse String Substitution

 

When you are in Metadata, Mapping, or Process editor, Metadata Link and Eclipse String Substitution syntaxes are now colored:

metadata link coloring

 

You want to know from where this Metadata Link is coming in your Project Explorer?

You can now also CTRL + click on it to find it back.

 

Example in a Metadata

metadata link shortcut in metadata

 

Example in Mapping

metadata link shortcut in mapping

 

Example in Process

metadata link shortcut in process

 

String substitution

 

Support Eclipse and local workspace environment variables in Metadata.

You can for instance dynamically use the path of the current Project, or the path of the current Stambia DI Designer installation folder.

This can be useful most notably to prepare autonomous Projects containing example files, for instance.

 

Example using the current project location

string substitution

For further information and to learn how to define your own varables, refer to reference documentation

 

Designer menus revised

 

We revised some of the Designer main top menus to be clearer and more convenient.

The "Help > About Stambia" and "Help > About Stambia Designer" menus have been merged to have only one menu where to find version information.

 

help stambia designer

 

License Manager is now in a new "Utilities" menu which will be a menu containing Stambia additional utilities.

 

help license manager

 

Local Runtime Java Virtual Machine

 

When starting the local Designer's Runtime directly from the Designer, the started Runtime will now use the same Java Virtual Machine (JVM) as the Designer.

This allows to make sure the execution environment is the same when you are using Designer's Local Runtime.

Note that this applies only when starting the Runtime from the Designer from the Runtime View / Environment button.

If you start the Runtime aside of the Designer using the startup scripts, it will use the standard Runtime's startup behavior (Use of STAMBIA_JAVA_HOME environment variable if defined, etc...).

 

New Process Action "Execute FTP Command"

 

The existing "Command With FTP" Process action has been moved to deprecated category in Process Palette.

This Process Action has been replaced with a new "Execute FTP Command" Process Action which supports commands aliases.

  • This new action supports both RAW FTP Commands and most commonly used aliases such as "ls", "dir", and more...
  • This new action publishes new variables such as "STORED_FILES_NUMBER" to inform about the operations performed.

 

You can find the complete list of supported aliases and published variables in the reference documentation of this Process Action.

Refer to this article for further information. 

 

RMIS Configuration

Designer does not anymore include the embedded certificate used for communicating with Runtimes through RMIS.

This certificate has been removed as from Stambia DI Runtime 2020 you can customize it to use your own.

Therefore, when you want to connect to Runtimes through RMIS, you may now have to configure your Designer to specify the public certificate to be used.

 

Refer to this article to learn how to configure the clients such as Designer, Analytics, or command line scripts to communicate with a Runtime which endpoints are secured with TLS.

It also proposes as a separate download the previously embedded public certificate which must be configured in the Designer if you want to connect from the Designer through RMIS on previous Runtimes which were using this certificate (Runtime S17.x.x).

 

Refer to this article to learn how to secure the various endpoints with TLS on Stambia DI Runtime 2020.

 

You have no rights to post comments

Articles

Suggest a new Article!