In this article
You can find below the major changes and improvements of Stambia DI Designer 2020 (S20.0.x) version.
This article only highlights the most important changes.
Java required version has changed in this major version.
Java 8 is now the minimum required to use this version.
Higher versions such as Java 11 are also supported now.
Stambia DI 2020 (S20) major version changes some important part of the software which require your attention.
There is a dedicated migration article that you must read if you want to upgrade from previous versions.
You can find this article here: migration guide
If you are using Stambia DI Production Analytics, note that the following versions are required to work with Stambia DI Stambia DI Designer 2020 (S20.0.x) and Stambia DI Runtime 2020 (S20.0.x).
- Stambia DI Production Analytics 3.1.1 or higher
- Stambia DI Production Analytics 2.3.3 or higher
Packaging / Platform improvements
Eclipse Platform update
As you may know, Stambia DI Designer is based on the Eclipse platform for its graphical interface.
The Eclipse version from which Stambia DI Designer is based has been updated to a recent version to benefit of all the performances, improvements, and fixes which has been done since the previous one which we were using.
This offers the ability to work with Java versions higher than Java 8 such as Java 11, 12, 13, ... and therefore also changes the minimum version required to use Stambia DI Designer, which is now Java 8.
This also offers performances and usability improvements when using Stambia DI Designer on Mac OS which had some graphical glitches and issues on previous versions.
We revised the packaging of Components (previously called Connectors.)
They now include everything related to the corresponding technology.
Installing a Component will now install automatically everything required to work with this technology:
- Technology and Product files
- Sample Projects
You want to install Elasticsearch Component to use Elasticsearch in Stambia DI?
Through a few clicks, you will install everything required from one archive file, and you will be ready to go.
On previous versions you had to install all those items separately: they are now installed all at once through the new install procedure.
For instance, Templates are now installed automatically in Stambia DI Designer's internal resources, you don't need to import them anymore!
Installation of a component in Stambia DI 2020 (S20.0.0), from "Install New Software" menu
Then simply select Component's archive and install it
Everything is now installed, including all the previous indicated items, such as Templates and sample projects to start playing with examples right away!
You can find more information on the migration guide if you are coming from a previous version.
You can also find more information about this in the new Component installation procedure.
Do not hesitate to take a look at those articles, there are important differences about installation procedure which we performed to simplify it!
Additional notes about Components
The main idea behind this new feature is to facilitate installation and update as you will now install everything at once for a given Component.
One of the major points to have in mind is that you don't have to import anymore any Templates in your workspace.
They are included in Components, including standard and generic Templates.
You can still of course modify them if needed by importing them from Stambia DI Designer's internal resources as explained in getting started article.
Welcome page / samples projects
Stambia DI Designer's Welcome page has been revised!
The new welcome page now displays and gives access to:
- Overview: access to documentation
- Tutorials: access to tutorials
- Samples: access to available sample projects
- What's New: access to an overview of the new features
One of the most interesting point of this new welcome page is the access to sample projects.
This is a brand-new feature which allows to import sample projects containing examples about various subjects.
Those sample projects are provided both from default installation and Components installation.
You have a default list of sample projects which is filled automatically with additional ones when installing additional Components.
Example of sample projects list:
To open the Welcome Page again after it has been closed, go to Help > Welcome menu
We introduced a new feature and mechanism in this version: Modules.
What is a Module?
When you need to communicate with a technology such as SQL Databases, NoSQL Databases, Cloud Platforms, and more... you will now need to use what we call a Module.
What is a Module? A Module is a set of all what is necessary to communicate with a technology.
Which means in the Java world all the libraries files which are required for the communication with the given technology, such as JDBC drivers, API libraries, libraries dependencies, ...
You will manage all of those through Modules.
Modules are uniquely identified through their name and can be reused in multiple Metadata.
When should I use a Module?
Using a Module to communicate with a technology is mandatory for most of them, as most require some libraries for communication.
The Module to be used must be defined most of the time directly in your Metadata.
For instance, in your PostgreSQL database Metadata, you’ll need to select a Module for communicating with PostgreSQL.
The Designer will help you to select and create Modules when necessary through dedicated wizards and helpers.
Here is a quick overview of the Module Wizard
Additional notes about Modules
The main idea behind this new feature is to:
- Facilitate installation as you will now be guided to install additional libraries such as JDBC Drivers
- Allow to compartmentalize libraries to be able to have different version of libraries (such as different version of a JDBC Driver in the same installation)
- No more confusion about what to put in Stambia DI Runtime lib folder, now everything is managed through Modules.
Metadata creation filter
When you create a Metadata, there is a new option to only see available technology in current installation.
This new option is automatically enabled when creating a new Metadata, and you can uncheck it to see other technologies that are not currently installed.
As a reminder, this usually means that you have not installed the corresponding Component in you Stambia DI Designer.
Some Components are shipped on default installation and others must be installed afterwards.
Metadata editor filter
On Metadata editor, you can now filter what type of nodes should be displayed.
For this simply click on the filter icon which appears on the left of the text filter box.
Everything is displayed when opening a Metadata
When unchecking a node, it will be filtered from the editor
This helps to display only specific nodes when you are searching for a given type of node for instance.
New utility to search for Metadata in current Workspace.
You can search by Metadata category and by Metadata name.
You can, with this new search tool, find in a few clicks all Elasticsearch Metadata of your workspace, for instance!
Accessing the search tool
Searching by Metadata category
Searching by Metadata name
You can then double click on a found Metadata to open it, or click on the link to find it back in Project Explorer
Metadata Link evaluation
Metadata Link evaluation has been improved to support evaluating from different manner.
It is now possible to retrieve values from Metadata Link.
It is supported only on Metadata Link from a Generic Element Metadata attribute for now.
In this example, we create an attribute to define the path of an XML File
Configuration attributes overview
Metadata Configuration user interface has been improved to provide an overview of the value that will be overriden, and the ability to remove the overriden value in one click.
Now, when you are in Metadata editor and you are modifying a Configuration, the attributes are greyed out with an hyperlink to override them:
Mapping stages can be created from a menu
In a Mapping, when you drag and drop a set of columns from a datastore, you have now a menu which allows you use those selected columns to create a filter, or to create a stage.
This is a new alternative way to create stages on the fly.
You have the possibility to define your favorite staging area Metadata nodes in preferences, which will be displayed in this menu.
Creating a Mapping stage through context menu
If you have defined some preferred staging area nodes in your preferences, they will appear in the menu
Under Window / Preferences / Stambia / Mapping menu, you can define your preferred staging area nodes that will then appear in the context menu in Mapping.
Mapping update key provider
On a Mapping, you can now choose from where the updated key of a datastore should be computed.
On previous versions, the update key of a datastore was initialized when adding the datastore in Mapping and then handled directly in the Mapping by the user if needed to perform some changes.
The updated key was therefore a manual definition in Mapping.
Changes in this version
You can now choose which provider to use for the update key.
You have the choice between "Manual Definition" which is the default value, and which corresponds to the actual behavior of using what is defined in Mapping.
And you have now also the choice to specify that the update key should be provided from a defined Metadata Key.
When using a Metadata key provider, such as a primary key, the update key of the datastore will reflect the primary key of the Metadata.
If the given key is updated in Metadata, the Mapping will reflect the changes automatically.
When a datastore is using a specific update key provider, the key is colored in purple, as you can see in previous screenshot, and the user cannot change it manually.
Mapping Extra Nodes
FILE, JSON, and XML Metadata have the ability to define additional fields to retrieve for instance while reading files the file name, path, the node iteration, and more...
Those fields used to be defined directly in Metadata on previous versions.
When you needed those extra fields, it was required to add them manually in your Metadata before being able to use them in Mapping.
From now, you'll be able to add them dynamically in your Mapping without having to define them in Metadata.
Example on File
Example on XML
Depending on the node chosen, Stambia directly offers you the list of available fields applicable to this type of node.
Note that you can of course still define those nodes in your Metadata if you prefer, it can be more efficient and practical depending on if you are using them a lot for instance, in such case it can be more practical to define them directly in Metadata.
Limitation: with JSON and XML, the Extra Nodes added dynamically in Mapping can be used only on expressions executed on "Source" execution location. Extra fields are not yet supported on staging area execution location. If you want to use them on such expressions, you can define them in Metadata as before.
Mapping Diagrams included in model
Mappings were originally physically separated in two files.
The Mapping file with extension ".map" which contains all the Mapping rules and definition.
The Mapping diagram file extension '.map_diagram" which includes the layout of the Mapping, such as how the user has positioned the different boxes graphically and the user notes.
Changes in this version
From now everything is stored inside the Mapping file itself.
There is now only one file with extension ".map" which contains everything.
This change has been done to ease the management of versioning systems to have only one file to handle for a given Mapping, so that the Mapping and its layout are always synchronized and that there is only one file to commit and manage.
You have nothing particular to do about this point.
On existing Mappings coming from previous versions which still have a separated diagram, the diagram will be included inside the Mapping automatically the next time the Mapping is saved.
The folder "indy.diagram/map" will therefore gradually be emptied as you re-save your Mappings on the new version.
Process Palette Tools
All applicable Templates and Tools such as Replicators are now displayed directly in Process Palette, under the 'Tools' section.
When installing new Components in Stambia DI Designer, the corresponding tools, if any, will appear in this list automatically.
Moreover, when using them, you will now have Metadata Link Helpers to know when they need Metadata!
As an example, you can now use the Replicator in a few clicks
Process Diagrams included in model
Process were originally physically separated in two files.
The Process file with extension ".proc" which contains all the Process rules and operations.
The Process diagram file extension '.proc_diagram" which includes the layout of the Process, such as how the user has positioned the different boxes graphically and the user notes.
Changes in this version
From now everything is stored inside the Process file itself.
There is now only one file with extension ".proc" which contains everything.
This change has been done to ease the management of versioning systems to have only one file to handle for a given Process, so that the Process and its layout are always synchronized and that there is only one file to commit and manage.
You have nothing particular to do about this point.
On existing Processes coming from previous versions which still have a separated diagram, the diagram will be included inside the Process automatically the next time the Process is saved.
The folder "indy.diagram/proc" will therefore gradually be emptied as you re-save your Processes on the new version.
Syntax coloring and helpers for Metadata Links and Eclipse String Substitution
When you are in Metadata, Mapping, or Process editor, Metadata Link and Eclipse String Substitution syntaxes are now colored:
You want to know from where this Metadata Link is coming in your Project Explorer?
You can now also CTRL + click on it to find it back.
Example in a Metadata
Example in Mapping
Example in Process
Support Eclipse and local workspace environment variables in Metadata.
You can for instance dynamically use the path of the current Project, or the path of the current Stambia DI Designer installation folder.
This can be useful most notably to prepare autonomous Projects containing example files, for instance.
Example using the current project location
For further information and to learn how to define your own varables, refer to reference documentation
Designer menus revised
We revised some of the Designer main top menus to be clearer and more convenient.
The "Help > About Stambia" and "Help > About Stambia Designer" menus have been merged to have only one menu where to find version information.
License Manager is now in a new "Utilities" menu which will be a menu containing Stambia additional utilities.
Local Runtime Java Virtual Machine
When starting the local Designer's Runtime directly from the Designer, the started Runtime will now use the same Java Virtual Machine (JVM) as the Designer.
This allows to make sure the execution environment is the same when you are using Designer's Local Runtime.
Note that this applies only when starting the Runtime from the Designer from the Runtime View / Environment button.
If you start the Runtime aside of the Designer using the startup scripts, it will use the standard Runtime's startup behavior (Use of STAMBIA_JAVA_HOME environment variable if defined, etc...).
New Process Action "Execute FTP Command"
The existing "Command With FTP" Process action has been moved to deprecated category in Process Palette.
This Process Action has been replaced with a new "Execute FTP Command" Process Action which supports commands aliases.
- This new action supports both RAW FTP Commands and most commonly used aliases such as "ls", "dir", and more...
- This new action publishes new variables such as "STORED_FILES_NUMBER" to inform about the operations performed.
You can find the complete list of supported aliases and published variables in the reference documentation of this Process Action.
Refer to this article for further information.
Designer does not anymore include the embedded certificate used for communicating with Runtimes through RMIS.
This certificate has been removed as from Stambia DI Runtime 2020 you can customize it to use your own.
Therefore, when you want to connect to Runtimes through RMIS, you may now have to configure your Designer to specify the public certificate to be used.
Refer to this article to learn how to configure the clients such as Designer, Analytics, or command line scripts to communicate with a Runtime which endpoints are secured with TLS.
It also proposes as a separate download the previously embedded public certificate which must be configured in the Designer if you want to connect from the Designer through RMIS on previous Runtimes which were using this certificate (Runtime S17.x.x).
Refer to this article to learn how to secure the various endpoints with TLS on Stambia DI Runtime 2020.