Welcome Guest! Log in

Replicating a directory with Replicator Rdbms

    The REPLICATOR Rdbms.tp provided with the generic templates can be used in various situations.

    This articles explains how to replicate an entire file directory.

     

    Prerequisites

    First, the source metadata directory must contain only one file type (Either Hierarchical files, or Delimited/Positional files), because the configuration is different for each.

    Note: it doesn't mean that the physical directory on the machine must contain only one of the file types; it is the directory metadata which must have the same file types as children.

    You can easily have multiple directory nodes in your metadata pointing to the same physical directory, each one with different file types.

    Example :

    directory

    Both directories are pointing to the same physical directory, be we separate the standard files and the hierarchical files.

    Configuring repetition

    Now, let's begin the specific configuration for replicating a directory.

    Like for a usual replication, drag and drop the source (a directory here), and the target on the template.

    But this time, we will not rename the Source metadata link to SOURCE, but to another name, SOURCEDIR, for example.

    Next, configure the repetition options in the Generation tab of the template.

    repetition

    The Variable Name must be set to SOURCE (See why here).

    This configuration will make the template executes for each file referenced in the SOURCEDIR metadata.

     

    Configuring parameters

    Force Generic Rdbms Mode parameter must be set to true.

    For Query parameter :

    • Keep the default value if you are replicating a directory containing hierarchical files
    • Set it to $SOURCE If you are replicating a directory containing Delimited/Positional files

     

    Getting started with Replicator Rdbms

      Introduction

      With the generic templates comes a powerfull replication template, named REPLICATOR Rdbms.tp

      This template allows to replicate schemas, tables or files to RDBMS with a lot of options and flexibility.

      You can decide to load or not to load the data during the replication, to create, drop, or empty the target tables, ...

      It is possible to transform columns or values during the process too...

      This article give the basics to use the replicator.

       

      Getting the template

      First, you have to drag and drop the template directly in a process.

      The replicator can be found in templates.generic/Replication/REPLICATOR Rdbms.tp.proc

      Now, you should have this :

       process

      You can take a look at the parameters to familiarize with what is available.

       

      Setting source and target

      The second step is to drag and drop on the Replicator :

      • The source schema, table, directory, file or hierarchical file which will be replicated
      • The target schema, in which the source will be replicated

      beforeRenaming

      Next, rename the two metadata links to "TARGET" and "SOURCE".

      It is important, otherwise the template has no way to know which one is the source or the target.

      renamed

       

      Configuring the template

      Now that source and target are set, you can configure the template options accordingly to your needs.

      Specific parameters

      Query

      This parameter defines how the tables will be picked from the source metadata. This is an Xpath query.

      If your source metadata link is a schema or a hierarchical file you can leave default value

      If your source metadata link is a table or a file set it to $SOURCE

      If your source metadata link is a directory see this article.

       

      Force generic Rdbms Mode

      By default, the template is using the specific tools provided by databases. Like the BULK INSERT statement of SQL Server, or Oracle's SQL Loader.

      These tools offer better performance, but may need a specific installation and configuration, like having a runtime installed on the same machine as the database for example.

      If you do not want to use these tools, set this parameter to true. The template will then use a generic way for the replication.

      Note :

      • It can increase execution time.
      • If you are replicating a directory, a file or a hierarchical file Force Generic Rdbms Mode must be set to true.

       

      Select Column Pattern and Insert Value pattern

      These two parameters allow to use xpath to control the SELECT and INSERT expressions that the replicator generated.

      See this article for an example and explanation

       

       

      Articles

      Suggest a new Article!