Description

A G-Migration+ Configuration (or Config) must be created and configured correctly to migrate a subset of data between a source and destination database. 


More than one configuration can be created. Each configuration represents a migration between two identical databases (MySQL to MySQL, Oracle to Oracle, etc.). 

Users can specify a different schema as needed and have the option to select sensitive columns for Synthetic Data Masking (SDM). 


In This Article


What is a G-Migration+ Configuration?

  • A G-Migration Configuration contains both required and optional configuration options for connecting to each database and migrating a subset of data. 

  • After creating a configuration, users can add tables, select sensitive columns within one or more tables, and add data subsetting conditions based on the imported table schema. 


When is a G-Migration+ Configuration Needed? 

  • Any time a user needs to perform data subsetting or synthetic Data Masking (SDM) from a source database to an identical destination database (e.g., MySQL to MySQL). 


How to Create a G-Migration Configuration

  • Click the Create G-Migration Config button.


  • Fill out the form accordingly by using the drop-down menus and fields. Items with an asterisk (*) are required. 



  • Enter a Name and Description for the configuration. For this example, the following has been entered: 
    • Name = G-MigrationDemo
    • Description = This is a sample configuration.


  • 'dump' has been entered for the Data-Dump SubDir, and the MySQLPartitionReceiver has been chosen for the Partition Receiver


  • Finally, information has been entered for both the source and destination databases. 
    • Source Database Name = source_db
    • Destination Database Name = target_db
    • JDBC Configuration (both source and destination) = config
    • Source JDBC Config Name = sourcedb.properties
    • Destination JDBC Config Name = targetdb.properties

  • Remember to click the Save button once finished.



  • The G-Migration+ Configuration will appear as shown below in the dashboard. 



  • Users add/exclude tables, add data subsetting conditions, and select sensitive columns within one or more tables after importing the table schema. For more information on how to import your table schema, click here


What is a Default Domain? 

  • A Default Domain is automatically created within the Project Version when a G-Migration+ Configuration is created. It does not contain any Attributes.

  • The Partition Receiver chosen within the G-Migration+ Configuration is assigned to the Default Domain and is used to perform data subsetting between the databases. Example: MySQLPartitionReceiver


What is a Default Scenario? 

  • A Default Scenario is also automatically created within the Project Version when a G-Migration+ Configuration is created. 

  • It functions like any other Scenario and is used when the command is run to migrate the subset of data to the destination database.

     

G-Migration+ Configuration Options

The following options can be configured within a G-Migration+ Configuration. Items marked as required (Y) within this section must be configured.


General Information

These options are used to identify the G-Migration+ Configuration when more than one exists for a selection Project Version within a given Project.

OptionDescriptionRequired?
NameA name for the configuration. This is what will appear within the G-Migration+ Dashboard for a given Project Version of a Project. Y
DescriptionA description to further aid in identifying the configuration and its purpose. N



Primary Scenario Configuration

These options determine the storage location of Scenarios and what Scenario will be used during the data migration process.

OptionDescriptionRequired?
Scenario DirectoryDefines a location where Scenarios are stored on the local computer. Y
Default ScenarioGenRocket will automatically create a Scenario for the G-Migration+ Configuration. However, it can be overridden by entering a value here for additional customization.
N


Migration Speed Optimization Configuration

These options determine the speed of the data migration. They can be used to optimize the migration based on available memory and CPU power.


OptionDescriptionRequired?
Thread CountDefines the number of threads to be run during the migration process. The default value is 1.

Note: Increasing the thread count allows the migration engine to divide the work of dumping the data across multiple threads, thus speeding up the migration process. Just ensure there is enough computer memory and CPU power to handle the load for the number of defined threads.
Y
Batch SizeDefines the number of rows of data that each thread may fetch from a given database table, process in memory, and store to a data-dump file during a data fetch lifecycle. The default is 10000. Y


Data-Dump File Location Configuration

These options determine where the data dumps will be stored during the migration.

OptionDescriptionRequired?
Data-Dump DirectoryDefines a base directory where data-dump files will be temporarily stored during data subsetting. The default value is the resource.output.directory. Y
Data-Dump SubDirDefines a subdirectory under the base data-dump directory, allowing the base directory to be shared for other purposes. Y


Receiver & Test Data Case Configuration

Users can define an optional Test Data Case (G-Case) that can be applied to all tables that are part of data subsetting and/or masking. A Partition Receiver must be selected and match the databases used for migrating the data subset. 


OptionDescriptionRequired?
Test Data CaseDefines an optional Test Data Case that will be used to modify any Scenario used during the migration. Conditions within the Test Daa Case, find matches to components within a given Scenario to modify. N
Partition ReceiverDefines a Partition Receiver that will be automatically assigned to all Scenarios during a migration via GenRocket's API.

Partition Receivers are designed specifically for the migration engine running under multiple threads.

Partition Receivers write data to CSV files and create Control files that are used to slam large amounts of data via the database's bulk load engine. 
Y


Source Database & JDBC Connection Configuration

These options are used to configure the source database  information, including JDBC connection details:


OptionDescriptionRequired?
Database NameDefines the name of the source database where data will be migrated from.Y
Schema NameDefines the name of the source schema where data will be migrated from. 

Note: When not provided, the migration is performed on the table's original schema. 
N
JDBC DirectoryDefines a base directory where the source JDBC configuration file is located.Y
JDBC SubDirectoryDefines a subdirectory under the base directory where the source JDBC configuration file is located.N
JDBC Config NameDefines the name of the source JDBC config file.Y


Destination Database & JDBC Connection Configuration

These options are used to configure the destination database information, including JDBC connection details: 


OptionDescriptionRequired?
Database NameDefines the name of the destination database to which data will be migrated.Y
Schema NameDefines the name of the destination schema to which data will be migrated. 

Note: When not provided, it will consider the table's original schema. 
N
JDBC DirectoryDefines a base directory where the destination JDBC configuration file is located.Y
JDBC SubDirectoryDefines a subdirectory under the base directory where the destination JDBC configuration file is located.N
JDBC Config NameDefines the name of the destination JDBC config file.Y