Description
A G-Migration+ Configuration (or Config) must be created and configured correctly to migrate a subset of data between a source and destination database. Each configuration represents a migration between two identical databases (MySQL to MySQL, Oracle to Oracle, etc.). Users can specify a different schema as needed and have the option so select sensitive columns for Synthetic Data Masking (SDM).
In This Article
- What is a G-Migration+ Configuration?
- When is a G-Migration+ Configuration Needed?
- What is a Default Domain?
- What is a Default Scenario?
- How to Create a G-Migration Configuration
- G-Migration+ Configuration Options
What is a G-Migration+ Configuration?
- A G-Migration Configuration contains both required and optional configuration options for connecting to each database and migrating a subset of data.
- Users can add tables, select sensitive columns within one or more tables, and add data subsetting conditions based on the imported table schema after creating a configuration.
When is a G-Migration+ Configuration Needed?
- Any time a user needs to perform data subsetting or synthetic Data Masking (SDM) from a source database to an identical destination database (e.g., MySQL to MySQL).
What is a Default Domain?
- A Default Domain is automatically created within the Project Version when a G-Migration+ Configuration is created. It does not contain any Attributes.
- The Partition Receiver chosen within the G-Migration+ Configuration is assigned to the Default Domain and is used to perform data subsetting between the databases. Example: MySQLPartitionReceiver
What is a Default Scenario?
- A Default Scenario is a Scenario that is automatically created within the Project Version when a G-Migration+ Configuration is created.
- It functions like any other Scenario and is used when the command is run to migrate the subset of data to the destination database.
How to Create a G-Migration Configuration
- Click the Create G-Migration Config button.
- Fill out the form accordingly by using the drop-down menus and fields. Items with an asterisk (*) are required.
- Enter a Name and Description for the configuration. For this example, the following has been entered:
- Name = G-MigrationDemo
- Description = This is a sample configuration.
- 'dump' has been entered for the Data-Dump SubDir, and the MySQLPartitionReceiver has been chosen for the Partition Receiver.
- Finally, information has been entered for both the source and destination databases.
- Source Database Name = source_db
- Destination Database Name = target_db
- JDBC Configuration (both source and destination) = config
- Source JDBC Config Name = sourcedb.properties
- Destination JDBC Config Name = targetdb.properties
- Source Database Name = source_db
- Remember to click the Save button once finished.
- The G-Migration+ Configuration will appear as shown below in the dashboard.
- Users add/exclude tables, add data subsetting conditions, and select sensitive columns within one or more tables after the table schema has been imported. For more information on how to import your table schema, click here.
G-Migration+ Configuration Options
The following options can be configured within a G-Migration+ Configuration. Items marked as required (Y) within this section must be configured.
General Information
These options are used to identify the G-Migration+ Configuration when more than one exists for a selection Project Version within a given Project.
| Option | Description | Required? |
| Name | A name for the configuration. This is what will appear within the G-Migration+ Dashboard for a given Project Version of a Project. | Y |
| Description | A description to further aid in identifying the configuration and its purpose. | N |
Primary Scenario Configuration
These options determine the storage location of Scenarios and what Scenario will be used during the data migration process.
| Option | Description | Required? |
| Scenario Directory | Defines a location where Scenarios are stored on the local computer. | Y |
| Default Scenario | GenRocket will automatically create a Scenario for the G-Migration+ Configuration. However, it can be overridden by entering a value here for additional customization. | N |
Migration Speed Optimization Configuration
These options determine the speed of the data migration. They can be used to optimize the migration based on available memory and CPU power.
| Option | Description | Required? |
| Thread Count | Defines the number of threads to be run during the migration process. The default value is 1. Note: Increasing the thread count allows the migration engine to divide the work of dumping the data across multiple threads, thus speeding up the migration process. Just ensure there is enough computer memory and CPU power to handle the load for the number of defined threads. | Y |
| Batch Size | Defines the number of rows of data that each thread may fetch from a given database table, process in memory, and store to a data-dump file during a data fetch lifecycle. The default is 10000. | Y |
Data-Dump File Location Configuration
These options determine where the data dumps will be stored during the migration.
| Option | Description | Required? |
| Data-Dump Directory | Defines a base directory where data-dump files will be temporarily stored during data subsetting. The default value is the resource.output.directory. | Y |
| Data-Dump SubDir | Defines a subdirectory under the base data-dump directory, allowing the base directory to be shared for other purposes. | Y |
Receiver & Test Data Case Configuration
Users can define an optional Test Data Case (G-Case) that can be applied to all tables that are part of data subsetting and/or masking. A Partition Receiver must be selected and match the databases used for migrating the data subset.
| Option | Description | Required? |
| Test Data Case | Defines an optional Test Data Case that will be used to modify any Scenario used during the migration. Conditions within the Test Daa Case, find matches to components within a given Scenario to modify. | N |
| Partition Receiver | Defines a Partition Receiver that will be automatically assigned to all Scenarios during a migration via GenRocket's API. Partition Receivers are designed specifically for the migration engine running under multiple threads. Partition Receivers write data to CSV files and create Control files that are used to slam large amounts of data via the database's bulk load engine. | Y |
Source Database & JDBC Connection Configuration
These options are used to configure the source database information, including JDBC connection details:
| Option | Description | Required? |
| Database Name | Defines the name of the source database where data will be migrated from. | Y |
| Schema Name | Defines the name of the source schema where data will be migrated from. Note: When not provided, the migration is performed on the table's original schema. | N |
| JDBC Directory | Defines a base directory where the source JDBC configuration file is located. | Y |
| JDBC SubDirectory | Defines a subdirectory under the base directory where the source JDBC configuration file is located. | N |
| JDBC Config Name | Defines the name of the source JDBC config file. | Y |
Destination Database & JDBC Connection Configuration
These options are used to configure the destination database information, including JDBC connection details:
| Option | Description | Required? |
| Database Name | Defines the name of the destination database to which data will be migrated. | Y |
| Schema Name | Defines the name of the destination schema to which data will be migrated. Note: When not provided, it will consider the table's original schema. | N |
| JDBC Directory | Defines a base directory where the destination JDBC configuration file is located. | Y |
| JDBC SubDirectory | Defines a subdirectory under the base directory where the destination JDBC configuration file is located. | N |
| JDBC Config Name | Defines the name of the destination JDBC config file. | Y |