Description
A G-Migration+ Configuration (or Config) must be created and configured correctly to migrate a subset of data between a source and destination database.
More than one configuration can be created. Each configuration represents a migration between two identical databases (MySQL to MySQL, Oracle to Oracle, etc.).
Users can specify a different schema as needed and have the option to select sensitive columns for Synthetic Data Masking (SDM).
In This Article
- What is a G-Migration+ Configuration?
- When is a G-Migration+ Configuration Needed?
- How to Create a G-Migration Configuration
- What is a Default Domain?
- What is a Default Scenario?
- G-Migration+ Configuration Options
What is a G-Migration+ Configuration?
- A G-Migration Configuration contains both required and optional configuration options for connecting to each database and migrating a subset of data.
- After creating a configuration, users can add tables, select sensitive columns within one or more tables, and add data subsetting conditions based on the imported table schema.
When is a G-Migration+ Configuration Needed?
- Any time a user needs to perform data subsetting or synthetic Data Masking (SDM) from a source database to an identical destination database (e.g., MySQL to MySQL).
How to Create a G-Migration Configuration
- Click the Create G-Migration Config button.
- Please complete the form using the drop-down menus and fields. Items with an asterisk (*) are required.
- Enter a Name and Description for the configuration. For this example, the following has been entered:
- Name = G-MigrationDemo
- Description = This is a sample configuration.
- 'dump' has been entered for the Data-Dump SubDir.
- The MySQLPartitionReceiver has been chosen for the Partition Receiver.
- A SQL Parameters section will appear on the form when the following partition receivers are chosen:
- The SQL Insert should be set to 'true', if you need G-Migration+ to generate SQL Insert files for the destination database.
- You can use the 'I' icon to learn more about a specific parameter. Detailed descriptions are also provided at the bottom of this article.
- Finally, information has been entered for both the source and destination databases.
- Source Database Name = source_db
- Destination Database Name = target_db
- JDBC Configuration (both source and destination) = config
- Source JDBC Config Name = sourcedb.properties
- Destination JDBC Config Name = targetdb.properties
- Source Database Name = source_db
- Remember to click the Save button once finished.
- The G-Migration+ Configuration will appear as shown below.
- Next, you will need to import your table schema.
What is a Default Domain?
- A Default Domain is automatically created within the Project Version when a G-Migration+ Configuration is created. It does not contain any Attributes.
- The Partition Receiver chosen within the G-Migration+ Configuration is assigned to the Default Domain and is used to perform data subsetting between the databases. Example: MySQLPartitionReceiver
What is a Default Scenario?
- A Default Scenario is also automatically created within the Project Version when a G-Migration+ Configuration is created.
- It functions like any other Scenario and is used when the command is run to migrate the subset of data to the destination database.
G-Migration+ Configuration Options
The following options can be configured within a G-Migration+ Configuration. Items marked as required (Y) within this section must be configured.
General Information
These options are used to identify the G-Migration+ Configuration when more than one exists for a selection Project Version within a given Project.
| Option | Description | Required? |
| Name | A name for the configuration. This is what will appear within the G-Migration+ Dashboard for a given Project Version of a Project. | Y |
| Description | A description to further aid in identifying the configuration and its purpose. | N |
Primary Scenario Configuration
These options determine the storage location of Scenarios and what Scenario will be used during the data migration process.
| Option | Description | Required? |
| Scenario Directory | Defines a location where Scenarios are stored on the local computer. | Y |
| Default Scenario | GenRocket will automatically create a Scenario for the G-Migration+ Configuration. However, it can be overridden by entering a value here for additional customization. | N |
Migration Speed Optimization Configuration
These options determine the speed of the data migration. They can be used to optimize the migration based on available memory and CPU power.
| Option | Description | Required? |
| Thread Count | Defines the number of threads to be run during the migration process. The default value is 1. Note: Increasing the thread count allows the migration engine to divide the work of dumping the data across multiple threads, thus speeding up the migration process. Just ensure there is enough computer memory and CPU power to handle the load for the number of defined threads. | Y |
| Batch Size | Defines the number of rows of data that each thread may fetch from a given database table, process in memory, and store to a data-dump file during a data fetch lifecycle. The default is 10000. | Y |
Data-Dump File Location Configuration
These options determine where the data dumps will be stored during the migration.
| Option | Description | Required? |
| Data-Dump Directory | Defines a base directory where data-dump files will be temporarily stored during data subsetting. The default value is the resource.output.directory. | Y |
| Data-Dump SubDir | Defines a subdirectory under the base data-dump directory, allowing the base directory to be shared for other purposes. | Y |
Receiver & Test Data Case Configuration
Users can define an optional Test Data Case (G-Case) that can be applied to all tables that are part of data subsetting and/or masking. A Partition Receiver must be selected and match the databases used for migrating the data subset.
| Option | Description | Required? |
| Test Data Case | Defines an optional Test Data Case that will be used to modify any Scenario used during the migration. Conditions within the Test Daa Case, find matches to components within a given Scenario to modify. | N |
| Partition Receiver | Defines a Partition Receiver that will be automatically assigned to all Scenarios during a migration via GenRocket's API. Partition Receivers are designed specifically for the migration engine running under multiple threads. Partition Receivers write data to CSV files and create Control files that are used to slam large amounts of data via the database's bulk load engine. | Y |
Source Database & JDBC Connection Configuration
These options are used to configure the source database information, including JDBC connection details:
| Option | Description | Required? |
| Database Name | Defines the name of the source database where data will be migrated from. | Y |
| Schema Name | Defines the name of the source schema where data will be migrated from. Note: When not provided, the migration is performed on the table's original schema. | N |
| JDBC Directory | Defines a base directory where the source JDBC configuration file is located. | Y |
| JDBC SubDirectory | Defines a subdirectory under the base directory where the source JDBC configuration file is located. | N |
| JDBC Config Name | Defines the name of the source JDBC config file. | Y |
Destination Database & JDBC Connection Configuration
These options are used to configure the destination database information, including JDBC connection details:
| Option | Description | Required? |
| Database Name | Defines the name of the destination database to which data will be migrated. | Y |
| Schema Name | Defines the name of the destination schema to which data will be migrated. Note: When not provided, it will consider the table's original schema. | N |
| JDBC Directory | Defines a base directory where the destination JDBC configuration file is located. | Y |
| JDBC SubDirectory | Defines a subdirectory under the base directory where the destination JDBC configuration file is located. | N |
| JDBC Config Name | Defines the name of the destination JDBC config file. | Y |
SQL Parameters
These options can be used to generate SQL Insert files for the destination database. When enabled, the bulk file will be converted to an SQL Insert Statement file, which will be used for data insertion. This is helpful in environments where direct JDBC insertion is not permitted.
| Option | Description | Required? |
| SQL Insert | Determines if migration should populate destination database or generate SQL Insert Statement files for destination database. Default value is 'False'. | Y |
| Text Quote Type | Defines the character to use when quoting text data. Options are CData, Single, or Double. | N |
| Escape Character | Defines the character used to form an escape sequence for all the special characters. The default value is a backslash (\). | Y |
| Null Value | Defines the value for null. The default value is (null). | Y |