This article provides a basic use case example of performing Synthetic Data Masking (SDM) when migrating data between two identical MySQL databases using the G-Migration+ feature.
GenRocket uses Synthetic Data Masking or SDM to mask sensitive data values when migrating data with the G-Migration+ feature. With SDM, synthetically generated data is inserted into the destination database for table columns selected as sensitive columns.
In This Article
A power user (e.g., Test Data Engineer/TDE) wants to mask sensitive data in three related tables in a MySQL database before inserting the data into the destination database.
- Source MySQL Database (source_db)
- Target MySQL Database (target_db)
- G-Migration+ Feature
- XTS File
- Config.properties Files
Source Database Tables/Records
- Customer - 1000 Customer Records
- Address - 1 Address Record per Customer Record (1000 total)
- Account - 1 Account Record per Customer Record (1000 total)
Synthetic Data Masking (SDM) Requirements
- Customer Table Columns
- Account Table Columns
Entity Relationship (ER) Diagram
The user must complete the following steps to mask the required data and migrate it to a destination database.
Step 1 - User Creates Source and Destination Database Property Files
Configuration files must be present to connect to the source and destination databases via JDBC. Two properties files must be present on the user's local computer and contain the appropriate database connection information. For example, they can be called source.properties and destination.properties.
These files can be placed within a subdirectory and must be present to connect to each database during the data migration. For more information on how to create these configuration files, please click here.
For this example, the following files will be used and placed in a Config subdirectory within the path specified for the user's resource.JDBC directory resource in the GenRocket web platform.
Note: Please note that 'root' has been entered for the user name and password information. The user will need to use each database's actual user and password information.
Step 2 - User Creates XTS File for Tables in Source Database
A table schema must be created using the genrocket -xts command. For this example, it will be created only for the customer, account, and address tables.
Sample Command (Windows Machine)
genrocket -xts C:\Users\username\config\source_db.properties customer,account,address
Sample Command (Mac OSx/Linux)
genrocket -xts /home/users/username/config/source_db.properties customer,account,address
Note: Users can specify a schema, tables, etc., as part of the genrocket -xts command. Click here to learn more.
Step 3 - User Creates a Project with Default Project Version
A project with the default project version must be created and available to mask sensitive data values with synthetic data values prior to database insertion.
Note: Click here to learn more about creating Projects in the GenRocket web platform.
Step 4 - User Imports Table Schema using created XTS File
- The user selects G-Migration+ in Self Service Menu for Project Version.
- The user clicks the Import Table Schema button and selects the XTS file.
- The XTS is imported, and the user can view the Table Schema.
Note: Click here to see the complete table schema import steps.
Step 5 - User Creates a G-Migration+ Configuration
- The user clicks the Create G-Migration+ Config button.
- The user enters/selects the following information:
- Name = MaskingOnlyDemo
- Description = Mask Sensitive Data Values
- DataDumpSubDir = dump
- Partition Receiver = MySQLPartitionReceiver
- Source Database
- Name = source_db
- JDBCSubDirectory = Config
- JDBCConfigName = source_db.properties
- Destination Database
- Name = target_db
- JDBCSubDirectory = Config
- JDBCConfigName = target_db.properties
- The user clicks Save.
Step 6 - User Adds Customer and Account Tables
Next, the user adds the Customer and Account Tables to the G-Migration+ configuration.
- The user clicks the Manage Tables button.
- The user selects the Customer and Account Tables.
- The user clicks Save.
Note: Click here for step-by-step instructions on how to add tables and select sensitive columns for SDM.
Step 7 - User Selects Sensitive Columns in the Customer Table
- The user clicks the checkboxes to select sensitive data columns (i.e., dob, ssn, last_name, phone_number).
- The user clicks Save and Next.
Step 8 - User Selects Sensitive Columns in the Account Table
The user selects the account_name column within the Account Table.
The user clicks Save.
Step 9 - User Downloads G-Migration+ Configuration and Scenarios
Note: It is unnecessary to download the files when G-Repository Server and Client have been set up and G-Repository Server is running.
Step 10 - User Runs Command at Command Line
The user enters the following command using the name of their G-Migration+ configuration:
genrocket -gmigp <G-Migration+ Name>.gredb
genrocket -gmigp SubsettingOnlyDemo.gredb
Synthetic Data Masking (SDM) Results
For this use case, all customer, account, and address records will be migrated to the destination database (1,000 records for each table). Data values in sensitive columns will be masked with Synthetic Data before database insertion.
source_db Customer Table
target_db Customer Table
The last name, dob, ssn, and phone number of each customer have been masked with random synthetic data:
Note: This is a simple example. Users can modify Generator parameters before migrating to achieve specific data patterns.
source_db Account Table
target_db Account Table
The account number has been masked with synthetic test data: