Description
When a Scenario is run, the generated output will be in a single file that contains every record for most configured Receivers. The Dynamic File and Directory Configuration feature allows you to customize the following for select Receivers:
- The number of records in each generated Output file.
- The number of files in each generated Output Subdirectory.
- The naming convention of the Output File.
- The naming convention of the Output Subdirectory.
In This Article
- What are the File and Directory Configuration Tabs?
- When should you use the File and Directory Configuration Tabs?
- How do you use the File and Directory Configuration Tabs?
- What Receivers have this Feature?
- File and Directory Configuration Options
- File and Directory Name Configuration Options
- Example File and Directory Configurations
- Example 1 - Generate 100 to 200 Records per File and 10 Files per Directory
- Example 2 - Generate 200 Records per File and 5 to 10 Files per Directory
- Example 3 - Reference Boolean True/False Attribute and 200 Files per Directory
- Example 4 - Reference Attribute to Include Generated Value in Output File Name
What are the File and Directory Configuration Tabs?
These configuration tabs can be used to dynamically configure the number of records per generated file and the number of files within each generated directory when a Scenario is run. The number of generated files and directories depends on the following:
- loopCount value for the Domain
- File and Directory Tab Configurations
Apart from that, you can also configure the naming of your output file and output sub-directory.
When should you use the File and Directory Configuration Tabs?
- These configuration tabs should be used when you want the generated records to be placed in multiple files and the files to be placed in one or more directories.
- These configuration tabs should be used when you want the name of the output files and output sub-directories to be dynamic.
How do you use the File and Directory Configuration Tabs?
- Select the File or Directory Configuration Tab
- Select a creation event (no event, random, constant, or reference)
- Configure the Parameters for the creation event
- Configure the Naming Convention for the file or directory
- Download and Run the Scenario
What Receivers have this Feature?
This feature has not been implemented for all Receivers at this time. The following Receivers currently have this functionality:
- XMLFileReceiver
- XMLSegmentMergeReceiver
- JSONFileReceiver
- JSONSegmentMergeReceiver
- DelimitedFileReceiver
- DelimitedFileSegmentMergeReceiver
- ExcelFileReceiver
- SQLFileInsertReceiver
- FixedFileReceiver
- ParquetFileReceiver
- ParquetSegmentMergeReceiver
- ParquetSegmentMergeV2Receiver
- ImageTemplateReceiver
File and Directory Configuration Options
The File and Directory Config Tabs are used to configure what event will trigger file/directory creation and the naming configuration for generated files/directories. Four options are provided for the creation event:
- No Event - The value defined within the Receiver Parameters will be used.
- Random - Each time the count reaches the chosen random value, it triggers the creation event and afterwards determines a new random value.
- startRange - The lowest number of records/files that will be in a file/directory.
- endRange - The highest number of records/files that will be in a file/directory.
- seed - The same number of files/directories will be generated each time when a seed value is entered.
- Constant - Each time the count reaches the defined value, it triggers the creation event and afterward starts the counter again.
- count - The create event is triggered each time the number of records/files matches the entered value.
- count - The create event is triggered each time the number of records/files matches the entered value.
- Reference - References an Attribute that must return a true/false value to trigger the creation event.
- booleanReference - The create event is triggered each time "true" is returned.
The image below shows a Random Dynamic File Configuration:
File and Directory Name Configuration Options
File and Directory Names can have up to 10 items within the configuration. The following options are provided for each configuration line:
- Constant - A constant value that will be part of the file/directory name.
- Counter - A value that increases by 1 for each file/directory that is created.
- Timestamp - A timestamp that will be part of the file/directory name.
- Reference - References a Domain Attribute that will be part of the file/directory name.
Important: The final configuration line for files must be the File Extension. The example below is .txt because delimited files are being created.
Example File and Directory Configurations
For each of the examples in this section, the loopCount for a User Domain has been set to 10,000 records.
The DelimitedFileReceiver will be used for all provided examples.
Example 1 - Generate 100 to 200 Records per File and 10 Files per Directory
In this example, the Random radio button has been selected. The range is between 100 and 200 records, meaning each file will have a number of records between this set number range.
The file name will have this naming convention: sample-1.txt, sample-2.txt, sample-3.txt,...
The Constant radio button has been selected for the directory configuration in this example. Each File Directory will contain 10 files and be named accordingly: Users-1, Users-2, Users-3,...., Users-9, and Users-10.
Sample Output
7 File Directories are created with this naming convention: Users-1, Users-2, etc.
Users-1 contains 10 files with this naming convention: Sample-1.txt, Sample-2.txt, etc.
Sample-1.txt contains 149 records. Each file will have somewhere between 100 and 200 records according to the startRange and endRange Parameters within the file configuration.
The last directory (Users-7) will hold the remaining files, which is 9 files in this example.
Example 2 - Generate 200 Records per File and 5 to 10 Files per Directory
In this example, the Constant radio button has been selected for the file configuration, and a value of 200 has been entered. Each time the count reaches the defined value, it triggers the creation event and afterward starts the counter again.
The Random radio button has been selected for the directory configuration. The range has been configured to be between 5 and 10 directories.
Sample Output
For this example, 8 file directories were created and contain a total of 50 files.
The Users-1 directory contains 8 files:
The Users-6 directory contains 6 files:
The Users-8 directory contains 2 files. The last directory will contain the remaining or leftover files.
Each file contains 200 records, and the last file will contain the remaining record count. Sample-1.txt is shown below:
Example 3 - Reference Boolean True/False Attribute and 200 Files per Directory
In this example, the Reference radio button has been selected. It will reference a Boolean Attribute titled "booleanValue". With this option, the file creation event will be triggered when a record has a "True" value.
The Constant radio button has been selected for the directory configuration. A new directory will be created for every 200 files.
Sample Output
In this example, 25 Users directories are created.
Each directory will contain 200 files, with the final directory containing the remaining files and records:
The Reference file configuration option will trigger file creation each time a "True" value is generated by the referenced Attribute. Sample-1.txt has 2 records in it.
The record that triggers file creation will be in the next file. Sample-2.txt is shown below:
The fourth record also had a "True" Boolean value and is in Sample-3.txt:
Example 4 - Reference Attribute and Include Generated Value in Output File Name
In this example, a file will be created for each generated record. The Reference option will be used in the File Config Tab to reference the firstName and lastName Attributes within a User Domain. The generated values of these Attributes will be included in the file name of each generated file. To do so, we will need to do the following:
- Set up a file creation event to create a new file for each new record.
- Set up a dynamic file naming convention to reference the lastName and firstName Attributes.
For each output file, the file name will appear like this: lastNamefirstName.txt (e.g., StantonJane.txt).
File Creation Event Configuration
To ensure one file is created for each record, Constant has been selected, and "1" has been entered for the count.
Dynamic File Name Creation Configuration
The Dynamic File Name Creation Configuration references the lastName and the firstName Attributes. Since the DelimitedFileReceiver is being used for each of these examples, a constant of ".txt" has been added in the last line as the file extension.
Sample Output
Each file name contains the user's last name and then their first name with the ".txt" extension.
Each file contains one record. An example is shown below: