The DynamoDBReceiver traverses one or more JSON payload subdirectories and for each file found within the directory passes generated data to the given DynamoDB target. This receiver requires a properties file to read AWS access key, AWS Secret Key, and AWS region.
DynamoDB supports two types of primary keys, a Hash Key and a Hash and Range Key. A Hash Key consists of a single attribute that uniquely identifies an item. A Hash and Range Key consists of two attributes that together, uniquely identify an item.
The RealtimeTestReceiver requires the following parameters to be defined:
- tableName - Defines the name of the table in DynamoDB.
- hashKeyName - Defines the name of the field in JSON that represents the hash key.
- hashKeyType - Defines the data type of the hash key.
- rangeKeyName - Defines the name of the field in JSON that represents the Range key.
- rangeKeyType - Defines the data type of range Key.
- jsonAttrName - Defines the name of the field in JSON that represents the data for the table.
- payloadFilesPath - Defines the base location of the files to send.
- payloadFilesSubDir - Defines the naming convention for subdirectories within the payloadFilesPath.
- payloadFileDescriptor - Defines the name of the descriptor of files to be sent from Payload SubDirectory. e.g. txt - Sends only files whose extension is txt, json - Sends only files whose extension is json
- propertiesFilePath - Defines the location of the properties file. Property file should have awsAccessKey and awsSecretKey and, region defined. The file must have the following properties:
- propertiesSubDir - Defines the subdirectory within the Property File path.
- propertiesFileName - Defines the name of the DynamoDb property file.
- logRequestCount - Defines the number of requests that are executed before logging a message to the console.
Sample JSON file:
Please refer to attached sample JSON below. Configuration and Output in next sections is based on the sample JSON file.
Sample Receiver Configuration and Output
- If you want to insert data in movies table for all columns:
Output in Dynamo will be:
- If you want to insert data for hashkey and specific column:
Output in Dynamo DB will be:
The DynamoDBReceiver must read from a properties file to attain the information to connect to the AWS account.
Sample properties file: