Description

The DynamoDBReceiver traverses one or more JSON payload subdirectories and for each file found within the directory passes generated data to the given DynamoDB target. This receiver requires a properties file to read AWS access key, AWS Secret Key, and AWS region. 


DynamoDB supports two types of primary keys, a Hash Key and a Hash and Range Key. A Hash Key consists of a single attribute that uniquely identifies an item. A Hash and Range Key consists of two attributes that together, uniquely identify an item.


Parameters

The RealtimeTestReceiver requires the following parameters to be defined:

  • tableName - Defines the name of the table in DynamoDB.
  • hashKeyName - Defines the name of the field in JSON that represents the hash key.
  • hashKeyType - Defines the data type of the hash key.
  • rangeKeyName - Defines the name of the field in JSON that represents the Range key.
  • rangeKeyType - Defines the data type of range Key.
  • jsonAttrName - Defines the name of the field in JSON that represents the data for the table.
  • payloadFilesPath - Defines the base location of the files to send. 
  • payloadFilesSubDir - Defines the naming convention for subdirectories within the payloadFilesPath.
  • payloadFileDescriptor - Defines the name of the descriptor of files to be sent from Payload SubDirectory. e.g. txt - Sends only files whose extension is txt, json - Sends only files whose extension is json
  • propertiesFilePath - Defines the location of the properties file. Property file should have awsAccessKey and awsSecretKey and, region defined. The file must have the following properties:
    • awsAccessKey=<awsAccessKey>
    • awsSecretKey=<awsSecretKey>
    • region=<region>
  • propertiesSubDir - Defines the subdirectory within the Property File path.
  • propertiesFileName - Defines the name of the DynamoDb property file.
  • logRequestCount - Defines the number of requests that are executed before logging a message to the console.


Sample JSON file:


Please refer to attached sample JSON below. Configuration and Output in next sections is based on the sample JSON file. 


Sample Receiver Configuration and Output

  • If you want to insert data in movies table for all columns:
    Output in Dynamo will be:

                

  • If you want to insert data for hashkey and specific column:
    Output in Dynamo DB will be:




Properties File

The DynamoDBReceiver must read from a properties file to attain the information to connect to the AWS account.


Sample properties file:


awsAccessKey=ABCDEFGHIJKLMNOP
awsSecretKey=a1b2c3d4+df6dfc6dfc6d
region=us-east-2
X