Description 

The DynamoDBReceiver traverses one or more JSON payload subdirectories, and for each file found within the directory passes generated data to the given DynamoDB target. This receiver requires a properties file to read the AWS access key, AWS Secret Key, and AWS region.


DynamoDB supports two types of primary keys, a Hash Key and a Hash and Range Key. A Hash Key consists of a single attribute that uniquely identifies an item. A Hash and Range Key consists of two attributes that, together, uniquely identify an item.


In This Article


Receiver Parameters

The following parameters should be defined for the DynamoDBReceiver. Items with an asterisk(*) are required.

  • tableName* - Defines the name of the table in DynamoDB.
  • hashKeyName - Defines the name of the field in JSON that represents the hash key.
  • hashKeyType* - Defines the data type of the hash key.
  • rangeKeyName - Defines the name of the field in JSON that represents the Range key.
  • rangeKeyType* - Defines the data type of range Key.
  • jsonAttrName - Defines the name of the field in JSON that represents the data for the table.
  • payloadFilesPath* - Defines the base location of the files to send. 
  • payloadFilesSubDir - Defines the naming convention for subdirectories within the payloadFilesPath.
  • payloadFileDescriptor - Defines the name of the descriptor of files to be sent from the Payload SubDirectory. e.g., txt - Sends only files whose extension is txt, json - Sends only files whose extension is json
  • propertiesFilePath* - Defines the location of the properties file. Property file should have awsAccessKey and awsSecretKey and region defined. The file must have the following properties:
    • awsAccessKey=<awsAccessKey>
    • awsSecretKey=<awsSecretKey>
    • region=<region>
  • propertiesSubDir - Defines the subdirectory within the Property File path.
  • propertiesFileName* - Defines the name of the DynamoDb property file.
  • logRequestCount* - Defines the number of requests that are executed before logging a message to the console.


Receiver Attributes Property Keys

  • No Property Keys need to be configured for Attributes with this Receiver.

Properties File

The DynamoDBReceiver must read from a properties file to attain the information to connect to the AWS account.


Sample properties file:

awsAccessKey=ABCDEFGHIJKLMNOP
awsSecretKey=a1b2c3d4+df6dfc6dfc6d
region=us-east-2


X

Sample JSON File

Please refer to the attached sample JSON below. Configuration and Output in the following sections are based on the sample JSON file. 


Sample Receiver Configuration and Output

Example 1 - Insert Data into a Movies Table for all Columns

This sample configuration shows how to configure the DynamoDBReceiver to insert data into a movies table for all columns.



Output in Dynamo will be:

                


Example 2 - Insert Data for Hash Key and Specific Column

If you want to insert data for hash key and specific column, see below:


Output in Dynamo DB will be: