AWS Kinesis Stream Integration

OVERVIEW

The Anodot Kinesis Data Collector currently supports reading JSON records from AWS Kinesis Streams.

TO CREATE A KINESIS DATA SOURCE

  1. On the Streams page, click  Sources+.
  2. In Sources Types, click START on the Kinesis tile. The Kinesis dialog box is displayed.

    Kinesis_Source_final.png

  3. From the drop-down menu select the Kinesis stream region.

  4. Enter the Stream name you created in your Kinesis account.

CREATE A ROLE

1. In your AWS console, create a new role called "anodot-kinesis-reader"
Create_role_Kinesis.png

2. Copy the External ID generated by Anodot to the role's trust policy.

3. "anodot-kinesis-reader" Role's Trust policy:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::340481513670:root"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "<REPLACE_WITH_EXTERNAL_ID_FROM_ANODOT>"
}
}
}
]
}

4."anodot-kinesis-reader" Role's permission policy:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"kinesis:ListStreams",
"kinesis:ListShards",
"kinesis:GetShardIterator",
"kinesis:GetRecords",
"kinesis:DescribeStream",
],
"Resource": "*"
}
]
}

5. Copy the "anodot-kinesis-reader" Role's ARN from the AWS console to the Role ARN field in the new Kinesis source modal:

arn:aws:iam::<YOUR_ACCOUNT_ID>:role/anodot-kinesis-reader

6. When you have completed creating the Anodot Data Source, click continue and define the Stream Query

TO CREATE A KINESIS DATA STREAM

Stream_Query_Kinesis_wCrumbs.png

  1. Set the Stream Name or use the default.
  2. Set the Stream Filter.
      i.   Choose Record Type: either a single JSON record or multiple JSON records.
      ii.  If you use multiple JSON records, choose a Record Delimiter
      iii. Filter by key and value. Choose the :
            - filtering JSON path
            - filtering value, or choose ANY [Optional]
  3. Set the Query Schedule
    i. Set the query interval, available interval are 1 minute up to 1 hour.
    ii. Set the delay to enable sufficient data record readiness on the stream.
  4. Select timestamp, measures and dimensions.
  5. After you click collect, choose:
  • The JSON path representing the timestamp to use
  • The JSON path(s) representing measure(s)
  • The JSON path(s) representing dimension(s)
Was this article helpful?
0 out of 0 found this helpful