Managing Delays in Composite Metrics

The Challenge: Delays in Composite Metrics
The Solution: Delays Mechanisms

    1. Watermark Timestamp-based Delay
    2. Automatic Composite Metric Delay
    3. Manual Composite Metric Delay
Choosing the Suitable Mechanism for Your Business
        Optimize for Accuracy of Insights
        Optimize for Faster Insights


  • Data is streamed to Anodot via numerous collectors and APIs, using a variety of schedulers, structures, and frequencies.
  • Anodot aggregates the data samples to partitions referred to as buckets.
    • As data samples arrive, they are added into an open bucket, until that bucket is closed.
  • Closing a bucket provides a signal to calculate composite metrics/alerts as defined by the users.
  • Since data samples may arrive at different delays, and even out of order, the indication to close a bucket and move to the next one becomes complex.
    • Hence the indication to calculate composite metrics based on the data samples in the bucket is complex.
  • Anodot provides a delay mechanism to provide control over this challenge.


1.  Watermark time-based delay 

How does it Work

- Using Anodot Data Collectors, or Metric 3.0 REST API allows you to send an explicit watermark timestamp.
- This watermark timestamp commits that no data samples with timestamps less than or equal to it will be sent. In other words, all data samples for “this bucket” have arrived and no need to wait anymore.
- The next steps, such as composite calculations, can start on all buckets closed by the timestamp watermark.

When to Use It

i.  All the raw metrics included in the composite are from the same stream, and the search expression includes @stream with the stream name.
ii. The stream is an Anodot DC stream, or Metrics 3.0 based.

How to Use It

i. Send the data using Anodot DC streams or Metric 3.0 protocol.

ii. Make sure relevant composite metrics are marked as "Auto calculated Delay Enabled" (see image below)

iii. You are all set

- The mechanism is turned on in the account and is applied on composite metrics according to their relevancy.

2.  Automatic Composite Metric Delay

How does it Work

Anodot calculates a delay that enables 85% of the metric samples to be included in the composite calculation. Anodot will keep monitoring and updating the required delay to enable these 85% through time.

When to Use It

i. The timestamp watermark solution cannot be applied.

How to Use It

i.  Create a composite metric according to your needs.
ii.  Go to the Composite delay option and check the Auto calculated delay enabled.


3.  Manual Composite Metric Delay

How does it Work

The user sets the desired delay to be used by Anodot.

When to Use it

i. The automatic composite delay solution cannot be applied.

How to Use It

i.   Create a composite metric according to your needs
ii.  Go to the Composite Delay option and uncheck Auto calculated delay enabled.
iii. Anodot provides a scale to choose the preferred combination according to your business requirements.



Optimize for Accuracy of Insights

  • Ensure samples for each timestamp bucket come to Anodot in chronological order to avoid buckets closing prematurely.
  • Willing to wait some extra amount of time to ensure all samples for the current bucket are received.

Optimize for Faster Insights

  • Willing to compromise some accuracy and false positive rate in order to gain speedier insights, noting that the indefinite time delay capabilities Anodot provides are more flexible than the company’s current tools. 
Was this article helpful?
0 out of 0 found this helpful