Package org.apache.nutch.metrics

# Class LatencyTracker

java.lang.Object
org.apache.nutch.metrics.LatencyTracker

public class LatencyTracker extends Object
A utility class for tracking latency metrics using TDigest for percentile calculation.

This class wraps a MergingDigest data structure to collect latency samples and emit Hadoop counters with count, sum, and percentile values (p50, p95, p99). MergingDigest supports merging digests from multiple tasks for job-level percentile computation.

Usage:

     // In mapper/reducer setup
     latencyTracker = new LatencyTracker(NutchMetrics.GROUP_FETCHER, NutchMetrics.FETCHER_LATENCY);

     // During processing
     long start = System.currentTimeMillis();
     // ... operation ...
     latencyTracker.record(System.currentTimeMillis() - start);

     // In cleanup
     latencyTracker.emitCounters(context);

Emits the following counters:

- {prefix}_count_total - total number of samples
- {prefix}_sum_ms - sum of all latencies in milliseconds
- {prefix}_p50_ms - 50th percentile (median) latency
- {prefix}_p95_ms - 95th percentile latency
- {prefix}_p99_ms - 99th percentile latency

Since:
  1.22

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final String`
  `SUFFIX_COUNT_TOTAL`
  Counter name suffix for total sample count.
  `static final String`
  `SUFFIX_P50_MS`
  Counter name suffix for 50th percentile latency in milliseconds.
  `static final String`
  `SUFFIX_P95_MS`
  Counter name suffix for 95th percentile latency in milliseconds.
  `static final String`
  `SUFFIX_P99_MS`
  Counter name suffix for 99th percentile latency in milliseconds.
  `static final String`
  `SUFFIX_SUM_MS`
  Counter name suffix for sum of latencies in milliseconds.

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `LatencyTracker(String group, String prefix)`
  Creates a new LatencyTracker.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `emitCountAndSumOnly(TaskInputOutputContext<?,?,?,?> context)`
  Emits only count and sum counters (not percentiles).
  `void`
  `emitCounters(TaskInputOutputContext<?,?,?,?> context)`
  Emits all latency counters to the Hadoop context.
  `static com.tdunning.math.stats.MergingDigest`
  `fromBytes(byte[] bytes)`
  Deserializes a MergingDigest from bytes (as produced by `toBytes()`).
  `long`
  `getCount()`
  Returns the number of recorded samples.
  `long`
  `getPercentile(double quantile)`
  Returns the percentile value for the given quantile.
  `long`
  `getSum()`
  Returns the sum of all recorded latencies.
  `void`
  `merge(LatencyTracker other)`
  Merges another LatencyTracker's digest and aggregates count/sum into this one.
  `void`
  `record(long latencyMs)`
  Records a latency sample.
  `static void`
  `setJobLevelCounters(TaskInputOutputContext<?,?,?,?> context, String group, String prefix, long mergedCount, long mergedSum, com.tdunning.math.stats.MergingDigest mergedDigest)`
  Sets job-level percentile counters from a merged digest (e.g.
  `byte[]`
  `toBytes()`
  Serializes the digest to bytes for transmission to a reducer or side file.

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### SUFFIX_COUNT_TOTAL

    public static final String SUFFIX_COUNT_TOTAL
    Counter name suffix for total sample count.
    See Also:
      - Constant Field Values

  - ### SUFFIX_SUM_MS

    public static final String SUFFIX_SUM_MS
    Counter name suffix for sum of latencies in milliseconds.
    See Also:
      - Constant Field Values

  - ### SUFFIX_P50_MS

    public static final String SUFFIX_P50_MS
    Counter name suffix for 50th percentile latency in milliseconds.
    See Also:
      - Constant Field Values

  - ### SUFFIX_P95_MS

    public static final String SUFFIX_P95_MS
    Counter name suffix for 95th percentile latency in milliseconds.
    See Also:
      - Constant Field Values

  - ### SUFFIX_P99_MS

    public static final String SUFFIX_P99_MS
    Counter name suffix for 99th percentile latency in milliseconds.
    See Also:
      - Constant Field Values

- ## Constructor Details

  - ### LatencyTracker

    public LatencyTracker(String group, String prefix)
    Creates a new LatencyTracker.
    Parameters:
      `group` - the Hadoop counter group name
      `prefix` - the prefix for counter names (e.g., "fetch_latency")

- ## Method Details

  - ### record

    public void record(long latencyMs)
    Records a latency sample.
    Parameters:
      `latencyMs` - the latency in milliseconds

  - ### merge

    public void merge(LatencyTracker other)
    Merges another LatencyTracker's digest and aggregates count/sum into this one. Used to combine per-thread or per-task metrics before emitting or serializing.
    Parameters:
      `other` - the other tracker to merge in (not modified)

  - ### getCount

    public long getCount()
    Returns the number of recorded samples.
    Returns:
      the count of recorded latency samples

  - ### getSum

    public long getSum()
    Returns the sum of all recorded latencies.
    Returns:
      the sum of latencies in milliseconds

  - ### getPercentile

    public long getPercentile(double quantile)
    Returns the percentile value for the given quantile.
    Parameters:
      `quantile` - the quantile (0.0 to 1.0)
    Returns:
      the percentile value in milliseconds

  - ### toBytes

    public byte[] toBytes()
    Serializes the digest to bytes for transmission to a reducer or side file. Returns an empty array if no samples have been recorded.
    Returns:
      serialized digest bytes, or empty array if count is 0

  - ### fromBytes

    public static com.tdunning.math.stats.MergingDigest fromBytes(byte[] bytes)
    Deserializes a MergingDigest from bytes (as produced by `toBytes()`).
    Parameters:
      `bytes` - serialized digest bytes
    Returns:
      MergingDigest instance, or null if bytes is null or empty

  - ### emitCountAndSumOnly

    public void emitCountAndSumOnly(TaskInputOutputContext<?,?,?,?> context)
    Emits only count and sum counters (not percentiles). Use in mappers when a reducer will merge TDigests and set job-level percentile counters.

  - ### emitCounters

    public void emitCounters(TaskInputOutputContext<?,?,?,?> context)
    Emits all latency counters to the Hadoop context.

    Should be called once during cleanup to emit aggregated metrics.

    Parameters:
      `context` - the Hadoop task context

  - ### setJobLevelCounters

    public static void setJobLevelCounters(TaskInputOutputContext<?,?,?,?> context, String group, String prefix, long mergedCount, long mergedSum, com.tdunning.math.stats.MergingDigest mergedDigest)
    Sets job-level percentile counters from a merged digest (e.g. in a reducer that merged TDigests from all tasks). Uses the same counter names as `emitCounters(TaskInputOutputContext)`.
    Parameters:
      `context` - the Hadoop task context
      `mergedCount` - total count from merged digest
      `mergedSum` - total sum from merged digest
      `mergedDigest` - the merged MergingDigest (may be null if mergedCount is 0)