Package org.apache.nutch.metrics

# Class ErrorTracker

java.lang.Object
org.apache.nutch.metrics.ErrorTracker

public class ErrorTracker extends Object
A utility class for tracking errors by category with automatic classification.

This class provides thread-safe error counting with automatic categorization based on exception type. It uses a bounded set of error categories to stay within Hadoop's counter limits (~120 counters).

Usage in mapper/reducer or task threads:

     // In mapper/reducer setup or thread initialization
     errorTracker = new ErrorTracker(NutchMetrics.GROUP_FETCHER);
     // or with context for cached counters:
     errorTracker = new ErrorTracker(NutchMetrics.GROUP_FETCHER, context);

     // When catching exceptions
     try {
         // ... operation ...
     } catch (Exception e) {
         errorTracker.recordError(e);  // Auto-categorizes
     }

     // Or with manual categorization
     errorTracker.recordError(ErrorTracker.ErrorType.NETWORK);

     // In cleanup - emit all error counters to the job
     errorTracker.emitCounters(context);

Usage in driver/client code (no task context): When used in a job driver or other code that does not run inside a mapper/reducer, create an ErrorTracker with the single-argument constructor (counter group only). Call `recordError(Throwable)` or `recordError(ErrorTracker.ErrorType)` for consistent error categorization. Do not call `emitCounters(TaskInputOutputContext)`; Hadoop counters can only be written from within a task, so counts remain in-memory only. This allows the same categorization and logging pattern (e.g. with LOG.error) as in tasks, without emitting to job counters.

Emits the following counters (when used inside a task and emitCounters is called):

- errors_total - total number of errors across all categories
- errors_network_total - network-related errors
- errors_protocol_total - protocol errors
- errors_parsing_total - parsing errors
- errors_url_total - URL-related errors
- errors_scoring_total - scoring filter errors
- errors_indexing_total - indexing filter errors
- errors_timeout_total - timeout errors
- errors_other_total - uncategorized errors

Since:
  1.22

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `static enum`
  `ErrorTracker.ErrorType`
  Error type categories for classification.

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `ErrorTracker(String group)`
  Creates a new ErrorTracker for the specified counter group.
  `ErrorTracker(String group, TaskInputOutputContext<?,?,?,?> context)`
  Creates a new ErrorTracker with cached counter references.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static ErrorTracker.ErrorType`
  `categorize(Throwable t)`
  Categorizes a throwable into an error type.
  `void`
  `emitCounters(TaskInputOutputContext<?,?,?,?> context)`
  Emits all error counters to the Hadoop context.
  `long`
  `getCount(ErrorTracker.ErrorType type)`
  Returns the count for a specific error type.
  `static String`
  `getCounterName(Throwable t)`
  Gets the counter name for a throwable based on its categorization.
  `static String`
  `getCounterName(ErrorTracker.ErrorType type)`
  Gets the counter name constant for a given error type.
  `long`
  `getTotalCount()`
  Returns the total count of all errors.
  `void`
  `incrementCounters(Throwable t)`
  Directly increments cached error counters without local accumulation.
  `void`
  `incrementCounters(ErrorTracker.ErrorType type)`
  Directly increments cached error counters without local accumulation.
  `void`
  `initCounters(TaskInputOutputContext<?,?,?,?> context)`
  Initializes cached counter references from the Hadoop context.
  `void`
  `recordError(Throwable t)`
  Records an error with automatic categorization based on the throwable type.
  `void`
  `recordError(ErrorTracker.ErrorType type)`
  Records an error with explicit category.

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### ErrorTracker

    public ErrorTracker(String group)
    Creates a new ErrorTracker for the specified counter group.

    Use in mapper/reducer setup or thread initialization: call `initCounters(TaskInputOutputContext)` in setup() to cache counter references, then `emitCounters(TaskInputOutputContext)` in cleanup to emit counts to the job.

    Use in driver/client code (no task context): do not call initCounters or emitCounters. Only `recordError(Throwable)` and `recordError(ErrorTracker.ErrorType)` are used; counts stay in-memory for consistent categorization and logging (e.g. with LOG.error).

    Parameters:
      `group` - the Hadoop counter group name (e.g., NutchMetrics.GROUP_FETCHER)

  - ### ErrorTracker

    public ErrorTracker(String group, TaskInputOutputContext<?,?,?,?> context)
    Creates a new ErrorTracker with cached counter references.

    This constructor caches all counter references at creation time, avoiding repeated counter lookups in hot paths.

    Parameters:
      `group` - the Hadoop counter group name
      `context` - the Hadoop task context for caching counters

- ## Method Details

  - ### initCounters

    public void initCounters(TaskInputOutputContext<?,?,?,?> context)
    Initializes cached counter references from the Hadoop context.

    Call this method in the mapper/reducer setup() method to cache counter references and avoid repeated lookups during processing.

    Parameters:
      `context` - the Hadoop task context

  - ### recordError

    public void recordError(Throwable t)
    Records an error with automatic categorization based on the throwable type.
    Parameters:
      `t` - the throwable to categorize and record

  - ### recordError

    public void recordError(ErrorTracker.ErrorType type)
    Records an error with explicit category.
    Parameters:
      `type` - the error type category

  - ### getCount

    public long getCount(ErrorTracker.ErrorType type)
    Returns the count for a specific error type.
    Parameters:
      `type` - the error type
    Returns:
      the count for that error type

  - ### getTotalCount

    public long getTotalCount()
    Returns the total count of all errors.
    Returns:
      the total error count

  - ### emitCounters

    public void emitCounters(TaskInputOutputContext<?,?,?,?> context)
    Emits all error counters to the Hadoop context.

    Should be called once during cleanup to emit aggregated metrics. Only emits counters for error types that have non-zero counts.

    If counters were cached via `initCounters(TaskInputOutputContext)`, uses the cached references for better performance.

    Parameters:
      `context` - the Hadoop task context

  - ### incrementCounters

    public void incrementCounters(Throwable t)
    Directly increments cached error counters without local accumulation.

    Use this method when you want to immediately update Hadoop counters rather than accumulating locally and emitting in cleanup. Requires `initCounters(TaskInputOutputContext)` to have been called.

    Parameters:
      `t` - the throwable to categorize and count
    Throws:
      `IllegalStateException` - if counters have not been initialized

  - ### incrementCounters

    public void incrementCounters(ErrorTracker.ErrorType type)
    Directly increments cached error counters without local accumulation.

    Use this method when you want to immediately update Hadoop counters rather than accumulating locally and emitting in cleanup. Requires `initCounters(TaskInputOutputContext)` to have been called.

    Parameters:
      `type` - the error type to count
    Throws:
      `IllegalStateException` - if counters have not been initialized

  - ### categorize

    public static ErrorTracker.ErrorType categorize(Throwable t)
    Categorizes a throwable into an error type.

    The categorization checks the exception class hierarchy to determine the most appropriate category. Timeout exceptions are checked first as they are a subclass of IOException.

    Parameters:
      `t` - the throwable to categorize
    Returns:
      the appropriate ErrorType for the throwable

  - ### getCounterName

    public static String getCounterName(ErrorTracker.ErrorType type)
    Gets the counter name constant for a given error type.
    Parameters:
      `type` - the error type
    Returns:
      the counter name constant from NutchMetrics

  - ### getCounterName

    public static String getCounterName(Throwable t)
    Gets the counter name for a throwable based on its categorization.

    This is a convenience method for direct use in catch blocks:

         } catch (Exception e) {
             context.getCounter(group, ErrorTracker.getCounterName(e)).increment(1);
         }

    Parameters:
      `t` - the throwable to get the counter name for
    Returns:
      the counter name constant from NutchMetrics