Package org.apache.nutch.crawl

# Class DeduplicationJob.DedupReducer<K extends Writable>

java.lang.Object
org.apache.hadoop.mapreduce.Reducer<K,CrawlDatum,Text,CrawlDatum>
org.apache.nutch.crawl.DeduplicationJob.DedupReducer<K>

Enclosing class:
  DeduplicationJob

public static class DeduplicationJob.DedupReducer<K extends Writable> extends Reducer<K,CrawlDatum,Text,CrawlDatum>

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

  `Reducer.Context`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `protected String[]`
  `compareOrder`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `DedupReducer()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `protected CrawlDatum`
  `getDuplicate(CrawlDatum existingDoc, CrawlDatum newDoc)`
  `void`
  `reduce(K key, Iterable<CrawlDatum> values, Reducer<K,CrawlDatum,Text,CrawlDatum>.Context context)`
  `void`
  `setup(Reducer<K,CrawlDatum,Text,CrawlDatum>.Context context)`
  `protected void`
  `writeOutAsDuplicate(CrawlDatum datum, Reducer<K,CrawlDatum,Text,CrawlDatum>.Context context)`

  ### Methods inherited from class org.apache.hadoop.mapreduce.Reducer

  `cleanup, run`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### compareOrder

    protected String[] compareOrder

- ## Constructor Details

  - ### DedupReducer

    public DedupReducer()

- ## Method Details

  - ### setup

    public void setup(Reducer<K,CrawlDatum,Text,CrawlDatum>.Context context)
    Overrides:
      `setup` in class `Reducer<K extends Writable,CrawlDatum,Text,CrawlDatum>`

  - ### writeOutAsDuplicate

    protected void writeOutAsDuplicate(CrawlDatum datum, Reducer<K,CrawlDatum,Text,CrawlDatum>.Context context) throws IOException, InterruptedException
    Throws:
      `IOException`
      `InterruptedException`

  - ### reduce

    public void reduce(K key, Iterable<CrawlDatum> values, Reducer<K,CrawlDatum,Text,CrawlDatum>.Context context) throws IOException, InterruptedException
    Overrides:
      `reduce` in class `Reducer<K extends Writable,CrawlDatum,Text,CrawlDatum>`
    Throws:
      `IOException`
      `InterruptedException`

  - ### getDuplicate

    protected CrawlDatum getDuplicate(CrawlDatum existingDoc, CrawlDatum newDoc)