Package org.apache.nutch.crawl

# Class CrawlDb

java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.nutch.util.NutchTool
org.apache.nutch.crawl.CrawlDb

All Implemented Interfaces:
  `Configurable`, `Tool`

public class CrawlDb extends NutchTool implements Tool
This class takes the output of the fetcher and updates the crawldb accordingly.

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final String`
  `CRAWLDB_ADDITIONS_ALLOWED`
  `static final String`
  `CRAWLDB_PURGE_404`
  `static final String`
  `CRAWLDB_PURGE_ORPHANS`
  `static final String`
  `CURRENT_NAME`
  `static final String`
  `LOCK_NAME`

  ### Fields inherited from class org.apache.nutch.util.NutchTool

  `currentJob, currentJobNum, numJobs, results, status`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `CrawlDb()`
  `CrawlDb(Configuration conf)`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static Job`
  `createJob(Configuration config, Path crawlDb)`
  `static void`
  `install(Job job, Path crawlDb)`
  `static Path`
  `lock(Configuration job, Path crawlDb, boolean force)`
  `static void`
  `main(String[] args)`
  `int`
  `run(String[] args)`
  `Map<String,Object>`
  `run(Map<String,Object> args, String crawlId)`
  Programmatic entry point for `NutchTool` callers.
  `void`
  `update(Path crawlDb, Path[] segments, boolean normalize, boolean filter)`
  `void`
  `update(Path crawlDb, Path[] segments, boolean normalize, boolean filter, boolean additionsAllowed, boolean force)`

  ### Methods inherited from class org.apache.nutch.util.NutchTool

  `getProgress, getStatus, killJob, setConf, stopJob`

  ### Methods inherited from class org.apache.hadoop.conf.Configured

  `getConf`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

  ### Methods inherited from interface org.apache.hadoop.conf.Configurable

  `getConf, setConf`

- ## Field Details

  - ### CRAWLDB_ADDITIONS_ALLOWED

    public static final String CRAWLDB_ADDITIONS_ALLOWED
    See Also:
      - Constant Field Values

  - ### CRAWLDB_PURGE_404

    public static final String CRAWLDB_PURGE_404
    See Also:
      - Constant Field Values

  - ### CRAWLDB_PURGE_ORPHANS

    public static final String CRAWLDB_PURGE_ORPHANS
    See Also:
      - Constant Field Values

  - ### CURRENT_NAME

    public static final String CURRENT_NAME
    See Also:
      - Constant Field Values

  - ### LOCK_NAME

    public static final String LOCK_NAME
    See Also:
      - Constant Field Values

- ## Constructor Details

  - ### CrawlDb

    public CrawlDb()

  - ### CrawlDb

    public CrawlDb(Configuration conf)

- ## Method Details

  - ### update

    public void update(Path crawlDb, Path[] segments, boolean normalize, boolean filter) throws IOException, InterruptedException, ClassNotFoundException
    Throws:
      `IOException`
      `InterruptedException`
      `ClassNotFoundException`

  - ### update

    public void update(Path crawlDb, Path[] segments, boolean normalize, boolean filter, boolean additionsAllowed, boolean force) throws IOException, InterruptedException, ClassNotFoundException
    Throws:
      `IOException`
      `InterruptedException`
      `ClassNotFoundException`

  - ### createJob

    public static Job createJob(Configuration config, Path crawlDb) throws IOException
    Throws:
      `IOException`

  - ### lock

    public static Path lock(Configuration job, Path crawlDb, boolean force) throws IOException
    Throws:
      `IOException`

  - ### install

    public static void install(Job job, Path crawlDb) throws IOException
    Throws:
      `IOException`

  - ### main

    public static void main(String[] args) throws Exception
    Throws:
      `Exception`

  - ### run

    public int run(String[] args) throws Exception
    Specified by:
      `run` in interface `Tool`
    Throws:
      `Exception`

  - ### run

    public Map<String,Object> run(Map<String,Object> args, String crawlId) throws Exception
    Programmatic entry point for `NutchTool` callers.
    Specified by:
      `run` in class `NutchTool`
    Parameters:
      `args` - a `Map` of arguments to be run with the tool
      `crawlId` - a crawl identifier to associate with the tool invocation
    Returns:
      Map results object if tool executes successfully otherwise null
    Throws:
      `Exception` - if there is an error during the tool execution