Package org.apache.nutch.crawl

Class Injector.InjectMapper

java.lang.Object

org.apache.hadoop.mapreduce.Mapper<Text,Writable,Text,CrawlDatum>

org.apache.nutch.crawl.Injector.InjectMapper

Enclosing class:

Injector

public static class Injector.InjectMapper
extends Mapper<Text,Writable,Text,CrawlDatum>

InjectMapper reads

• the CrawlDb seeds are injected into
• the plain-text seed files and parses each line into the URL and metadata. Seed URLs are passed to the reducer with STATUS_INJECTED.

Depending on configuration and command-line parameters the URLs are normalized and filtered using the configured plugins.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

Mapper.Context

Field Summary

Fields

Modifier and Type	Field	Description
static final String	EQUAL_CHARACTER
static final String	TAB_CHARACTER
static final String	URL_NORMALIZING_SCOPE

Constructor Summary

Constructors

Constructor	Description
InjectMapper()

Method Summary

Modifier and Type	Method	Description
void	map(Text key, Writable value, Mapper<Text,Writable,Text,CrawlDatum>.Context context)
void	setup(Mapper<Text,Writable,Text,CrawlDatum>.Context context)

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

cleanup, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

URL_NORMALIZING_SCOPE

public static final String URL_NORMALIZING_SCOPE

See Also:

• Constant Field Values

TAB_CHARACTER

public static final String TAB_CHARACTER

See Also:

• Constant Field Values

EQUAL_CHARACTER

public static final String EQUAL_CHARACTER

See Also:

• Constant Field Values

Constructor Details

InjectMapper

public InjectMapper()

Method Details

setup

public void setup(Mapper<Text,Writable,Text,CrawlDatum>.Context context)

Overrides:

setup in class Mapper<Text,Writable,Text,CrawlDatum>

map

public void map(Text key,
 Writable value,
 Mapper<Text,Writable,Text,CrawlDatum>.Context context)
         throws IOException,
InterruptedException

Overrides:

map in class Mapper<Text,Writable,Text,CrawlDatum>

Throws:

IOException

InterruptedException