Package org.apache.nutch.crawl

Class LinkDbFilter

java.lang.Object

org.apache.hadoop.mapreduce.Mapper<Text,Inlinks,Text,Inlinks>

org.apache.nutch.crawl.LinkDbFilter

public class LinkDbFilter
extends Mapper<Text,Inlinks,Text,Inlinks>

This class provides a way to separate the URL normalization and filtering steps from the rest of LinkDb manipulation code.

Author:

Andrzej Bialecki

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

Mapper.Context

Field Summary

Fields

Modifier and Type	Field	Description
static final String	URL_FILTERING
static final String	URL_NORMALIZING
static final String	URL_NORMALIZING_SCOPE

Constructor Summary

Constructors

Constructor	Description
LinkDbFilter()

Method Summary

Modifier and Type	Method	Description
void	map(Text key, Inlinks value, Mapper<Text,Inlinks,Text,Inlinks>.Context context)
void	setup(Mapper<Text,Inlinks,Text,Inlinks>.Context context)

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

cleanup, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

URL_FILTERING

public static final String URL_FILTERING

See Also:

• Constant Field Values

URL_NORMALIZING

public static final String URL_NORMALIZING

See Also:

• Constant Field Values

URL_NORMALIZING_SCOPE

public static final String URL_NORMALIZING_SCOPE

See Also:

• Constant Field Values

Constructor Details

LinkDbFilter

public LinkDbFilter()

Method Details

setup

public void setup(Mapper<Text,Inlinks,Text,Inlinks>.Context context)

Overrides:

setup in class Mapper<Text,Inlinks,Text,Inlinks>

map

public void map(Text key,
 Inlinks value,
 Mapper<Text,Inlinks,Text,Inlinks>.Context context)
         throws IOException,
InterruptedException

Overrides:

map in class Mapper<Text,Inlinks,Text,Inlinks>

Throws:

IOException

InterruptedException