Package org.apache.nutch.indexer.basic

Class BasicIndexingFilter

java.lang.Object

org.apache.nutch.indexer.basic.BasicIndexingFilter

All Implemented Interfaces:

Configurable, IndexingFilter, Pluggable

public class BasicIndexingFilter
extends Object
implements IndexingFilter

Adds basic searchable fields to a document. The fields added are : domain, host, url, content, title, cache, tstamp domain is included depending on indexer.add.domain in nutch-default.xml. title is truncated as per indexer.max.title.length in nutch-default.xml. (As per NUTCH-1004, a zero-length title is not added) content is truncated as per indexer.max.content.length in nutch-default.xml.

Field Summary

Fields inherited from interface org.apache.nutch.indexer.IndexingFilter

X_POINT_ID

Constructor Summary

Constructors

Constructor	Description
BasicIndexingFilter()

Method Summary

Modifier and Type	Method	Description
NutchDocument	filter(NutchDocument doc, Parse parse, Text url, CrawlDatum datum, Inlinks inlinks)	The BasicIndexingFilter filter object which supports few configuration settings for adding basic searchable fields.
Configuration	getConf()	Get the Configuration object
void	setConf(Configuration conf)	Set the Configuration object

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

BasicIndexingFilter

public BasicIndexingFilter()

Method Details

filter

public NutchDocument filter(NutchDocument doc,
 Parse parse,
 Text url,
 CrawlDatum datum,
 Inlinks inlinks)
                     throws IndexingException

The BasicIndexingFilter filter object which supports few configuration settings for adding basic searchable fields. See indexer.add.domain, indexer.max.title.length, indexer.max.content.length in nutch-default.xml.

Specified by:

filter in interface IndexingFilter

Parameters:

doc - The NutchDocument object

parse - The relevant Parse object passing through the filter

url - URL to be filtered for anchor text

datum - The CrawlDatum entry

inlinks - The Inlinks containing anchor text

Returns:

filtered NutchDocument

Throws:

IndexingException - if an error occurs during during filtering

setConf

public void setConf(Configuration conf)

Set the Configuration object

Specified by:

setConf in interface Configurable

getConf

public Configuration getConf()

Get the Configuration object

Specified by:

getConf in interface Configurable