Package org.apache.nutch.indexer.more

Class MoreIndexingFilter

java.lang.Object

org.apache.nutch.indexer.more.MoreIndexingFilter

All Implemented Interfaces:

Configurable, IndexingFilter, Pluggable

public class MoreIndexingFilter
extends Object
implements IndexingFilter

Add (or reset) a few metaData properties as respective fields (if they are available), so that they can be accurately used within the search index. 'lastModifed' is indexed to support query by date, 'contentLength' obtains content length from the HTTP header, 'type' field is indexed to support query by type and finally the 'title' field is an attempt to reset the title if a content-disposition hint exists. The logic is that such a presence is indicative that the content provider wants the filename therein to be used as the title. Still need to make content-length searchable!

Author:

John Xing

Field Summary

Fields inherited from interface org.apache.nutch.indexer.IndexingFilter

X_POINT_ID

Constructor Summary

Constructors

Constructor	Description
MoreIndexingFilter()

Method Summary

Modifier and Type	Method	Description
NutchDocument	filter(NutchDocument doc, Parse parse, Text url, CrawlDatum datum, Inlinks inlinks)	Adds fields or otherwise modifies the document that will be indexed for a parse.
Configuration	getConf()
void	setConf(Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

MoreIndexingFilter

public MoreIndexingFilter()

Method Details

filter

public NutchDocument filter(NutchDocument doc,
 Parse parse,
 Text url,
 CrawlDatum datum,
 Inlinks inlinks)
                     throws IndexingException

Description copied from interface: IndexingFilter

Adds fields or otherwise modifies the document that will be indexed for a parse. Unwanted documents can be removed from indexing by returning a null value.

Specified by:

filter in interface IndexingFilter

Parameters:

doc - document instance for collecting fields

parse - parse data instance

url - page url

datum - crawl datum for the page (fetch datum from segment containing fetch status and fetch time)

inlinks - page inlinks

Returns:

modified (or a new) document instance, or null (meaning the document should be discarded)

Throws:

IndexingException - if an error occurs during during filtering

setConf

public void setConf(Configuration conf)

Specified by:

setConf in interface Configurable

getConf

public Configuration getConf()

Specified by:

getConf in interface Configurable