Package org.apache.nutch.metadata

Interface Nutch

All Known Implementing Classes:

CaseInsensitiveMetadata, Metadata, SpellCheckedMetadata

public interface Nutch

A collection of Nutch internal metadata constants.

Author:

Chris Mattmann, Jérôme Charron

Field Summary

Fields

Modifier and Type	Field	Description
static final String	ARG_CRAWLDB	Argument key for crawldb location in programmatic jobs.
static final String	ARG_HOSTDB	Argument key for hostdb location in programmatic jobs.
static final String	ARG_LINKDB	Argument key for linkdb location in programmatic jobs.
static final String	ARG_SEEDDIR	Argument key for seed URL directory path.
static final String	ARG_SEGMENTDIR	Argument key for a directory of segments; similar to the -dir option in bin/nutch.
static final String	ARG_SEGMENTS	Argument key for one segment or a list of segments (job-dependent).
static final String	CACHING_FORBIDDEN_ALL	Don't show either original forbidden content or summaries.
static final String	CACHING_FORBIDDEN_CONTENT	Don't show original forbidden content, but show summaries.
static final String	CACHING_FORBIDDEN_KEY	Sites may request that search engines don't provide access to cached documents.
static final String	CACHING_FORBIDDEN_NONE	Show both original forbidden content and summaries (default).
static final String	CHAR_ENCODING_FOR_CONVERSION
static final String	CRAWL_ID_KEY	Crawl id key for programmatic jobs.
static final String	FETCH_EVENT_CONTENTLANG	Content-lanueage key in the Pub/Sub event metadata for the content-language of the parsed page
static final String	FETCH_EVENT_CONTENTTYPE	Content-type key in the Pub/Sub event metadata for the content-type of the parsed page
static final String	FETCH_EVENT_FETCHTIME	Fetch time key in the Pub/Sub event metadata for the fetch time of the parsed page
static final String	FETCH_EVENT_SCORE	Score key in the Pub/Sub event metadata for the score of the parsed page
static final String	FETCH_EVENT_TITLE	Title key in the Pub/Sub event metadata for the title of the parsed page
static final String	FETCH_STATUS_KEY
static final String	FETCH_TIME_KEY
static final String	FIXED_INTERVAL_KEY	Used by AdaptiveFetchSchedule to maintain custom fetch interval
static final String	GENERATE_TIME_KEY
static final String	ORIGINAL_CHAR_ENCODING
static final String	PROTO_STATUS_KEY
static final Text	PROTOCOL_STATUS_CODE_KEY
static final String	REPR_URL_KEY
static final String	ROBOTS_METATAG	Name to store the robots metatag in ParseData's metadata.
static final String	SCORE_KEY
static final String	SEGMENT_NAME_KEY
static final String	SIGNATURE_KEY
static final String	STAT_PROGRESS	For progress of job (programmatic / tooling).
static final String	VAL_RESULT	Name of the key used in the result map from NutchTool.run(java.util.Map<java.lang.String, java.lang.Object>, java.lang.String).
static final Text	WRITABLE_FIXED_INTERVAL_KEY
static final Text	WRITABLE_GENERATE_TIME_KEY
static final Text	WRITABLE_PROTO_STATUS_KEY
static final Text	WRITABLE_REPR_URL_KEY

Field Details

ORIGINAL_CHAR_ENCODING

static final String ORIGINAL_CHAR_ENCODING

See Also:

• Constant Field Values

CHAR_ENCODING_FOR_CONVERSION

static final String CHAR_ENCODING_FOR_CONVERSION

See Also:

• Constant Field Values

SIGNATURE_KEY

static final String SIGNATURE_KEY

See Also:

• Constant Field Values

SEGMENT_NAME_KEY

static final String SEGMENT_NAME_KEY

See Also:

• Constant Field Values

SCORE_KEY

static final String SCORE_KEY

See Also:

• Constant Field Values

GENERATE_TIME_KEY

static final String GENERATE_TIME_KEY

See Also:

• Constant Field Values

WRITABLE_GENERATE_TIME_KEY

static final Text WRITABLE_GENERATE_TIME_KEY

PROTOCOL_STATUS_CODE_KEY

static final Text PROTOCOL_STATUS_CODE_KEY

PROTO_STATUS_KEY

static final String PROTO_STATUS_KEY

See Also:

• Constant Field Values

WRITABLE_PROTO_STATUS_KEY

static final Text WRITABLE_PROTO_STATUS_KEY

FETCH_TIME_KEY

static final String FETCH_TIME_KEY

See Also:

• Constant Field Values

FETCH_STATUS_KEY

static final String FETCH_STATUS_KEY

See Also:

• Constant Field Values

ROBOTS_METATAG

static final String ROBOTS_METATAG

Name to store the robots metatag in ParseData's metadata.

See Also:

• Constant Field Values

CACHING_FORBIDDEN_KEY

static final String CACHING_FORBIDDEN_KEY

Sites may request that search engines don't provide access to cached documents.

See Also:

• Constant Field Values

CACHING_FORBIDDEN_NONE

static final String CACHING_FORBIDDEN_NONE

Show both original forbidden content and summaries (default).

See Also:

• Constant Field Values

CACHING_FORBIDDEN_ALL

static final String CACHING_FORBIDDEN_ALL

Don't show either original forbidden content or summaries.

See Also:

• Constant Field Values

CACHING_FORBIDDEN_CONTENT

static final String CACHING_FORBIDDEN_CONTENT

Don't show original forbidden content, but show summaries.

See Also:

• Constant Field Values

REPR_URL_KEY

static final String REPR_URL_KEY

See Also:

• Constant Field Values

WRITABLE_REPR_URL_KEY

static final Text WRITABLE_REPR_URL_KEY

FIXED_INTERVAL_KEY

static final String FIXED_INTERVAL_KEY

Used by AdaptiveFetchSchedule to maintain custom fetch interval

See Also:

• Constant Field Values

WRITABLE_FIXED_INTERVAL_KEY

static final Text WRITABLE_FIXED_INTERVAL_KEY

STAT_PROGRESS

static final String STAT_PROGRESS

For progress of job (programmatic / tooling).

See Also:

• Constant Field Values

CRAWL_ID_KEY

static final String CRAWL_ID_KEY

Crawl id key for programmatic jobs.

See Also:

• Constant Field Values

ARG_SEEDDIR

static final String ARG_SEEDDIR

Argument key for seed URL directory path.

See Also:

• Constant Field Values

ARG_CRAWLDB

static final String ARG_CRAWLDB

Argument key for crawldb location in programmatic jobs.

See Also:

• Constant Field Values

ARG_LINKDB

static final String ARG_LINKDB

Argument key for linkdb location in programmatic jobs.

See Also:

• Constant Field Values

VAL_RESULT

static final String VAL_RESULT

Name of the key used in the result map from NutchTool.run(java.util.Map<java.lang.String, java.lang.Object>, java.lang.String).

See Also:

• Constant Field Values

ARG_SEGMENTDIR

static final String ARG_SEGMENTDIR

Argument key for a directory of segments; similar to the -dir option in bin/nutch.

See Also:

• Constant Field Values

ARG_SEGMENTS

static final String ARG_SEGMENTS

Argument key for one segment or a list of segments (job-dependent).

See Also:

• Constant Field Values

ARG_HOSTDB

static final String ARG_HOSTDB

Argument key for hostdb location in programmatic jobs.

See Also:

• Constant Field Values

FETCH_EVENT_TITLE

static final String FETCH_EVENT_TITLE

Title key in the Pub/Sub event metadata for the title of the parsed page

See Also:

• Constant Field Values

FETCH_EVENT_CONTENTTYPE

static final String FETCH_EVENT_CONTENTTYPE

Content-type key in the Pub/Sub event metadata for the content-type of the parsed page

See Also:

• Constant Field Values

FETCH_EVENT_SCORE

static final String FETCH_EVENT_SCORE

Score key in the Pub/Sub event metadata for the score of the parsed page

See Also:

• Constant Field Values

FETCH_EVENT_FETCHTIME

static final String FETCH_EVENT_FETCHTIME

Fetch time key in the Pub/Sub event metadata for the fetch time of the parsed page

See Also:

• Constant Field Values

FETCH_EVENT_CONTENTLANG

static final String FETCH_EVENT_CONTENTLANG

Content-lanueage key in the Pub/Sub event metadata for the content-language of the parsed page

See Also:

• Constant Field Values