Package org.apache.nutch.segment
package org.apache.nutch.segment
A segment stores all data from on generate/fetch/update cycle:
fetch list, protocol status, raw content, parsed content, and extracted outgoing links.
-
ClassDescriptionAn input format that takes Nutch Content objects and converts them to text while converting newline endings to spaces.Checks whether a segment is valid, or has a certain status (generated, fetched, parsed), or can be used safely for a certain processing step (e.g., indexing).Interface used to filter segments during segment merge.This class wraps all
SegmentMergeFilterextensions in a single object so it is easier to operate on them.This tool takes several segments and merges their data together.Wraps inputs in anMetaWrapper, to permit merging different types in reduce and use additional metadata.NOTE: in selecting the latest version we rely exclusively on the segment name (not all segment data contain time information).Utility class for handling information about segment parts.Dump the content of a segment.Implements a text output format