# Index

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form

## A

AbstractChecker - Class in org.apache.nutch.util
  Scaffolding class for the various Checker implementations.
AbstractChecker() - Constructor for class org.apache.nutch.util.AbstractChecker
AbstractCommonCrawlFormat - Class in org.apache.nutch.tools
  Abstract class that implements { @see org.apache.nutch.tools.CommonCrawlFormat } interface.
AbstractCommonCrawlFormat(String, Content, Metadata, Configuration, CommonCrawlConfig) - Constructor for class org.apache.nutch.tools.AbstractCommonCrawlFormat
AbstractFetchSchedule - Class in org.apache.nutch.crawl
  This class provides common methods for implementations of `FetchSchedule`.
AbstractFetchSchedule() - Constructor for class org.apache.nutch.crawl.AbstractFetchSchedule
AbstractFetchSchedule(Configuration) - Constructor for class org.apache.nutch.crawl.AbstractFetchSchedule
AbstractScoringFilter - Class in org.apache.nutch.scoring
AbstractScoringFilter() - Constructor for class org.apache.nutch.scoring.AbstractScoringFilter
accept - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The "Accept" request header value.
accept() - Method in class org.apache.nutch.urlfilter.api.RegexRule
  Return if this rule is used for filtering-in or out.
accept(InetAddress) - Method in class org.apache.nutch.protocol.okhttp.IPFilterRules
acceptCharset - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The "Accept-Charset" request header value.
acceptLanguage - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The "Accept-Language" request header value.
ACCESS_DENIED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Access denied - authorization required, but missing/incorrect.
action - Variable in class org.apache.nutch.indexer.NutchIndexAction
AdaptiveFetchSchedule - Class in org.apache.nutch.crawl
  This class implements an adaptive re-fetch algorithm.
AdaptiveFetchSchedule() - Constructor for class org.apache.nutch.crawl.AdaptiveFetchSchedule
add(Object) - Method in class org.apache.nutch.indexer.NutchField
add(String, Object) - Method in class org.apache.nutch.indexer.NutchDocument
add(String, String) - Method in class org.apache.nutch.metadata.Metadata
  Add a metadata name/value mapping.
add(String, String) - Method in class org.apache.nutch.metadata.SpellCheckedMetadata
add(Inlink) - Method in class org.apache.nutch.crawl.Inlinks
add(Inlinks) - Method in class org.apache.nutch.crawl.Inlinks
add(NutchDocument, String, String) - Method in class org.apache.nutch.indexer.metadata.MetadataIndexer
ADD - Static variable in class org.apache.nutch.indexer.NutchIndexAction
addAll(Metadata) - Method in class org.apache.nutch.metadata.Metadata
  Add all name/value mappings (merge two metadata mappings).
addAttribute(String, String) - Method in class org.apache.nutch.plugin.Extension
  Adds a attribute and is only used until model creation at plugin system start up.
addClue(String, String) - Method in class org.apache.nutch.util.EncodingDetector
addClue(String, String, int) - Method in class org.apache.nutch.util.EncodingDetector
addDependency(String) - Method in class org.apache.nutch.plugin.PluginDescriptor
  Adds a dependency
addEventData(String, Object) - Method in class org.apache.nutch.fetcher.FetcherThreadEvent
  Add new data to the eventData object.
addExportedLibRelative(String) - Method in class org.apache.nutch.plugin.PluginDescriptor
  Adds a exported library with a relative path to the plugin directory.
addExtension(Extension) - Method in class org.apache.nutch.plugin.ExtensionPoint
  Install a coresponding extension to this extension point.
addExtension(Extension) - Method in class org.apache.nutch.plugin.PluginDescriptor
  Adds a extension.
addExtensionPoint(ExtensionPoint) - Method in class org.apache.nutch.plugin.PluginDescriptor
  Adds a extension point.
addFetchItem(Text, CrawlDatum) - Method in class org.apache.nutch.fetcher.FetchItemQueues
addFetchItem(FetchItem) - Method in class org.apache.nutch.fetcher.FetchItemQueue
addFetchItem(FetchItem) - Method in class org.apache.nutch.fetcher.FetchItemQueues
addHeader(String, Object) - Method in class org.apache.nutch.rabbitmq.RabbitMQMessage
addInProgressFetchItem(FetchItem) - Method in class org.apache.nutch.fetcher.FetchItemQueue
addMeta(String, String) - Method in class org.apache.nutch.metadata.MetaWrapper
  Add metadata.
addNotExportedLibRelative(String) - Method in class org.apache.nutch.plugin.PluginDescriptor
  Adds a non-exported library with a relative path to the plugin directory.
addOutlinksToEventData(Collection<Outlink>) - Method in class org.apache.nutch.fetcher.FetcherThreadEvent
  Given a collection of lists this method will add it the oultink metadata
addPatternBackward(String) - Method in class org.apache.nutch.util.TrieStringMatcher
  Adds any necessary nodes to the trie so that the given `String` can be decoded in reverse and the first character is represented by a terminal node.
addPatternForward(String) - Method in class org.apache.nutch.util.TrieStringMatcher
  Adds any necessary nodes to the trie so that the given `String` can be decoded and the last character is represented by a terminal node.
addRobotsContent(List<Content>, URL, Response) - Method in class org.apache.nutch.protocol.http.api.HttpRobotRulesParser
  Append `Content` of robots.txt to robotsTxtContent
addUrlFeatures(NutchDocument, String) - Method in class org.creativecommons.nutch.CCIndexingFilter
  Add the features represented by a license URL.
agentNames - Variable in class org.apache.nutch.protocol.RobotRulesParser
AJAX_URL_PART - Static variable in class org.apache.nutch.net.urlnormalizer.ajax.AjaxURLNormalizer
AjaxURLNormalizer - Class in org.apache.nutch.net.urlnormalizer.ajax
  URLNormalizer capable of dealing with AJAX URL's.
AjaxURLNormalizer() - Constructor for class org.apache.nutch.net.urlnormalizer.ajax.AjaxURLNormalizer
  Default constructor.
allowForbidden - Variable in class org.apache.nutch.protocol.http.api.HttpRobotRulesParser
allowList - Variable in class org.apache.nutch.protocol.RobotRulesParser
  set of host names or IPs to be explicitly excluded from robots.txt checking
analyze(Path) - Method in class org.apache.nutch.scoring.webgraph.LinkRank
  Runs the complete link analysis job.
AnchorIndexingFilter - Class in org.apache.nutch.indexer.anchor
  Indexing filter that offers an option to either index all inbound anchor text for a document or deduplicate anchors.
AnchorIndexingFilter() - Constructor for class org.apache.nutch.indexer.anchor.AnchorIndexingFilter
ANONYMOUS - Enum constant in enum class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter.DatabaseType
append(Node) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Append a node to the current container.
ArbitraryIndexingFilter - Class in org.apache.nutch.indexer.arbitrary
  Adds arbitrary searchable fields to a document from the class and method the user identifies in the config.
ArbitraryIndexingFilter() - Constructor for class org.apache.nutch.indexer.arbitrary.ArbitraryIndexingFilter
ArcInputFormat - Class in org.apache.nutch.tools.arc
  A input format the reads arc files.
ArcInputFormat() - Constructor for class org.apache.nutch.tools.arc.ArcInputFormat
ArcRecordReader - Class in org.apache.nutch.tools.arc
  The `ArchRecordReader` class provides a record reader which reads records from arc files.
ArcRecordReader(Configuration, FileSplit) - Constructor for class org.apache.nutch.tools.arc.ArcRecordReader
  Constructor that sets the configuration and file split.
ArcSegmentCreator - Class in org.apache.nutch.tools.arc
  The `ArcSegmentCreator` is a replacement for fetcher that will take arc files as input and produce a nutch segment as output.
ArcSegmentCreator() - Constructor for class org.apache.nutch.tools.arc.ArcSegmentCreator
ArcSegmentCreator(Configuration) - Constructor for class org.apache.nutch.tools.arc.ArcSegmentCreator
  Constructor that sets the job configuration.
ArcSegmentCreator.ArcSegmentCreatorMapper - Class in org.apache.nutch.tools.arc
ArcSegmentCreatorMapper() - Constructor for class org.apache.nutch.tools.arc.ArcSegmentCreator.ArcSegmentCreatorMapper
areAvailableExchanges() - Method in class org.apache.nutch.exchange.Exchanges
ARG_CRAWLDB - Static variable in interface org.apache.nutch.metadata.Nutch
  Argument key for crawldb location in programmatic jobs.
ARG_HOSTDB - Static variable in interface org.apache.nutch.metadata.Nutch
  Argument key for hostdb location in programmatic jobs.
ARG_LINKDB - Static variable in interface org.apache.nutch.metadata.Nutch
  Argument key for linkdb location in programmatic jobs.
ARG_SEEDDIR - Static variable in interface org.apache.nutch.metadata.Nutch
  Argument key for seed URL directory path.
ARG_SEGMENTDIR - Static variable in interface org.apache.nutch.metadata.Nutch
  Argument key for a directory of segments; similar to the -dir option in bin/nutch.
ARG_SEGMENTS - Static variable in interface org.apache.nutch.metadata.Nutch
  Argument key for one segment or a list of segments (job-dependent).
args - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
ASN - Enum constant in enum class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter.DatabaseType
attrName - Variable in class org.apache.nutch.parse.html.DOMContentUtils.LinkParams
AUTH_HEADER_NAME - Static variable in interface org.apache.nutch.indexwriter.solr.SolrConstants
AUTH_HEADER_VALUE - Static variable in interface org.apache.nutch.indexwriter.solr.SolrConstants
autoDetectClues(Content, boolean) - Method in class org.apache.nutch.util.EncodingDetector
AutomatonURLFilter - Class in org.apache.nutch.urlfilter.automaton
  RegexURLFilterBase implementation based on the dk.brics.automaton Finite-State Automata for JavaTM.
AutomatonURLFilter() - Constructor for class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
AutomatonURLFilter(String) - Constructor for class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
autoResolveContentType(String, String, byte[]) - Method in class org.apache.nutch.util.MimeUtil
  A facade interface to trying all the possible mime type resolution strategies available within Tika.

## B

BasicIndexingFilter - Class in org.apache.nutch.indexer.basic
  Adds basic searchable fields to a document.
BasicIndexingFilter() - Constructor for class org.apache.nutch.indexer.basic.BasicIndexingFilter
BasicURLNormalizer - Class in org.apache.nutch.net.urlnormalizer.basic
  Converts URLs to a normal form: remove dot segments in path: `/./` or `/../` remove default ports, e.g.
BasicURLNormalizer() - Constructor for class org.apache.nutch.net.urlnormalizer.basic.BasicURLNormalizer
BATCH_DUMP - Static variable in interface org.apache.nutch.indexwriter.cloudsearch.CloudSearchConstants
bind(String, String, String, String, String, String) - Method in class org.apache.nutch.rabbitmq.RabbitMQClient
  Creates a relationship between an exchange and a queue.
BLOCKED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Deprecated.
BlockedException - Exception in org.apache.nutch.protocol.http.api
BlockedException(String) - Constructor for exception org.apache.nutch.protocol.http.api.BlockedException
booleanValue() - Method in class org.apache.nutch.protocol.okhttp.OkHttpResponse.TruncatedContent
buffer - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
BUFFER_SIZE - Static variable in class org.apache.nutch.protocol.http.api.HttpBase
BULK_CLOSE_TIMEOUT - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
BULK_CLOSE_TIMEOUT - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
bulkProcessorListener() - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
  Generates a default BulkProcessor.Listener
bulkProcessorListener() - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
  Generates a default BulkProcessor.Listener
bytes - Variable in class org.apache.nutch.indexwriter.csv.CSVIndexWriter.Separator

## C

CACHE - Static variable in class org.apache.nutch.protocol.RobotRulesParser
CACHING_FORBIDDEN_ALL - Static variable in interface org.apache.nutch.metadata.Nutch
  Don't show either original forbidden content or summaries.
CACHING_FORBIDDEN_CONTENT - Static variable in interface org.apache.nutch.metadata.Nutch
  Don't show original forbidden content, but show summaries.
CACHING_FORBIDDEN_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
  Sites may request that search engines don't provide access to cached documents.
CACHING_FORBIDDEN_NONE - Static variable in interface org.apache.nutch.metadata.Nutch
  Show both original forbidden content and summaries (default).
calculate(Content, Parse) - Method in class org.apache.nutch.crawl.MD5Signature
calculate(Content, Parse) - Method in class org.apache.nutch.crawl.Signature
calculate(Content, Parse) - Method in class org.apache.nutch.crawl.TextMD5Signature
calculate(Content, Parse) - Method in class org.apache.nutch.crawl.TextProfileSignature
calculateLastFetchTime(CrawlDatum) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
  This method return the last fetch time of the CrawlDatum
calculateLastFetchTime(CrawlDatum) - Method in interface org.apache.nutch.crawl.FetchSchedule
  Calculates last fetch time of the given CrawlDatum.
CaseInsensitiveMetadata - Class in org.apache.nutch.metadata
  A decorator to Metadata that adds for case-insensitive lookup of keys.
CaseInsensitiveMetadata() - Constructor for class org.apache.nutch.metadata.CaseInsensitiveMetadata
  Constructs a new, empty metadata.
categorize(Throwable) - Static method in class org.apache.nutch.metrics.ErrorTracker
  Categorizes a throwable into an error type.
CCIndexingFilter - Class in org.creativecommons.nutch
  Adds basic searchable fields to a document.
CCIndexingFilter() - Constructor for class org.creativecommons.nutch.CCIndexingFilter
CCParseFilter - Class in org.creativecommons.nutch
  Adds metadata identifying the Creative Commons license used, if any.
CCParseFilter() - Constructor for class org.creativecommons.nutch.CCParseFilter
CCParseFilter.Walker - Class in org.creativecommons.nutch
  Walks DOM tree, looking for RDF in comments and licenses in anchors.
cdata(char[], int, int) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive notification of cdata.
CHAR_ENCODING_FOR_CONVERSION - Static variable in interface org.apache.nutch.metadata.Nutch
characters(char[], int, int) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive notification of character data.
charactersRaw(char[], int, int) - Method in class org.apache.nutch.parse.html.DOMBuilder
  If available, when the disable-output-escaping attribute is used, output raw text without escaping.
chars - Variable in class org.apache.nutch.indexwriter.csv.CSVIndexWriter.Separator
CHARSET_UTF8 - Static variable in class org.apache.nutch.parse.feed.FeedParser
checkAndReplace(String, String) - Method in class org.apache.nutch.indexer.replace.FieldReplacer
  Return a replacement value for a field.
checkAny - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
checkClientTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.htmlunit.DummyX509TrustManager
checkClientTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.http.DummyX509TrustManager
checkClientTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
checkClientTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.interactiveselenium.DummyX509TrustManager
checkClientTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.selenium.DummyX509TrustManager
checkExceptionThreshold(String) - Method in class org.apache.nutch.fetcher.FetchItemQueues
  Increment the exception counter of a queue in case of an exception e.g.
checkExceptionThreshold(String, int, long) - Method in class org.apache.nutch.fetcher.FetchItemQueues
  Increment the exception counter of a queue in case of an exception e.g.
checkFailed - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
checkKnown - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
checkNew - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
checkOutputSpecs(JobContext) - Method in class org.apache.nutch.fetcher.FetcherOutputFormat
checkOutputSpecs(JobContext) - Method in class org.apache.nutch.indexer.IndexerOutputFormat
checkOutputSpecs(JobContext) - Method in class org.apache.nutch.parse.ParseOutputFormat
checkQueueMode(String) - Static method in class org.apache.nutch.fetcher.FetchItemQueues
  Check whether queue mode is valid, fall-back to default mode if not.
checkRobotsTxt - Variable in class org.apache.nutch.indexer.IndexingFiltersChecker
checkRobotsTxt - Variable in class org.apache.nutch.parse.ParserChecker
checkSegmentDir(Path, FileSystem) - Static method in class org.apache.nutch.segment.SegmentChecker
  Check the segment to see if it is valid based on the sub directories.
checkServerTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.htmlunit.DummyX509TrustManager
checkServerTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.http.DummyX509TrustManager
checkServerTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
checkServerTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.interactiveselenium.DummyX509TrustManager
checkServerTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.selenium.DummyX509TrustManager
checkTimelimit() - Method in class org.apache.nutch.fetcher.FetchItemQueues
childLen - Variable in class org.apache.nutch.parse.html.DOMContentUtils.LinkParams
children - Variable in class org.apache.nutch.util.TrieStringMatcher.TrieNode
childrenList - Variable in class org.apache.nutch.util.TrieStringMatcher.TrieNode
chooseRepr(String, String, boolean) - Static method in class org.apache.nutch.util.URLUtil
  Given two urls, a src and a destination of a redirect, it returns the representative url.
CIDR - Class in org.apache.nutch.protocol.okhttp
  Parse a CIDR block notation and test whether an IP address is contained in the subnet range defined by the CIDR.
CIDR(String) - Constructor for class org.apache.nutch.protocol.okhttp.CIDR
CIDR(InetAddress, int) - Constructor for class org.apache.nutch.protocol.okhttp.CIDR
CIPHER_SUITES - Static variable in interface org.apache.nutch.net.protocols.Response
  Key to hold the SSL/TLS cipher suites `store.protocol.versions` is true.
CircularDependencyException - Exception in org.apache.nutch.plugin
  `CircularDependencyException` will be thrown if a circular dependency is detected.
CircularDependencyException(String) - Constructor for exception org.apache.nutch.plugin.CircularDependencyException
CircularDependencyException(Throwable) - Constructor for exception org.apache.nutch.plugin.CircularDependencyException
CITY - Enum constant in enum class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter.DatabaseType
CLASSIC - Enum constant in enum class org.apache.nutch.scoring.similarity.util.LuceneTokenizer.TokenizerType
classify(String) - Static method in class org.apache.nutch.parsefilter.naivebayes.Classify
classify(String) - Method in class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
Classify - Class in org.apache.nutch.parsefilter.naivebayes
Classify() - Constructor for class org.apache.nutch.parsefilter.naivebayes.Classify
cleanField(String) - Static method in class org.apache.nutch.util.StringUtil
  Simple character substitution which cleans/removes all � chars from a given String.
CLEANING_DELETED_DOCUMENTS_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents deleted during cleaning.
CleaningJob - Class in org.apache.nutch.indexer
  The class scans CrawlDB looking for entries with status DB_GONE (404) or DB_DUPLICATE and sends delete requests to indexers for those documents.
CleaningJob() - Constructor for class org.apache.nutch.indexer.CleaningJob
CleaningJob.DBFilter - Class in org.apache.nutch.indexer
CleaningJob.DeleterReducer - Class in org.apache.nutch.indexer
cleanMimeType(String) - Static method in class org.apache.nutch.util.MimeUtil
  Cleans a `MimeType` name by removing out the actual `MimeType`, from a string of the form:
cleanup(Mapper.Context) - Method in class org.apache.nutch.parse.ParseSegment.ParseSegmentMapper
cleanup(Reducer.Context) - Method in class org.apache.nutch.indexer.CleaningJob.DeleterReducer
cleanup(Reducer.Context) - Method in class org.apache.nutch.crawl.Generator.SelectorReducer
cleanup(Reducer.Context) - Method in class org.apache.nutch.hostdb.UpdateHostDbReducer
  Shut down all running threads and wait for completion.
cleanup(Reducer.Context) - Method in class org.apache.nutch.indexer.IndexerMapReduce.IndexerReducer
cleanupAfterFailure(Path, FileSystem) - Static method in class org.apache.nutch.util.NutchJob
  Clean up the file system in case of a job failure.
cleanupAfterFailure(Path, Path, FileSystem) - Static method in class org.apache.nutch.util.NutchJob
  Clean up the file system in case of a job failure.
cleanUpDriver(WebDriver) - Static method in class org.apache.nutch.protocol.htmlunit.HtmlUnitWebDriver
cleanUpDriver(WebDriver) - Static method in class org.apache.nutch.protocol.selenium.HttpWebClient
clear() - Method in class org.apache.nutch.crawl.Inlinks
clear() - Method in class org.apache.nutch.metadata.Metadata
  Remove all mappings from metadata.
clearClues() - Method in class org.apache.nutch.util.EncodingDetector
  Clears all clues.
Client - Class in org.apache.nutch.protocol.ftp
  Client.java encapsulates functionalities necessary for nutch to get dir list and retrieve file from an FTP server.
Client() - Constructor for class org.apache.nutch.protocol.ftp.Client
  Public default constructor
CLIENT_TRANSFER_ENCODING - Static variable in interface org.apache.nutch.metadata.HttpHeaders
clone() - Method in class org.apache.nutch.crawl.CrawlDatum
clone() - Method in class org.apache.nutch.hostdb.HostDatum
clone() - Method in class org.apache.nutch.indexer.NutchDocument
clone() - Method in class org.apache.nutch.indexer.NutchField
close() - Method in class org.apache.nutch.crawl.CrawlDbReader
close() - Method in class org.apache.nutch.crawl.LinkDbReader
close() - Method in class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter
close() - Method in interface org.apache.nutch.indexer.IndexWriter
close() - Method in class org.apache.nutch.indexer.IndexWriters
close() - Method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
close() - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
close() - Method in class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
close() - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
close() - Method in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
close() - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
close() - Method in class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
close() - Method in class org.apache.nutch.indexwriter.solr.SolrIndexWriter
close() - Method in class org.apache.nutch.rabbitmq.RabbitMQClient
  Closes the channel and the connection with the server.
close() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
close() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
  Closes the record reader resources.
close() - Method in class org.apache.nutch.tools.arc.ArcSegmentCreator
close() - Method in interface org.apache.nutch.tools.CommonCrawlFormat
  Optional method that could be implemented if the actual format needs some close procedure.
close() - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
close(TaskAttemptContext) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumCsvOutputFormat.LineRecordWriter
close(TaskAttemptContext) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumJsonOutputFormat.LineRecordWriter
closeArray(String, boolean, boolean) - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
closeArray(String, boolean, boolean) - Method in class org.apache.nutch.tools.CommonCrawlFormatJackson
closeArray(String, boolean, boolean) - Method in class org.apache.nutch.tools.CommonCrawlFormatJettinson
closeArray(String, boolean, boolean) - Method in class org.apache.nutch.tools.CommonCrawlFormatSimple
closeArray(String, boolean, boolean) - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
closeObject(String) - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
closeObject(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatJackson
closeObject(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatJettinson
closeObject(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatSimple
closeObject(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
closeReaders(MapFile.Reader[]) - Static method in class org.apache.nutch.util.FSUtils
  Closes a group of MapFile readers.
closeReaders(SequenceFile.Reader[]) - Static method in class org.apache.nutch.util.FSUtils
  Closes a group of SequenceFile readers.
CloudSearchConstants - Interface in org.apache.nutch.indexwriter.cloudsearch
CloudSearchIndexWriter - Class in org.apache.nutch.indexwriter.cloudsearch
  Writes documents to CloudSearch.
CloudSearchIndexWriter() - Constructor for class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
CloudSearchUtils - Class in org.apache.nutch.indexwriter.cloudsearch
CloudSearchUtils() - Constructor for class org.apache.nutch.indexwriter.cloudsearch.CloudSearchUtils
COLLECTION - Static variable in interface org.apache.nutch.indexwriter.solr.SolrConstants
CollectionManager - Class in org.apache.nutch.collection
CollectionManager() - Constructor for class org.apache.nutch.collection.CollectionManager
  Used for testing
CollectionManager(Configuration) - Constructor for class org.apache.nutch.collection.CollectionManager
COLONSP - Static variable in class org.apache.nutch.tools.WARCUtils
CommandRunner - Class in org.apache.nutch.util
CommandRunner() - Constructor for class org.apache.nutch.util.CommandRunner
comment(char[], int, int) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Report an XML comment anywhere in the document.
commit() - Method in interface org.apache.nutch.indexer.IndexWriter
commit() - Method in class org.apache.nutch.indexer.IndexWriters
commit() - Method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
commit() - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
  (nothing to commit)
commit() - Method in class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
commit() - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
commit() - Method in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
commit() - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
commit() - Method in class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
commit() - Method in class org.apache.nutch.indexwriter.solr.SolrIndexWriter
COMMIT_SIZE - Static variable in interface org.apache.nutch.indexwriter.solr.SolrConstants
CommonCrawlConfig - Class in org.apache.nutch.tools
CommonCrawlConfig() - Constructor for class org.apache.nutch.tools.CommonCrawlConfig
  Default constructor
CommonCrawlConfig(InputStream) - Constructor for class org.apache.nutch.tools.CommonCrawlConfig
CommonCrawlDataDumper - Class in org.apache.nutch.tools
  The Common Crawl Data Dumper tool enables one to reverse generate the raw content from Nutch segment data directories into a common crawling data format, consumed by many applications.
CommonCrawlDataDumper() - Constructor for class org.apache.nutch.tools.CommonCrawlDataDumper
  Constructor
CommonCrawlDataDumper(CommonCrawlConfig) - Constructor for class org.apache.nutch.tools.CommonCrawlDataDumper
  Configurable constructor
CommonCrawlFormat - Interface in org.apache.nutch.tools
  Interface for all CommonCrawl formatter.
CommonCrawlFormatFactory - Class in org.apache.nutch.tools
  Factory class that creates new `CommonCrawlFormat` objects (a.k.a.
CommonCrawlFormatFactory() - Constructor for class org.apache.nutch.tools.CommonCrawlFormatFactory
CommonCrawlFormatJackson - Class in org.apache.nutch.tools
  This class provides methods to map crawled data on JSON using Jackson Streaming APIs.
CommonCrawlFormatJackson(String, Content, Metadata, Configuration, CommonCrawlConfig) - Constructor for class org.apache.nutch.tools.CommonCrawlFormatJackson
CommonCrawlFormatJackson(Configuration, CommonCrawlConfig) - Constructor for class org.apache.nutch.tools.CommonCrawlFormatJackson
CommonCrawlFormatJettinson - Class in org.apache.nutch.tools
  This class provides methods to map crawled data on JSON using Jettinson APIs.
CommonCrawlFormatJettinson(String, Content, Metadata, Configuration, CommonCrawlConfig) - Constructor for class org.apache.nutch.tools.CommonCrawlFormatJettinson
CommonCrawlFormatSimple - Class in org.apache.nutch.tools
  This class provides methods to map crawled data on JSON using a StringBuilder object.
CommonCrawlFormatSimple(String, Content, Metadata, Configuration, CommonCrawlConfig) - Constructor for class org.apache.nutch.tools.CommonCrawlFormatSimple
CommonCrawlFormatWARC - Class in org.apache.nutch.tools
CommonCrawlFormatWARC(String, Content, Metadata, Configuration, CommonCrawlConfig, ParseData) - Constructor for class org.apache.nutch.tools.CommonCrawlFormatWARC
CommonCrawlFormatWARC(Configuration, CommonCrawlConfig) - Constructor for class org.apache.nutch.tools.CommonCrawlFormatWARC
Comparator() - Constructor for class org.apache.nutch.crawl.CrawlDatum.Comparator
compare(byte[], int, int, byte[], int, int) - Method in class org.apache.nutch.crawl.CrawlDatum.Comparator
compare(byte[], int, int, byte[], int, int) - Method in class org.apache.nutch.crawl.Generator.DecreasingFloatComparator
  Compares two FloatWritables decreasing.
compare(byte[], int, int, byte[], int, int) - Method in class org.apache.nutch.crawl.Generator.HashComparator
compare(Object, Object) - Method in class org.apache.nutch.crawl.SignatureComparator
compare(WritableComparable, WritableComparable) - Method in class org.apache.nutch.crawl.Generator.HashComparator
compareOrder - Variable in class org.apache.nutch.crawl.DeduplicationJob.DedupReducer
compareTo(CrawlDatum) - Method in class org.apache.nutch.crawl.CrawlDatum
  Sort two `CrawlDatum` objects by decreasing score.
compareTo(TrieStringMatcher.TrieNode) - Method in class org.apache.nutch.util.TrieStringMatcher.TrieNode
computeCosineSimilarity(DocVector) - Static method in class org.apache.nutch.scoring.similarity.cosine.Model
conf - Variable in class org.apache.nutch.crawl.Signature
conf - Variable in class org.apache.nutch.hostdb.FetchOverdueCrawlDatumProcessor
conf - Variable in class org.apache.nutch.plugin.Plugin
conf - Variable in class org.apache.nutch.protocol.RobotRulesParser
conf - Variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
conf - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
CONFORMS_TO - Static variable in class org.apache.nutch.tools.WARCUtils
CONNECTION - Enum constant in enum class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter.DatabaseType
connectionFailures - Variable in class org.apache.nutch.hostdb.HostDatum
contains(InetAddress) - Method in class org.apache.nutch.protocol.okhttp.CIDR
containsWord(String, ArrayList<String>) - Method in class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
content - Variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
Content - Class in org.apache.nutch.protocol
Content() - Constructor for class org.apache.nutch.protocol.Content
Content(String, String, byte[], String, Metadata, Configuration) - Constructor for class org.apache.nutch.protocol.Content
Content(String, String, byte[], String, Metadata, MimeUtil) - Constructor for class org.apache.nutch.protocol.Content
CONTENT_DISPOSITION - Static variable in interface org.apache.nutch.metadata.HttpHeaders
CONTENT_ENCODING - Static variable in interface org.apache.nutch.metadata.HttpHeaders
CONTENT_LANGUAGE - Static variable in interface org.apache.nutch.metadata.HttpHeaders
CONTENT_LENGTH - Static variable in interface org.apache.nutch.metadata.HttpHeaders
CONTENT_LOCATION - Static variable in interface org.apache.nutch.metadata.HttpHeaders
CONTENT_MD5 - Static variable in interface org.apache.nutch.metadata.HttpHeaders
CONTENT_REDIR - Static variable in class org.apache.nutch.fetcher.Fetcher
CONTENT_TYPE - Static variable in interface org.apache.nutch.metadata.HttpHeaders
ContentAsTextInputFormat - Class in org.apache.nutch.segment
  An input format that takes Nutch Content objects and converts them to text while converting newline endings to spaces.
ContentAsTextInputFormat() - Constructor for class org.apache.nutch.segment.ContentAsTextInputFormat
context - Variable in class org.apache.nutch.hostdb.ResolverThread
CONTRIBUTOR - Static variable in interface org.apache.nutch.metadata.DublinCore
  An entity responsible for making contributions to the content of the resource.
convertIDNA2003(String, boolean, boolean) - Static method in class org.apache.nutch.util.URLUtil
  Convert IDN host to ASCII or Unicode using Java's built-in `IDN` class.
convertIDNA2008(String, boolean) - Static method in class org.apache.nutch.util.URLUtil
  Convert IDN host to ASCII or Unicode using ICU's `IDNA` class.
COOKIE - Static variable in class org.apache.nutch.protocol.http.api.HttpBase
CosineSimilarity - Class in org.apache.nutch.scoring.similarity.cosine
CosineSimilarity() - Constructor for class org.apache.nutch.scoring.similarity.cosine.CosineSimilarity
count(CrawlDatum) - Method in interface org.apache.nutch.hostdb.CrawlDatumProcessor
  Process a single crawl datum instance to aggregate custom counts.
count(CrawlDatum) - Method in class org.apache.nutch.hostdb.FetchOverdueCrawlDatumProcessor
COUNTRY - Enum constant in enum class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter.DatabaseType
COVERAGE - Static variable in interface org.apache.nutch.metadata.DublinCore
  The extent or scope of the content of the resource.
CRAWL_ID_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
  Crawl id key for programmatic jobs.
CrawlCompletionStats - Class in org.apache.nutch.util
  Extracts some simple crawl completion stats from the crawldb Stats will be sorted by host/domain and will be of the form: 1 www.spitzer.caltech.edu FETCHED 50 www.spitzer.caltech.edu UNFETCHED
CrawlCompletionStats() - Constructor for class org.apache.nutch.util.CrawlCompletionStats
CrawlCompletionStats.CrawlCompletionStatsCombiner - Class in org.apache.nutch.util
CrawlCompletionStatsCombiner() - Constructor for class org.apache.nutch.util.CrawlCompletionStats.CrawlCompletionStatsCombiner
crawlDatum - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
CrawlDatum - Class in org.apache.nutch.crawl
CrawlDatum() - Constructor for class org.apache.nutch.crawl.CrawlDatum
CrawlDatum(int, int) - Constructor for class org.apache.nutch.crawl.CrawlDatum
CrawlDatum(int, int, float) - Constructor for class org.apache.nutch.crawl.CrawlDatum
CrawlDatum.Comparator - Class in org.apache.nutch.crawl
  A Comparator optimized for CrawlDatum.
CrawlDatumCsvOutputFormat() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumCsvOutputFormat
CrawlDatumJsonOutputFormat() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumJsonOutputFormat
CrawlDatumProcessor - Interface in org.apache.nutch.hostdb
  These are instantiated once for each host.
crawlDatumProcessors - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
crawlDb - Variable in class org.apache.nutch.crawl.CrawlDbReader
CrawlDb - Class in org.apache.nutch.crawl
  This class takes the output of the fetcher and updates the crawldb accordingly.
CrawlDb() - Constructor for class org.apache.nutch.crawl.CrawlDb
CrawlDb(Configuration) - Constructor for class org.apache.nutch.crawl.CrawlDb
CRAWLDB_ADDITIONS_ALLOWED - Static variable in class org.apache.nutch.crawl.CrawlDb
CRAWLDB_GONE_RECORDS_REMOVED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Gone (404) records removed during CrawlDb operations.
CRAWLDB_ORPHAN_RECORDS_REMOVED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Orphan records removed during CrawlDb operations.
CRAWLDB_PURGE_404 - Static variable in class org.apache.nutch.crawl.CrawlDb
CRAWLDB_PURGE_ORPHANS - Static variable in class org.apache.nutch.crawl.CrawlDb
CRAWLDB_URLS_FILTERED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs filtered during CrawlDb operations.
CrawlDbDumpMapper() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDbDumpMapper
CrawlDbFilter - Class in org.apache.nutch.crawl
  This class provides a way to separate the URL normalization and filtering steps from the rest of CrawlDb manipulation code.
CrawlDbFilter() - Constructor for class org.apache.nutch.crawl.CrawlDbFilter
CrawlDbMerger - Class in org.apache.nutch.crawl
  This tool merges several CrawlDb-s into one, optionally filtering URLs through the current URLFilters, to skip prohibited pages.
CrawlDbMerger() - Constructor for class org.apache.nutch.crawl.CrawlDbMerger
CrawlDbMerger(Configuration) - Constructor for class org.apache.nutch.crawl.CrawlDbMerger
CrawlDbMerger.Merger - Class in org.apache.nutch.crawl
CrawlDbReader - Class in org.apache.nutch.crawl
  Read utility for the CrawlDB.
CrawlDbReader() - Constructor for class org.apache.nutch.crawl.CrawlDbReader
CrawlDbReader.CrawlDatumCsvOutputFormat - Class in org.apache.nutch.crawl
CrawlDbReader.CrawlDatumCsvOutputFormat.LineRecordWriter - Class in org.apache.nutch.crawl
CrawlDbReader.CrawlDatumJsonOutputFormat - Class in org.apache.nutch.crawl
CrawlDbReader.CrawlDatumJsonOutputFormat.LineRecordWriter - Class in org.apache.nutch.crawl
CrawlDbReader.CrawlDatumJsonOutputFormat.WritableSerializer - Class in org.apache.nutch.crawl
CrawlDbReader.CrawlDbDumpMapper - Class in org.apache.nutch.crawl
CrawlDbReader.CrawlDbStatMapper - Class in org.apache.nutch.crawl
CrawlDbReader.CrawlDbStatReducer - Class in org.apache.nutch.crawl
CrawlDbReader.CrawlDbTopNMapper - Class in org.apache.nutch.crawl
CrawlDbReader.CrawlDbTopNReducer - Class in org.apache.nutch.crawl
CrawlDbReader.JsonIndenter - Class in org.apache.nutch.crawl
CrawlDbReducer - Class in org.apache.nutch.crawl
  Merge new page entries with existing entries.
CrawlDbReducer() - Constructor for class org.apache.nutch.crawl.CrawlDbReducer
CrawlDbStatMapper() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatMapper
CrawlDbStatReducer() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatReducer
CrawlDbTopNMapper() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNMapper
CrawlDbTopNReducer() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNReducer
CrawlDbUpdateMapper() - Constructor for class org.apache.nutch.crawl.Generator.CrawlDbUpdater.CrawlDbUpdateMapper
CrawlDbUpdater() - Constructor for class org.apache.nutch.crawl.Generator.CrawlDbUpdater
CrawlDbUpdateReducer() - Constructor for class org.apache.nutch.crawl.Generator.CrawlDbUpdater.CrawlDbUpdateReducer
create() - Static method in class org.apache.nutch.util.NutchConfiguration
  Create a `Configuration` for Nutch.
create(boolean, Properties) - Static method in class org.apache.nutch.util.NutchConfiguration
  Create a `Configuration` from supplied properties.
create(Text, CrawlDatum, String) - Static method in class org.apache.nutch.fetcher.FetchItem
  Create an item.
create(Text, CrawlDatum, String, int) - Static method in class org.apache.nutch.fetcher.FetchItem
  Create an item.
createChromeRemoteWebDriver(URL, boolean) - Static method in class org.apache.nutch.protocol.selenium.HttpWebClient
createChromeWebDriver(String, boolean) - Static method in class org.apache.nutch.protocol.selenium.HttpWebClient
createComponents(String) - Method in class org.apache.nutch.scoring.similarity.util.LuceneAnalyzerUtil
createDefaultRemoteWebDriver(URL, boolean) - Static method in class org.apache.nutch.protocol.selenium.HttpWebClient
createDocFromAnonymousIpDb(String, NutchDocument, DatabaseReader) - Static method in class org.apache.nutch.indexer.geoip.GeoIPDocumentCreator
  Populate a `NutchDocument` based on lookup of IP in Anonymous IP database.
createDocFromAsnDb(String, NutchDocument, DatabaseReader) - Static method in class org.apache.nutch.indexer.geoip.GeoIPDocumentCreator
  Populate a `NutchDocument` based on lookup of IP in ASN database.
createDocFromCityDb(String, NutchDocument, DatabaseReader) - Static method in class org.apache.nutch.indexer.geoip.GeoIPDocumentCreator
  Populate a `NutchDocument` based on lookup of IP in City database.
createDocFromConnectionDb(String, NutchDocument, DatabaseReader) - Static method in class org.apache.nutch.indexer.geoip.GeoIPDocumentCreator
  Populate a `NutchDocument` based on lookup of IP in ConnectionDb.
createDocFromCountryDb(String, NutchDocument, DatabaseReader) - Static method in class org.apache.nutch.indexer.geoip.GeoIPDocumentCreator
  Populate a `NutchDocument` based on lookup of IP in Country database.
createDocFromDomainDb(String, NutchDocument, DatabaseReader) - Static method in class org.apache.nutch.indexer.geoip.GeoIPDocumentCreator
  Populate a `NutchDocument` based on lookup of IP in Domain database.
createDocFromInsightsService(String, NutchDocument, WebServiceClient) - Static method in class org.apache.nutch.indexer.geoip.GeoIPDocumentCreator
  Populate a `NutchDocument` based on lookup of IP using the MaxMind Insights web service.
createDocFromIspDb(String, NutchDocument, DatabaseReader) - Static method in class org.apache.nutch.indexer.geoip.GeoIPDocumentCreator
  Populate a `NutchDocument` based on lookup of IP in ISP database.
createDocVector(String, int, int) - Static method in class org.apache.nutch.scoring.similarity.cosine.Model
  Used to create a DocVector from given String text.
createFileName(String, String, String) - Static method in class org.apache.nutch.util.DumpFileUtil
createFileNameFromUrl(String, String, String, String, String, boolean) - Static method in class org.apache.nutch.util.DumpFileUtil
createFirefoxRemoteWebDriver(URL, boolean) - Static method in class org.apache.nutch.protocol.selenium.HttpWebClient
createFirefoxWebDriver(String, boolean) - Static method in class org.apache.nutch.protocol.selenium.HttpWebClient
createJob(Configuration, Path) - Static method in class org.apache.nutch.crawl.CrawlDb
createKey() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
  Creates a new instance of the `Text` object for the key.
createLockFile(Configuration, Path, boolean) - Static method in class org.apache.nutch.util.LockUtil
  Create a lock file.
createLockFile(FileSystem, Path, boolean) - Static method in class org.apache.nutch.util.LockUtil
  Create a lock file.
createMergeJob(Configuration, Path, boolean, boolean) - Static method in class org.apache.nutch.crawl.CrawlDbMerger
createMergeJob(Configuration, Path, boolean, boolean) - Static method in class org.apache.nutch.crawl.LinkDbMerger
createModel(Configuration) - Static method in class org.apache.nutch.scoring.similarity.cosine.Model
createParseResult(String, Parse) - Static method in class org.apache.nutch.parse.ParseResult
  Convenience method for obtaining `ParseResult` from a single `Parse` output.
createRandomRemoteWebDriver(URL, boolean) - Static method in class org.apache.nutch.protocol.selenium.HttpWebClient
createRecordReader(InputSplit, TaskAttemptContext) - Method in class org.apache.nutch.segment.SegmentMerger.ObjectInputFormat
createRecordReader(InputSplit, TaskAttemptContext) - Method in class org.apache.nutch.tools.arc.ArcInputFormat
createRule(boolean, String) - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
  Creates a new `RegexRule`.
createRule(boolean, String) - Method in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
createRule(boolean, String) - Method in class org.apache.nutch.urlfilter.regex.RegexURLFilter
createRule(boolean, String, String) - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
  Creates a new `RegexRule`.
createRule(boolean, String, String) - Method in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
createRule(boolean, String, String) - Method in class org.apache.nutch.urlfilter.regex.RegexURLFilter
createSegments(Path, Path) - Method in class org.apache.nutch.tools.arc.ArcSegmentCreator
  Creates the arc files to segments job.
createSocket(String, int) - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
createSocket(String, int, InetAddress, int) - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
createSocket(String, int, InetAddress, int, HttpConnectionParams) - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
  Attempts to get a new socket connection to the given host within the given time limit.
createSocket(Socket, String, int, boolean) - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
createSubCollection(String, String) - Method in class org.apache.nutch.collection.CollectionManager
  Create a new subcollection.
createTwoLevelsDirectory(String, String) - Static method in class org.apache.nutch.util.DumpFileUtil
createTwoLevelsDirectory(String, String, boolean) - Static method in class org.apache.nutch.util.DumpFileUtil
createURLStreamHandler(String) - Method in class org.apache.nutch.plugin.PluginRepository
  Invoked whenever a `URL` needs to be instantiated.
createURLStreamHandler(String) - Method in class org.apache.nutch.plugin.URLStreamHandlerFactory
createValue() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
  Creates a new instance of the `BytesWritable` object for the key
createWebGraph(Path, Path[], boolean, boolean) - Method in class org.apache.nutch.scoring.webgraph.WebGraph
  Creates the three different WebGraph databases, Outlinks, Inlinks, and Node.
CreativeCommons - Interface in org.apache.nutch.metadata
  A collection of Creative Commons properties names.
CREATOR - Static variable in interface org.apache.nutch.metadata.DublinCore
  An entity primarily responsible for making the content of the resource.
CRLF - Static variable in class org.apache.nutch.tools.WARCUtils
CSV_CHARSET - Static variable in interface org.apache.nutch.indexwriter.csv.CSVConstants
CSV_ESCAPECHARACTER - Static variable in interface org.apache.nutch.indexwriter.csv.CSVConstants
CSV_FIELD_SEPARATOR - Static variable in interface org.apache.nutch.indexwriter.csv.CSVConstants
CSV_FIELDS - Static variable in interface org.apache.nutch.indexwriter.csv.CSVConstants
CSV_MAXFIELDLENGTH - Static variable in interface org.apache.nutch.indexwriter.csv.CSVConstants
CSV_MAXFIELDVALUES - Static variable in interface org.apache.nutch.indexwriter.csv.CSVConstants
CSV_OUTPATH - Static variable in interface org.apache.nutch.indexwriter.csv.CSVConstants
CSV_QUOTECHARACTER - Static variable in interface org.apache.nutch.indexwriter.csv.CSVConstants
CSV_VALUESEPARATOR - Static variable in interface org.apache.nutch.indexwriter.csv.CSVConstants
CSV_WITHHEADER - Static variable in interface org.apache.nutch.indexwriter.csv.CSVConstants
CSVConstants - Interface in org.apache.nutch.indexwriter.csv
CSVIndexWriter - Class in org.apache.nutch.indexwriter.csv
  Write Nutch documents to a CSV file (comma separated values), i.e., dump index as CSV or tab-separated plain text table.
CSVIndexWriter() - Constructor for class org.apache.nutch.indexwriter.csv.CSVIndexWriter
CSVIndexWriter.Separator - Class in org.apache.nutch.indexwriter.csv
  represent separators (also quote and escape characters) as char(s) and byte(s) in the output encoding for efficiency.
csvout - Variable in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
CURRENT_NAME - Static variable in class org.apache.nutch.crawl.CrawlDb
CURRENT_NAME - Static variable in class org.apache.nutch.crawl.LinkDb
CURRENT_NAME - Static variable in class org.apache.nutch.util.SitemapProcessor
currentJob - Variable in class org.apache.nutch.util.NutchTool
currentJobNum - Variable in class org.apache.nutch.util.NutchTool

## D

DATE - Static variable in interface org.apache.nutch.metadata.DublinCore
  A date associated with an event in the life cycle of the resource.
dateFormatStr - Static variable in class org.apache.nutch.indexer.feed.FeedIndexingFilter
datum - Variable in class org.apache.nutch.crawl.Generator.SelectorEntry
datum - Variable in class org.apache.nutch.hostdb.ResolverThread
DB_IGNORE_EXTERNAL_EXEMPTIONS_FILE - Static variable in class org.apache.nutch.urlfilter.ignoreexempt.ExemptionUrlFilter
DBFilter() - Constructor for class org.apache.nutch.crawl.DeduplicationJob.DBFilter
DBFilter() - Constructor for class org.apache.nutch.indexer.CleaningJob.DBFilter
DebugParseFilter - Class in org.apache.nutch.parsefilter.debug
  Adds serialized DOM to parse data, useful for debugging, to understand how the parser implementation interprets a document (not only HTML).
DebugParseFilter() - Constructor for class org.apache.nutch.parsefilter.debug.DebugParseFilter
DEC_RATE - Variable in class org.apache.nutch.crawl.AdaptiveFetchSchedule
DecreasingFloatComparator() - Constructor for class org.apache.nutch.crawl.Generator.DecreasingFloatComparator
DEDUP_DOCUMENTS_MARKED_DUPLICATE_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents marked as duplicate.
DEDUPLICATION_COMPARE_ORDER - Static variable in class org.apache.nutch.crawl.DeduplicationJob
DEDUPLICATION_GROUP_MODE - Static variable in class org.apache.nutch.crawl.DeduplicationJob
DeduplicationJob - Class in org.apache.nutch.crawl
  Generic deduplicator which groups fetched URLs with the same digest and marks all of them as duplicate except the one with the highest score (based on the score in the crawldb, which is not necessarily the same as the score indexed).
DeduplicationJob() - Constructor for class org.apache.nutch.crawl.DeduplicationJob
DeduplicationJob.DBFilter - Class in org.apache.nutch.crawl
DeduplicationJob.DedupReducer<K extends Writable> - Class in org.apache.nutch.crawl
DeduplicationJob.StatusUpdateReducer - Class in org.apache.nutch.crawl
  Combine multiple new entries for a url.
DedupReducer() - Constructor for class org.apache.nutch.crawl.DeduplicationJob.DedupReducer
DefalultMultiInteractionHandler - Class in org.apache.nutch.protocol.interactiveselenium.handlers
  This is a placeholder/example of a technique or use case where we do multiple interaction with the web driver and need data from each such interaction in the end.
DefalultMultiInteractionHandler() - Constructor for class org.apache.nutch.protocol.interactiveselenium.handlers.DefalultMultiInteractionHandler
DEFAULT_FILE_NAME - Static variable in class org.apache.nutch.collection.CollectionManager
DEFAULT_ID - Static variable in class org.apache.nutch.fetcher.FetchItemQueues
DEFAULT_MAX_DEPTH - Static variable in class org.apache.nutch.scoring.depth.DepthScoringFilter
DEFAULT_PLUGIN - Static variable in class org.apache.nutch.parse.ParserFactory
  Wildcard for default plugins.
DefaultClickAllAjaxLinksHandler - Class in org.apache.nutch.protocol.interactiveselenium.handlers
  This handler clicks all the `` tags because it considers them as not usual links but ajax links/interactions.
DefaultClickAllAjaxLinksHandler() - Constructor for class org.apache.nutch.protocol.interactiveselenium.handlers.DefaultClickAllAjaxLinksHandler
DefaultFetchSchedule - Class in org.apache.nutch.crawl
  This class implements the default re-fetch schedule.
DefaultFetchSchedule() - Constructor for class org.apache.nutch.crawl.DefaultFetchSchedule
DefaultHandler - Class in org.apache.nutch.protocol.interactiveselenium.handlers
DefaultHandler() - Constructor for class org.apache.nutch.protocol.interactiveselenium.handlers.DefaultHandler
defaultInterval - Variable in class org.apache.nutch.crawl.AbstractFetchSchedule
defaultProtocolImplMapping - Variable in class org.apache.nutch.protocol.ProtocolFactory
DEFER_VISIT_RULES - Static variable in class org.apache.nutch.protocol.RobotRulesParser
  A `BaseRobotRules` object appropriate for use when the `robots.txt` file failed to fetch with a 503 "Internal Server Error" (or other 5xx) status code.
deferVisits503 - Variable in class org.apache.nutch.protocol.http.api.HttpRobotRulesParser
deflate(byte[]) - Static method in class org.apache.nutch.util.DeflateUtils
  Returns a deflated copy of the input array.
DeflateUtils - Class in org.apache.nutch.util
  A collection of utility methods for working on deflated data.
DeflateUtils() - Constructor for class org.apache.nutch.util.DeflateUtils
delete(String) - Method in interface org.apache.nutch.indexer.IndexWriter
delete(String) - Method in class org.apache.nutch.indexer.IndexWriters
delete(String) - Method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
delete(String) - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
  (deletion of documents is not supported)
delete(String) - Method in class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
delete(String) - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
delete(String) - Method in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
delete(String) - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
delete(String) - Method in class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
delete(String) - Method in class org.apache.nutch.indexwriter.solr.SolrIndexWriter
delete(String, boolean) - Method in class org.apache.nutch.indexer.CleaningJob
DELETE - Static variable in class org.apache.nutch.indexer.NutchIndexAction
DELETE - Static variable in interface org.apache.nutch.indexwriter.dummy.DummyConstants
DELETE_FAILED_PARSE - Static variable in class org.apache.nutch.parse.ParseSegment
  Configuration property to delete documents that failed to be parsed.
DeleterReducer() - Constructor for class org.apache.nutch.indexer.CleaningJob.DeleterReducer
deleteSubCollection(String) - Method in class org.apache.nutch.collection.CollectionManager
  Delete named subcollection
DenyPathQueryRule(String) - Constructor for class org.apache.nutch.urlfilter.fast.FastURLFilter.DenyPathQueryRule
DenyPathRule(String) - Constructor for class org.apache.nutch.urlfilter.fast.FastURLFilter.DenyPathRule
DEPTH_KEY - Static variable in class org.apache.nutch.scoring.depth.DepthScoringFilter
DEPTH_KEY_W - Static variable in class org.apache.nutch.scoring.depth.DepthScoringFilter
DepthScoringFilter - Class in org.apache.nutch.scoring.depth
  This scoring filter limits the number of hops from the initial seed urls.
DepthScoringFilter() - Constructor for class org.apache.nutch.scoring.depth.DepthScoringFilter
describe() - Method in interface org.apache.nutch.indexer.IndexWriter
  Returns `Map` with the specific parameters the IndexWriter instance can take.
describe() - Method in class org.apache.nutch.indexer.IndexWriters
  Lists the active IndexWriters and their configuration.
describe() - Method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
  Returns `Map` with the specific parameters the IndexWriter instance can take.
describe() - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
  Returns `Map` with the specific parameters the IndexWriter instance can take.
describe() - Method in class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
  Returns `Map` with the specific parameters the IndexWriter instance can take.
describe() - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
  Returns `Map` with the specific parameters the IndexWriter instance can take.
describe() - Method in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
describe() - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
  Returns `Map` with the specific parameters the IndexWriter instance can take.
describe() - Method in class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
  Returns `Map` with the specific parameters the IndexWriter instance can take.
describe() - Method in class org.apache.nutch.indexwriter.solr.SolrIndexWriter
  Returns `Map` with the specific parameters the IndexWriter instance can take.
DESCRIPTION - Static variable in interface org.apache.nutch.metadata.DublinCore
  An account of the content of the resource.
DICTFILE_MODELFILTER - Static variable in class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
DIR_NAME - Static variable in class org.apache.nutch.parse.ParseData
DIR_NAME - Static variable in class org.apache.nutch.parse.ParseText
DIR_NAME - Static variable in class org.apache.nutch.protocol.Content
DISABLE_SANDBOX_KEY - Static variable in class org.apache.nutch.util.JexlUtil
  When `true`, JEXL parsing skips the sandbox (unsafe).
disconnect() - Method in class org.apache.nutch.protocol.ftp.Client
  Closes the connection to the FTP server and restores connection parameters to the default values.
DISCONNECT - Enum constant in enum class org.apache.nutch.net.protocols.Response.TruncatedContentReason
  network disconnect or timeout during fetch
displayFileTypes(Map<String, Integer>, Map<String, Integer>) - Static method in class org.apache.nutch.util.DumpFileUtil
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in class org.apache.nutch.scoring.AbstractScoringFilter
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in class org.apache.nutch.scoring.depth.DepthScoringFilter
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in class org.apache.nutch.scoring.metadata.MetadataScoringFilter
  This will take the metadata that you have listed in your "scoring.parse.md" property, and looks for them inside the parseData object.
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
  Get a float value from Fetcher.SCORE_KEY, divide it by the number of outlinks and apply.
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in interface org.apache.nutch.scoring.ScoringFilter
  Distribute score value from the current page to all its outlinked pages.
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in class org.apache.nutch.scoring.ScoringFilters
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in class org.apache.nutch.scoring.similarity.cosine.CosineSimilarity
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in interface org.apache.nutch.scoring.similarity.SimilarityModel
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in class org.apache.nutch.scoring.similarity.SimilarityScoringFilter
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in class org.apache.nutch.scoring.urlmeta.URLMetaScoringFilter
  This will take the metatags that you have listed in your "urlmeta.tags" property, and looks for them inside the parseData object.
DmozParser - Class in org.apache.nutch.tools
  Utility that converts DMOZ RDF into a flat file of URLs to be injected.
DmozParser() - Constructor for class org.apache.nutch.tools.DmozParser
dnsFailures - Variable in class org.apache.nutch.hostdb.HostDatum
doc - Variable in class org.apache.nutch.indexer.NutchIndexAction
docToMetadata(NutchDocument) - Static method in class org.apache.nutch.tools.WARCUtils
DocVector - Class in org.apache.nutch.scoring.similarity.cosine
DocVector() - Constructor for class org.apache.nutch.scoring.similarity.cosine.DocVector
docVectors - Static variable in class org.apache.nutch.scoring.similarity.cosine.Model
doIndex - Variable in class org.apache.nutch.indexer.IndexingFiltersChecker
DOMAIN - Enum constant in enum class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter.DatabaseType
DOMAIN_STATS_EMPTY_RESULT_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Empty results in domain statistics.
DOMAIN_STATS_FETCHED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Fetched URLs in domain statistics.
DOMAIN_STATS_NOT_FETCHED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Not fetched URLs in domain statistics.
DomainDenylistURLFilter - Class in org.apache.nutch.urlfilter.domaindenylist
  Filters URLs based on a file containing domain suffixes, domain names, and hostnames.
DomainDenylistURLFilter() - Constructor for class org.apache.nutch.urlfilter.domaindenylist.DomainDenylistURLFilter
DomainStatistics - Class in org.apache.nutch.util
  Extracts some very basic statistics about domains from the crawldb
DomainStatistics() - Constructor for class org.apache.nutch.util.DomainStatistics
DomainStatistics.DomainStatisticsCombiner - Class in org.apache.nutch.util
DomainStatisticsCombiner() - Constructor for class org.apache.nutch.util.DomainStatistics.DomainStatisticsCombiner
DomainURLFilter - Class in org.apache.nutch.urlfilter.domain
  Filters URLs based on a file containing domain suffixes, domain names, and hostnames.
DomainURLFilter() - Constructor for class org.apache.nutch.urlfilter.domain.DomainURLFilter
DOMBuilder - Class in org.apache.nutch.parse.html
  This class takes SAX events (in addition to some extra events that SAX doesn't handle yet) and adds the result to a document or document fragment.
DOMBuilder(Document) - Constructor for class org.apache.nutch.parse.html.DOMBuilder
  DOMBuilder instance constructor...
DOMBuilder(Document, DocumentFragment) - Constructor for class org.apache.nutch.parse.html.DOMBuilder
  DOMBuilder instance constructor...
DOMBuilder(Document, Node) - Constructor for class org.apache.nutch.parse.html.DOMBuilder
  DOMBuilder instance constructor...
DOMContentUtils - Class in org.apache.nutch.parse.html
  A collection of methods for extracting content from DOM trees.
DOMContentUtils - Class in org.apache.nutch.parse.tika
  A collection of methods for extracting content from DOM trees.
DOMContentUtils(Configuration) - Constructor for class org.apache.nutch.parse.html.DOMContentUtils
DOMContentUtils(Configuration) - Constructor for class org.apache.nutch.parse.tika.DOMContentUtils
DOMContentUtils.LinkParams - Class in org.apache.nutch.parse.html
DomUtil - Class in org.apache.nutch.util
DomUtil() - Constructor for class org.apache.nutch.util.DomUtil
dotProduct(DocVector) - Method in class org.apache.nutch.scoring.similarity.cosine.DocVector
DublinCore - Interface in org.apache.nutch.metadata
  A collection of Dublin Core metadata names.
DummyConstants - Interface in org.apache.nutch.indexwriter.dummy
DummyIndexWriter - Class in org.apache.nutch.indexwriter.dummy
  DummyIndexWriter.
DummyIndexWriter() - Constructor for class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
DummySSLProtocolSocketFactory - Class in org.apache.nutch.protocol.httpclient
DummySSLProtocolSocketFactory() - Constructor for class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
  Constructor for DummySSLProtocolSocketFactory.
DummyX509TrustManager - Class in org.apache.nutch.protocol.htmlunit
DummyX509TrustManager - Class in org.apache.nutch.protocol.http
DummyX509TrustManager - Class in org.apache.nutch.protocol.httpclient
DummyX509TrustManager - Class in org.apache.nutch.protocol.interactiveselenium
DummyX509TrustManager - Class in org.apache.nutch.protocol.selenium
DummyX509TrustManager(KeyStore) - Constructor for class org.apache.nutch.protocol.htmlunit.DummyX509TrustManager
  Constructor for DummyX509TrustManager.
DummyX509TrustManager(KeyStore) - Constructor for class org.apache.nutch.protocol.http.DummyX509TrustManager
  Constructor for DummyX509TrustManager.
DummyX509TrustManager(KeyStore) - Constructor for class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
  Constructor for DummyX509TrustManager.
DummyX509TrustManager(KeyStore) - Constructor for class org.apache.nutch.protocol.interactiveselenium.DummyX509TrustManager
  Constructor for DummyX509TrustManager.
DummyX509TrustManager(KeyStore) - Constructor for class org.apache.nutch.protocol.selenium.DummyX509TrustManager
  Constructor for DummyX509TrustManager.
dump() - Method in class org.apache.nutch.fetcher.FetchItemQueue
dump() - Method in class org.apache.nutch.fetcher.FetchItemQueues
dump(File, File, File, boolean, String[], boolean, String, boolean) - Method in class org.apache.nutch.tools.CommonCrawlDataDumper
  Dumps the reverse engineered CBOR content from the provided segment directories if a parent directory contains more than one segment, otherwise a single segment can be passed as an argument.
dump(File, File, String[], boolean, boolean, boolean) - Method in class org.apache.nutch.tools.FileDumper
  Dumps the reverse engineered raw content from the provided segment directories if a parent directory contains more than one segment, otherwise a single segment can be passed as an argument.
dump(Path, Path) - Method in class org.apache.nutch.segment.SegmentReader
DUMP_DIR - Static variable in class org.apache.nutch.scoring.webgraph.LinkDumper
Dumper() - Constructor for class org.apache.nutch.scoring.webgraph.NodeDumper.Dumper
DumperMapper() - Constructor for class org.apache.nutch.scoring.webgraph.NodeDumper.Dumper.DumperMapper
DumperReducer() - Constructor for class org.apache.nutch.scoring.webgraph.NodeDumper.Dumper.DumperReducer
DumpFileUtil - Class in org.apache.nutch.util
DumpFileUtil() - Constructor for class org.apache.nutch.util.DumpFileUtil
dumpLinks(Path) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper
  Runs the inverter and merger jobs of the LinkDumper tool to create the url to inlink node database.
dumpNodes(Path, NodeDumper.DumpType, long, Path, boolean, NodeDumper.NameType, NodeDumper.AggrType, boolean) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper
  Runs the process to dump the top urls out to a text file.
dumpText - Variable in class org.apache.nutch.indexer.IndexingFiltersChecker
dumpText - Variable in class org.apache.nutch.parse.ParserChecker
dumpUrl(Path, String) - Method in class org.apache.nutch.scoring.webgraph.NodeReader
  Prints the content of the Node represented by the url to system out.

## E

elapsedTime(long, long) - Static method in class org.apache.nutch.util.TimingUtil
  Calculate the elapsed time between two times specified in milliseconds.
ElasticConstants - Interface in org.apache.nutch.indexwriter.elastic
ElasticIndexWriter - Class in org.apache.nutch.indexwriter.elastic
  Sends NutchDocuments to a configured Elasticsearch index.
ElasticIndexWriter() - Constructor for class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
elName - Variable in class org.apache.nutch.parse.html.DOMContentUtils.LinkParams
emitCountAndSumOnly(TaskInputOutputContext<?, ?, ?, ?>) - Method in class org.apache.nutch.metrics.LatencyTracker
  Emits only count and sum counters (not percentiles).
emitCounters(TaskInputOutputContext<?, ?, ?, ?>) - Method in class org.apache.nutch.metrics.ErrorTracker
  Emits all error counters to the Hadoop context.
emitCounters(TaskInputOutputContext<?, ?, ?, ?>) - Method in class org.apache.nutch.metrics.LatencyTracker
  Emits all latency counters to the Hadoop context.
EMPTY_RULES - Static variable in class org.apache.nutch.protocol.RobotRulesParser
  A `BaseRobotRules` object appropriate for use when the `robots.txt` file is empty or missing; all requests are allowed.
emptyMetaDataWritableSerialized - Static variable in class org.apache.nutch.hostdb.HostDatum
emptyQueue() - Method in class org.apache.nutch.fetcher.FetchItemQueue
emptyQueues() - Method in class org.apache.nutch.fetcher.FetchItemQueues
enableCookieHeader - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Controls whether or not to set Cookie HTTP header based on CrawlDatum metadata
enableIfModifiedsinceHeader - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Configuration directive for If-Modified-Since HTTP header
encoding - Variable in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
  encoding of CSV file
EncodingDetector - Class in org.apache.nutch.util
  A simple class for detecting character encodings.
EncodingDetector(Configuration) - Constructor for class org.apache.nutch.util.EncodingDetector
end - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
END - Enum constant in enum class org.apache.nutch.fetcher.FetcherThreadEvent.PublishEventType
endCDATA() - Method in class org.apache.nutch.parse.html.DOMBuilder
  Report the end of a CDATA section.
endDocument() - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive notification of the end of a document.
endDTD() - Method in class org.apache.nutch.parse.html.DOMBuilder
  Report the end of DTD declarations.
endElement(String, String, String) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive notification of the end of an element.
endEntity(String) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Report the end of an entity.
ENDPOINT - Static variable in interface org.apache.nutch.indexwriter.cloudsearch.CloudSearchConstants
endPrefixMapping(String) - Method in class org.apache.nutch.parse.html.DOMBuilder
  End the scope of a prefix-URI mapping.
ENGLISHMINIMALSTEM_FILTER - Enum constant in enum class org.apache.nutch.scoring.similarity.util.LuceneAnalyzerUtil.StemFilterType
entityReference(String) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive notivication of a entityReference.
EQUAL_CHARACTER - Static variable in class org.apache.nutch.crawl.Injector.InjectMapper
equals(Object) - Method in class org.apache.nutch.crawl.CrawlDatum
equals(Object) - Method in class org.apache.nutch.crawl.Inlink
equals(Object) - Method in class org.apache.nutch.metadata.Metadata
equals(Object) - Method in class org.apache.nutch.parse.Outlink
equals(Object) - Method in class org.apache.nutch.parse.ParseData
equals(Object) - Method in class org.apache.nutch.parse.ParseStatus
equals(Object) - Method in class org.apache.nutch.parse.ParseText
equals(Object) - Method in class org.apache.nutch.plugin.PluginClassLoader
equals(Object) - Method in class org.apache.nutch.protocol.Content
equals(Object) - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
equals(Object) - Method in class org.apache.nutch.protocol.ProtocolStatus
ERROR_INDEXING_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Indexing filter errors.
ERROR_NETWORK_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Network-related errors.
ERROR_OTHER_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Other uncategorized errors.
ERROR_PARSING_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Parsing errors.
ERROR_PROTOCOL_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Protocol errors.
ERROR_SCORING_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Scoring filter errors.
ERROR_TIMEOUT_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Timeout errors.
ERROR_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Total errors across all categories.
ERROR_URL_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URL-related errors.
errorTracker - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
ErrorTracker - Class in org.apache.nutch.metrics
  A utility class for tracking errors by category with automatic classification.
ErrorTracker(String) - Constructor for class org.apache.nutch.metrics.ErrorTracker
  Creates a new ErrorTracker for the specified counter group.
ErrorTracker(String, TaskInputOutputContext<?, ?, ?, ?>) - Constructor for class org.apache.nutch.metrics.ErrorTracker
  Creates a new ErrorTracker with cached counter references.
ErrorTracker.ErrorType - Enum Class in org.apache.nutch.metrics
  Error type categories for classification.
escape(String) - Method in class org.apache.nutch.net.urlnormalizer.ajax.AjaxURLNormalizer
  Escape some exotic characters in the fragment part
ESCAPED_URL_PART - Static variable in class org.apache.nutch.net.urlnormalizer.ajax.AjaxURLNormalizer
evaluate() - Method in class org.apache.nutch.util.CommandRunner
EXCEPTION - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Unspecified exception occurred.
Exchange - Interface in org.apache.nutch.exchange
ExchangeConfig - Class in org.apache.nutch.exchange
Exchanges - Class in org.apache.nutch.exchange
Exchanges(Configuration) - Constructor for class org.apache.nutch.exchange.Exchanges
exec() - Method in class org.apache.nutch.util.CommandRunner
  Execute the command
execute(JexlScript, String) - Method in class org.apache.nutch.crawl.CrawlDatum
executor - Variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
ExemptionUrlFilter - Class in org.apache.nutch.urlfilter.ignoreexempt
  This implementation of `URLExemptionFilter` uses regex configuration to check if URL is eligible for exemption from the `db.ignore.external.links` configuration property.
ExemptionUrlFilter() - Constructor for class org.apache.nutch.urlfilter.ignoreexempt.ExemptionUrlFilter
EXPONENTIAL_BACKOFF_MILLIS - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
EXPONENTIAL_BACKOFF_MILLIS - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
EXPONENTIAL_BACKOFF_RETRIES - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
EXPONENTIAL_BACKOFF_RETRIES - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
Extension - Class in org.apache.nutch.plugin
  An `Extension` is a kind of listener descriptor that will be installed on a concrete `ExtensionPoint` that acts as kind of Publisher.
Extension(PluginDescriptor, String, String, String, Configuration, PluginRepository) - Constructor for class org.apache.nutch.plugin.Extension
ExtensionPoint - Class in org.apache.nutch.plugin
  The `ExtensionPoint` provide meta information of a extension point.
ExtensionPoint(String, String, String) - Constructor for class org.apache.nutch.plugin.ExtensionPoint
  Constructor
ExtParser - Class in org.apache.nutch.parse.ext
  A wrapper that invokes external command to do real parsing job.
ExtParser() - Constructor for class org.apache.nutch.parse.ext.ExtParser
extractText(InputStream, String, List<Outlink>) - Method in class org.apache.nutch.parse.zip.ZipTextExtractor

## F

FAILED - Static variable in class org.apache.nutch.parse.ParseStatus
  General failure.
FAILED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Content was not retrieved.
FAILED_EXCEPTION - Static variable in class org.apache.nutch.parse.ParseStatus
  Parsing failed.
FAILED_INVALID_FORMAT - Static variable in class org.apache.nutch.parse.ParseStatus
  Parsing failed.
FAILED_MISSING_CONTENT - Static variable in class org.apache.nutch.parse.ParseStatus
  Parsing failed.
FAILED_MISSING_PARTS - Static variable in class org.apache.nutch.parse.ParseStatus
  Parsing failed.
FAILED_TRUNCATED - Static variable in class org.apache.nutch.parse.ParseStatus
  Parsing failed.
failures - Variable in class org.apache.nutch.hostdb.HostDatum
FastURLFilter - Class in org.apache.nutch.urlfilter.fast
  Filters URLs based on a file of regular expressions using host/domains matching first.
FastURLFilter() - Constructor for class org.apache.nutch.urlfilter.fast.FastURLFilter
FastURLFilter.DenyAllRule - Class in org.apache.nutch.urlfilter.fast
  Rule for `DenyPath .*` or `DenyPath .?`
FastURLFilter.DenyPathQueryRule - Class in org.apache.nutch.urlfilter.fast
FastURLFilter.DenyPathRule - Class in org.apache.nutch.urlfilter.fast
FastURLFilter.Rule - Class in org.apache.nutch.urlfilter.fast
Feed - Interface in org.apache.nutch.metadata
  A collection of Feed property names extracted by the ROME library.
FEED - Static variable in interface org.apache.nutch.metadata.Feed
FEED_AUTHOR - Static variable in interface org.apache.nutch.metadata.Feed
FEED_PUBLISHED - Static variable in interface org.apache.nutch.metadata.Feed
FEED_TAGS - Static variable in interface org.apache.nutch.metadata.Feed
FEED_UPDATED - Static variable in interface org.apache.nutch.metadata.Feed
FeedIndexingFilter - Class in org.apache.nutch.indexer.feed
FeedIndexingFilter() - Constructor for class org.apache.nutch.indexer.feed.FeedIndexingFilter
FeedParser - Class in org.apache.nutch.parse.feed
  A RSS/ATOM Feed`Parser` that rapidly parses all referenced links and content present in the feed.
FeedParser() - Constructor for class org.apache.nutch.parse.feed.FeedParser
fetch(Path, int) - Method in class org.apache.nutch.fetcher.Fetcher
FETCH_DIR_NAME - Static variable in class org.apache.nutch.crawl.CrawlDatum
FETCH_EVENT_CONTENTLANG - Static variable in interface org.apache.nutch.metadata.Nutch
  Content-lanueage key in the Pub/Sub event metadata for the content-language of the parsed page
FETCH_EVENT_CONTENTTYPE - Static variable in interface org.apache.nutch.metadata.Nutch
  Content-type key in the Pub/Sub event metadata for the content-type of the parsed page
FETCH_EVENT_FETCHTIME - Static variable in interface org.apache.nutch.metadata.Nutch
  Fetch time key in the Pub/Sub event metadata for the fetch time of the parsed page
FETCH_EVENT_SCORE - Static variable in interface org.apache.nutch.metadata.Nutch
  Score key in the Pub/Sub event metadata for the score of the parsed page
FETCH_EVENT_TITLE - Static variable in interface org.apache.nutch.metadata.Nutch
  Title key in the Pub/Sub event metadata for the title of the parsed page
FETCH_STATUS_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
FETCH_TIME - Static variable in interface org.apache.nutch.net.protocols.Response
  Key to hold the time when the page has been fetched
FETCH_TIME_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
fetched - Variable in class org.apache.nutch.hostdb.HostDatum
fetched - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
Fetcher - Class in org.apache.nutch.fetcher
  A queue-based fetcher.
Fetcher() - Constructor for class org.apache.nutch.fetcher.Fetcher
Fetcher(Configuration) - Constructor for class org.apache.nutch.fetcher.Fetcher
FETCHER_ABOVE_EXCEPTION_THRESHOLD_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs dropped due to exception threshold in queue.
FETCHER_BYTES_DOWNLOADED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Total bytes downloaded by fetcher.
FETCHER_FILTERED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs filtered during fetching.
FETCHER_HIT_BY_THROUGHPUT_THRESHOLD_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs hit by throughput threshold.
FETCHER_HIT_BY_TIMELIMIT_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs hit by time limit.
FETCHER_HIT_BY_TIMEOUT_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs hit by timeout.
FETCHER_HUNG_THREADS_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Threads that hung during fetching.
FETCHER_LATENCY - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Prefix for fetch latency metrics.
FETCHER_OUTLINKS_DETECTED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Outlinks detected during parsing.
FETCHER_OUTLINKS_FOLLOWING_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Outlinks being followed.
FETCHER_REDIRECT_COUNT_EXCEEDED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Redirects that exceeded maximum redirect count.
FETCHER_REDIRECT_DEDUPLICATED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Redirects deduplicated (already seen).
FETCHER_REDIRECT_NOT_CREATED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  FetchItems not created for redirects.
FETCHER_ROBOTS_DEFER_VISITS_DROPPED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs dropped due to robots.txt deferred visits.
FETCHER_ROBOTS_DENIED_MAXCRAWLDELAY_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs denied due to crawl delay exceeding maximum.
FETCHER_ROBOTS_DENIED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs denied by robots.txt.
Fetcher.FetcherReducer - Class in org.apache.nutch.fetcher
  Reducer that passes through (url, datum) records and merges TDigests from map tasks to set job-level latency percentile counters.
Fetcher.FetcherRun - Class in org.apache.nutch.fetcher
Fetcher.InputFormat - Class in org.apache.nutch.fetcher
FetcherOutputFormat - Class in org.apache.nutch.fetcher
  Splits FetcherOutput entries into multiple map files.
FetcherOutputFormat() - Constructor for class org.apache.nutch.fetcher.FetcherOutputFormat
FetcherReducer() - Constructor for class org.apache.nutch.fetcher.Fetcher.FetcherReducer
fetchErrors - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
FetcherRun() - Constructor for class org.apache.nutch.fetcher.Fetcher.FetcherRun
FetcherThread - Class in org.apache.nutch.fetcher
  This class picks items from queues and fetches the pages.
FetcherThread(Configuration, AtomicInteger, FetchItemQueues, QueueFeeder, AtomicInteger, AtomicLong, Mapper.Context, AtomicInteger, String, boolean, boolean, AtomicInteger, AtomicLong) - Constructor for class org.apache.nutch.fetcher.FetcherThread
FetcherThreadEvent - Class in org.apache.nutch.fetcher
  This class is used to capture the various events occurring at fetch time.
FetcherThreadEvent(FetcherThreadEvent.PublishEventType, String) - Constructor for class org.apache.nutch.fetcher.FetcherThreadEvent
  Constructor to create an event to be published
FetcherThreadEvent.PublishEventType - Enum Class in org.apache.nutch.fetcher
  Type of event to specify start, end or reporting of a fetch item.
FetcherThreadPublisher - Class in org.apache.nutch.fetcher
  This class handles the publishing of the events to the queue implementation.
FetcherThreadPublisher(Configuration) - Constructor for class org.apache.nutch.fetcher.FetcherThreadPublisher
  Configure all registered publishers
FetchItem - Class in org.apache.nutch.fetcher
  This class describes the item to be fetched.
FetchItem(Text, URL, CrawlDatum, String) - Constructor for class org.apache.nutch.fetcher.FetchItem
FetchItem(Text, URL, CrawlDatum, String, int) - Constructor for class org.apache.nutch.fetcher.FetchItem
FetchItemQueue - Class in org.apache.nutch.fetcher
  This class handles FetchItems which come from the same host ID (be it a proto/hostname or proto/IP pair).
FetchItemQueue(Configuration, int, long, long) - Constructor for class org.apache.nutch.fetcher.FetchItemQueue
FetchItemQueues - Class in org.apache.nutch.fetcher
  A collection of queues that keeps track of the total number of items, and provides items eligible for fetching from any queue.
FetchItemQueues(Configuration) - Constructor for class org.apache.nutch.fetcher.FetchItemQueues
FetchOverdueCrawlDatumProcessor - Class in org.apache.nutch.hostdb
  Simple custom crawl datum processor that counts the number of records that are overdue for fetching, e.g.
FetchOverdueCrawlDatumProcessor(Configuration) - Constructor for class org.apache.nutch.hostdb.FetchOverdueCrawlDatumProcessor
FetchSchedule - Interface in org.apache.nutch.crawl
  This interface defines the contract for implementations that manipulate fetch times and re-fetch intervals.
FetchScheduleFactory - Class in org.apache.nutch.crawl
  Creates and caches a `FetchSchedule` implementation.
FG() - Constructor for class org.apache.nutch.tools.FreeGenerator.FG
FGMapper() - Constructor for class org.apache.nutch.tools.FreeGenerator.FG.FGMapper
FGReducer() - Constructor for class org.apache.nutch.tools.FreeGenerator.FG.FGReducer
FIELD - Static variable in class org.creativecommons.nutch.CCIndexingFilter
  The name of the document field we use.
fieldName - Static variable in class org.apache.nutch.indexer.subcollection.SubcollectionIndexingFilter
  Doc field name
FieldReplacer - Class in org.apache.nutch.indexer.replace
  POJO to store a filename, its match pattern and its replacement string.
FieldReplacer(String, String, String, Integer) - Constructor for class org.apache.nutch.indexer.replace.FieldReplacer
  Field replacer with the input and output field the same.
FieldReplacer(String, String, String, String, Integer) - Constructor for class org.apache.nutch.indexer.replace.FieldReplacer
  Create a FieldReplacer for a field.
File - Class in org.apache.nutch.protocol.file
  This class is a protocol plugin used for file: scheme.
File() - Constructor for class org.apache.nutch.protocol.file.File
FileDumper - Class in org.apache.nutch.tools
  The file dumper tool enables one to reverse generate the raw content from Nutch segment data directories.
FileDumper() - Constructor for class org.apache.nutch.tools.FileDumper
FileError - Exception in org.apache.nutch.protocol.file
  Thrown for File error codes.
FileError(int) - Constructor for exception org.apache.nutch.protocol.file.FileError
FileException - Exception in org.apache.nutch.protocol.file
FileException() - Constructor for exception org.apache.nutch.protocol.file.FileException
FileException(String) - Constructor for exception org.apache.nutch.protocol.file.FileException
FileException(String, Throwable) - Constructor for exception org.apache.nutch.protocol.file.FileException
FileException(Throwable) - Constructor for exception org.apache.nutch.protocol.file.FileException
fileLen - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
FileResponse - Class in org.apache.nutch.protocol.file
  FileResponse.java mimics file replies as http response.
FileResponse(URL, CrawlDatum, File, Configuration) - Constructor for class org.apache.nutch.protocol.file.FileResponse
  Default public constructor
filter - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
filter() - Method in class org.apache.nutch.parse.ParseResult
  Remove all results where status is not successful (as determined by `ParseStatus.isSuccess()`).
filter(String) - Method in class org.apache.nutch.collection.Subcollection
  Simple "indexOf" currentFilter for matching patterns.
filter(String) - Method in interface org.apache.nutch.net.URLFilter
  Interface for a filter that transforms a URL: it can pass the original URL through or "delete" the URL by returning null
filter(String) - Method in class org.apache.nutch.net.URLFilters
  Run all defined filters.
filter(String) - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
filter(String) - Method in class org.apache.nutch.urlfilter.domain.DomainURLFilter
filter(String) - Method in class org.apache.nutch.urlfilter.domaindenylist.DomainDenylistURLFilter
filter(String) - Method in class org.apache.nutch.urlfilter.fast.FastURLFilter
filter(String) - Method in class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
filter(String) - Method in class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
filter(String) - Method in class org.apache.nutch.urlfilter.validator.UrlValidator
filter(String, String) - Method in interface org.apache.nutch.net.URLExemptionFilter
  Checks if toUrl is exempted when the ignore external is enabled
filter(String, String) - Method in class org.apache.nutch.urlfilter.ignoreexempt.ExemptionUrlFilter
filter(Text, CrawlDatum, CrawlDatum, CrawlDatum, Content, ParseData, ParseText, Collection<CrawlDatum>) - Method in interface org.apache.nutch.segment.SegmentMergeFilter
  The filtering method which gets all information being merged for a given key (URL).
filter(Text, CrawlDatum, CrawlDatum, CrawlDatum, Content, ParseData, ParseText, Collection<CrawlDatum>) - Method in class org.apache.nutch.segment.SegmentMergeFilters
  Iterates over all `SegmentMergeFilter` extensions and if any of them returns false, it will return false as well.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.analysis.lang.LanguageIndexingFilter
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.anchor.AnchorIndexingFilter
  The `AnchorIndexingFilter` filter object which supports boolean configuration settings for the deduplication of anchors.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.arbitrary.ArbitraryIndexingFilter
  The `ArbitraryIndexingFilter` filter object uses reflection to instantiate the configured class and invoke the configured method.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.basic.BasicIndexingFilter
  The `BasicIndexingFilter` filter object which supports few configuration settings for adding basic searchable fields.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.feed.FeedIndexingFilter
  Extracts out the relevant fields: FEED_AUTHOR FEED_TAGS FEED_PUBLISHED FEED_UPDATED FEED And sends them to the `Indexer` for indexing within the Nutch index.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.filter.MimeTypeIndexingFilter
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in interface org.apache.nutch.indexer.IndexingFilter
  Adds fields or otherwise modifies the document that will be indexed for a parse.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.IndexingFilters
  Run all defined filters.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.jexl.JexlIndexingFilter
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.links.LinksIndexingFilter
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.metadata.MetadataIndexer
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.more.MoreIndexingFilter
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.replace.ReplaceIndexer
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.staticfield.StaticFieldIndexer
  The `StaticFieldIndexer` filter object which adds fields as per configuration setting.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.subcollection.SubcollectionIndexingFilter
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.tld.TLDIndexingFilter
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.urlmeta.URLMetaIndexingFilter
  This will take the metatags that you have listed in your "urlmeta.tags" property, and looks for them inside the CrawlDatum object.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.microformats.reltag.RelTagIndexingFilter
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.creativecommons.nutch.CCIndexingFilter
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.analysis.lang.HTMLLanguageParser
  Scan the HTML document looking at possible indications of content language
  1.
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.microformats.reltag.RelTagParser
  Scan the HTML document looking at possible rel-tags
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.parse.headings.HeadingsParseFilter
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in interface org.apache.nutch.parse.HtmlParseFilter
  Adds metadata or otherwise modifies a parse of HTML content, given the DOM tree of a page.
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.parse.HtmlParseFilters
  Run all defined filters.
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.parse.js.JSParseFilter
  Scan the JavaScript fragments of a HTML page looking for possible `Outlink`'s
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.parse.metatags.MetaTagsParser
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.parsefilter.debug.DebugParseFilter
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.parsefilter.regex.RegexParseFilter
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.creativecommons.nutch.CCParseFilter
  Adds metadata or otherwise modifies a parse of an HTML document, given the DOM tree of a page.
filteredRecordsCounter - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
filterNormalize(String) - Method in class org.apache.nutch.hostdb.UpdateHostDbMapper
  Filters and or normalizes the input hostname by applying the configured URL filters and normalizers the URL "http://hostname/".
filterNormalize(String, String, String, boolean, boolean, String, URLFilters, URLExemptionFilters, URLNormalizers) - Static method in class org.apache.nutch.parse.ParseOutputFormat
filterNormalize(String, String, String, boolean, boolean, String, URLFilters, URLExemptionFilters, URLNormalizers, String) - Static method in class org.apache.nutch.parse.ParseOutputFormat
filterParse(String) - Method in class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
filters - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
filterUrl(String) - Method in class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
finalize() - Method in class org.apache.nutch.plugin.Plugin
finalize() - Method in class org.apache.nutch.plugin.PluginRepository
  Deprecated.
finalize() - Method in class org.apache.nutch.protocol.ftp.Ftp
finalize(HostDatum) - Method in interface org.apache.nutch.hostdb.CrawlDatumProcessor
  Process the final host datum instance and store the aggregated custom counts in the HostDatum.
finalize(HostDatum) - Method in class org.apache.nutch.hostdb.FetchOverdueCrawlDatumProcessor
find(String, int) - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter.Separator
  Get index of first occurrence of any separator characters.
findAuthentication(Metadata) - Method in class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
finishFetchItem(FetchItem) - Method in class org.apache.nutch.fetcher.FetchItemQueues
finishFetchItem(FetchItem, boolean) - Method in class org.apache.nutch.fetcher.FetchItemQueue
finishFetchItem(FetchItem, boolean) - Method in class org.apache.nutch.fetcher.FetchItemQueues
FIXED_INTERVAL_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
  Used by AdaptiveFetchSchedule to maintain custom fetch interval
fixHttpHeaders(String, int) - Static method in class org.apache.nutch.tools.WARCUtils
  Modify verbatim HTTP response headers: fix, remove or replace headers `Content-Length`, `Content-Encoding` and `Transfer-Encoding` which may confuse WARC readers.
flattenHashMap(HashMap<String, Integer>) - Static method in class org.apache.nutch.parsefilter.naivebayes.Train
followRedirects - Variable in class org.apache.nutch.indexer.IndexingFiltersChecker
followRedirects - Variable in class org.apache.nutch.parse.ParserChecker
FORBID_ALL_RULES - Static variable in class org.apache.nutch.protocol.RobotRulesParser
  A `BaseRobotRules` object appropriate for use when the `robots.txt` file is not fetched due to a `403/Forbidden` response; all requests are disallowed.
force - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
forceAsContentType - Variable in class org.apache.nutch.parse.ParserChecker
forceRefetch(Text, CrawlDatum, boolean) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
  This method resets fetchTime, fetchInterval, modifiedTime, retriesSinceFetch and page signature, so that it forces refetching.
forceRefetch(Text, CrawlDatum, boolean) - Method in interface org.apache.nutch.crawl.FetchSchedule
  This method resets fetchTime, fetchInterval, modifiedTime and page signature, so that it forces refetching.
FORMAT - Static variable in interface org.apache.nutch.metadata.DublinCore
  Typically, Format may include the media-type or dimensions of the resource.
FORMAT - Static variable in class org.apache.nutch.net.protocols.HttpDateFormat
FORMAT - Static variable in class org.apache.nutch.tools.WARCUtils
forName(String) - Method in class org.apache.nutch.util.MimeUtil
  A facade interface to Tika's underlying `MimeTypes.forName(String)` method.
FreeGenerator - Class in org.apache.nutch.tools
  This tool generates fetchlists (segments to be fetched) from plain text files containing one URL per line.
FreeGenerator() - Constructor for class org.apache.nutch.tools.FreeGenerator
FreeGenerator.FG - Class in org.apache.nutch.tools
FreeGenerator.FG.FGMapper - Class in org.apache.nutch.tools
FreeGenerator.FG.FGReducer - Class in org.apache.nutch.tools
fromBytes(byte[]) - Static method in class org.apache.nutch.metrics.LatencyTracker
  Deserializes a MergingDigest from bytes (as produced by `LatencyTracker.toBytes()`).
fromHexString(String) - Static method in class org.apache.nutch.util.StringUtil
  Convert a String containing consecutive (no inside whitespace) hexadecimal digits into a corresponding byte array.
FSUtils - Class in org.apache.nutch.util
  Utility methods for common filesystem operations.
FSUtils() - Constructor for class org.apache.nutch.util.FSUtils
Ftp - Class in org.apache.nutch.protocol.ftp
  This class is a protocol plugin used for ftp: scheme.
Ftp() - Constructor for class org.apache.nutch.protocol.ftp.Ftp
FtpError - Exception in org.apache.nutch.protocol.ftp
  Thrown for Ftp error codes.
FtpError(int) - Constructor for exception org.apache.nutch.protocol.ftp.FtpError
FtpException - Exception in org.apache.nutch.protocol.ftp
  Superclass for important exceptions thrown during FTP talk, that must be handled with care.
FtpException() - Constructor for exception org.apache.nutch.protocol.ftp.FtpException
FtpException(String) - Constructor for exception org.apache.nutch.protocol.ftp.FtpException
FtpException(String, Throwable) - Constructor for exception org.apache.nutch.protocol.ftp.FtpException
FtpException(Throwable) - Constructor for exception org.apache.nutch.protocol.ftp.FtpException
FtpExceptionBadSystResponse - Exception in org.apache.nutch.protocol.ftp
  Exception indicating bad reply of SYST command.
FtpExceptionCanNotHaveDataConnection - Exception in org.apache.nutch.protocol.ftp
  Exception indicating failure of opening data connection.
FtpExceptionControlClosedByForcedDataClose - Exception in org.apache.nutch.protocol.ftp
  Exception indicating control channel is closed by server end, due to forced closure of data channel at client (our) end.
FtpExceptionUnknownForcedDataClose - Exception in org.apache.nutch.protocol.ftp
  Exception indicating unrecognizable reply from server after forced closure of data channel by client (our) side.
FtpResponse - Class in org.apache.nutch.protocol.ftp
  FtpResponse.java mimics ftp replies as http response.
FtpResponse(URL, CrawlDatum, Ftp, Configuration) - Constructor for class org.apache.nutch.protocol.ftp.FtpResponse
FtpRobotRulesParser - Class in org.apache.nutch.protocol.ftp
  This class is used for parsing robots for urls belonging to FTP protocol.
FtpRobotRulesParser(Configuration) - Constructor for class org.apache.nutch.protocol.ftp.FtpRobotRulesParser

## G

generate(Path, Path, int, long, long) - Method in class org.apache.nutch.crawl.Generator
generate(Path, Path, int, long, long, boolean, boolean) - Method in class org.apache.nutch.crawl.Generator
  Deprecated.
  since 1.19 use `Generator.generate(Path, Path, int, long, long, boolean, boolean, boolean, int, String, String)` or `Generator.generate(Path, Path, int, long, long, boolean, boolean, boolean, int, String)` in the instance that no hostdb is available
generate(Path, Path, int, long, long, boolean, boolean, boolean, int, String) - Method in class org.apache.nutch.crawl.Generator
  This signature should be used in the instance that no hostdb is available.
generate(Path, Path, int, long, long, boolean, boolean, boolean, int, String, String) - Method in class org.apache.nutch.crawl.Generator
  Generate fetchlists in one or more segments.
GENERATE_DIR_NAME - Static variable in class org.apache.nutch.crawl.CrawlDatum
GENERATE_TIME_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
GENERATE_UPDATE_CRAWLDB - Static variable in class org.apache.nutch.crawl.Generator
generated - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
generateJson() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
generateJson() - Method in class org.apache.nutch.tools.CommonCrawlFormatJackson
generateJson() - Method in class org.apache.nutch.tools.CommonCrawlFormatJettinson
generateJson() - Method in class org.apache.nutch.tools.CommonCrawlFormatSimple
generateJson() - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
generateSegmentName() - Static method in class org.apache.nutch.crawl.Generator
generateSegmentName() - Static method in class org.apache.nutch.tools.arc.ArcSegmentCreator
  Generates a random name for the segments.
generateWARC(String, List<Path>, boolean, boolean, boolean) - Method in class org.apache.nutch.tools.warc.WARCExporter
generator - Static variable in class org.apache.nutch.tools.WARCUtils
Generator - Class in org.apache.nutch.crawl
  Generates a subset of a CrawlDb to fetch.
Generator() - Constructor for class org.apache.nutch.crawl.Generator
Generator(Configuration) - Constructor for class org.apache.nutch.crawl.Generator
GENERATOR_COUNT_MODE - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_COUNT_VALUE_DOMAIN - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_COUNT_VALUE_HOST - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_CUR_TIME - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_DELAY - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_EXPR - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_EXPR_REJECTED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs rejected by JEXL expression.
GENERATOR_FETCH_DELAY_EXPR - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_FILTER - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_HOSTDB - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_HOSTS_AFFECTED_PER_HOST_OVERFLOW_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Hosts affected by per-host overflow.
GENERATOR_INTERVAL_REJECTED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs rejected due to fetch interval exceeding threshold.
GENERATOR_MAX_COUNT - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_MAX_COUNT_EXPR - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_MAX_NUM_SEGMENTS - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_MIN_INTERVAL - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_MIN_SCORE - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_NORMALISE - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_RESTRICT_STATUS - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_SCHEDULE_REJECTED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs rejected by fetch schedule.
GENERATOR_SCORE_TOO_LOW_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs rejected due to score below threshold.
GENERATOR_STATUS_REJECTED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs rejected due to status restriction.
GENERATOR_TOP_N - Static variable in class org.apache.nutch.crawl.Generator
GENERATOR_URL_FILTERS_REJECTED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs rejected by URL filters.
GENERATOR_URLS_SKIPPED_PER_HOST_OVERFLOW_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs skipped due to per-host overflow.
GENERATOR_WAIT_FOR_UPDATE_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs waiting for CrawlDb update.
Generator.CrawlDbUpdater - Class in org.apache.nutch.crawl
  Update the CrawlDB so that the next generate won't include the same URLs.
Generator.CrawlDbUpdater.CrawlDbUpdateMapper - Class in org.apache.nutch.crawl
Generator.CrawlDbUpdater.CrawlDbUpdateReducer - Class in org.apache.nutch.crawl
Generator.DecreasingFloatComparator - Class in org.apache.nutch.crawl
Generator.HashComparator - Class in org.apache.nutch.crawl
  Sort fetch lists by hash of URL.
Generator.PartitionReducer - Class in org.apache.nutch.crawl
Generator.Selector - Class in org.apache.nutch.crawl
  Selects entries due for fetch.
Generator.SelectorEntry - Class in org.apache.nutch.crawl
Generator.SelectorInverseMapper - Class in org.apache.nutch.crawl
Generator.SelectorMapper - Class in org.apache.nutch.crawl
  Select and invert subset due for fetch.
Generator.SelectorReducer - Class in org.apache.nutch.crawl
  Collect until limit is reached.
generatorSortValue(Text, CrawlDatum, float) - Method in class org.apache.nutch.scoring.AbstractScoringFilter
generatorSortValue(Text, CrawlDatum, float) - Method in class org.apache.nutch.scoring.depth.DepthScoringFilter
generatorSortValue(Text, CrawlDatum, float) - Method in class org.apache.nutch.scoring.link.LinkAnalysisScoringFilter
generatorSortValue(Text, CrawlDatum, float) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
  Use `CrawlDatum.getScore()`.
generatorSortValue(Text, CrawlDatum, float) - Method in interface org.apache.nutch.scoring.ScoringFilter
  This method prepares a sort value for the purpose of sorting and selecting top N scoring pages during fetchlist generation.
generatorSortValue(Text, CrawlDatum, float) - Method in class org.apache.nutch.scoring.ScoringFilters
  Calculate a sort value for Generate.
GenericWritableConfigurable - Class in org.apache.nutch.util
  A generic Writable wrapper that can inject Configuration to `Configurable`s
GenericWritableConfigurable() - Constructor for class org.apache.nutch.util.GenericWritableConfigurable
GeoIPDocumentCreator - Class in org.apache.nutch.indexer.geoip
  Simple utility class which builds a `NutchDocument` based on input from `GeoIPIndexingFilter`, where configuration is also read.
GeoIPIndexingFilter - Class in org.apache.nutch.indexer.geoip
  This plugin implements an indexing filter which takes advantage of the GeoIP2-java API.
GeoIPIndexingFilter() - Constructor for class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter
  Default constructor for this plugin
GeoIPIndexingFilter.DatabaseType - Enum Class in org.apache.nutch.indexer.geoip
  Supported GeoIP database types.
get(String) - Method in class org.apache.nutch.metadata.Metadata
  Get the value associated to a metadata name.
get(String) - Method in class org.apache.nutch.metadata.SpellCheckedMetadata
get(String) - Method in class org.apache.nutch.parse.ParseResult
  Retrieve a single parse output.
get(String) - Static method in class org.apache.nutch.segment.SegmentPart
  Create SegmentPart from a full path of a location inside any segment part.
get(String, String) - Method in class org.apache.nutch.indexer.IndexWriterParams
get(String, String, Configuration) - Method in class org.apache.nutch.crawl.CrawlDbReader
get(Configuration) - Static method in class org.apache.nutch.indexer.IndexWriters
get(Configuration) - Static method in class org.apache.nutch.plugin.PluginRepository
  Get a cached instance of the `PluginRepository`
get(Configuration) - Static method in class org.apache.nutch.util.ObjectCache
get(Path, Text, Writer, Map<String, List<Writable>>) - Method in class org.apache.nutch.segment.SegmentReader
get(Text) - Method in class org.apache.nutch.parse.ParseResult
  Retrieve a single parse output.
get(FileSplit) - Static method in class org.apache.nutch.segment.SegmentPart
  Create SegmentPart from a FileSplit.
getAccept() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getAcceptCharset() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getAcceptedIssuers() - Method in class org.apache.nutch.protocol.htmlunit.DummyX509TrustManager
getAcceptedIssuers() - Method in class org.apache.nutch.protocol.http.DummyX509TrustManager
getAcceptedIssuers() - Method in class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
getAcceptedIssuers() - Method in class org.apache.nutch.protocol.interactiveselenium.DummyX509TrustManager
getAcceptedIssuers() - Method in class org.apache.nutch.protocol.selenium.DummyX509TrustManager
getAcceptLanguage() - Method in class org.apache.nutch.protocol.http.api.HttpBase
  Value of "Accept-Language" request header sent by Nutch.
getAdditionalPostHeaders() - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
getAgentString(String, String, String, String, String) - Static method in class org.apache.nutch.tools.WARCUtils
getAll() - Method in class org.apache.nutch.collection.CollectionManager
  Returns all collections
getAnchor() - Method in class org.apache.nutch.crawl.Inlink
getAnchor() - Method in class org.apache.nutch.parse.Outlink
getAnchor() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
getAnchors() - Method in class org.apache.nutch.crawl.Inlinks
  Get all anchor texts.
getAnchors(Text) - Method in class org.apache.nutch.crawl.LinkDbReader
getArgs() - Method in class org.apache.nutch.parse.ParseStatus
getArgs() - Method in class org.apache.nutch.protocol.ProtocolStatus
getAttribute(String) - Method in class org.apache.nutch.plugin.Extension
  Returns a attribute value, that is setuped in the manifest file and is definied by the extension point xml schema.
getAuthentication(String, Configuration) - Static method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
  This method is responsible for providing Basic authentication information.
getBase(Node) - Method in class org.apache.nutch.parse.html.DOMContentUtils
  If Node contains a BASE tag then it's HREF is returned.
getBase(Node) - Method in class org.apache.nutch.parse.tika.DOMContentUtils
  If Node contains a BASE tag then it's HREF is returned.
getBaseHref() - Method in class org.apache.nutch.parse.HTMLMetaTags
getBaseUrl() - Method in class org.apache.nutch.protocol.Content
  The base url for relative links contained in the content.
getBasicPattern() - Static method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
  Provides a pattern which can be used by an outside resource to determine if this class can provide credentials based on simple header information.
getBlackListString() - Method in class org.apache.nutch.collection.Subcollection
  Returns blacklist String
getBody() - Method in class org.apache.nutch.rabbitmq.RabbitMQMessage
getBoolean(String, boolean) - Method in class org.apache.nutch.indexer.IndexWriterParams
getBufferSize() - Method in class org.apache.nutch.protocol.ftp.Ftp
getCachedClass(PluginDescriptor, String) - Method in class org.apache.nutch.plugin.PluginRepository
getCacheKey(URL) - Static method in class org.apache.nutch.protocol.http.api.HttpRobotRulesParser
  Compose unique key to store and access robot rules in cache for given URL
getCharset(Metadata) - Static method in class org.apache.nutch.segment.SegmentReader
  Try to get HTML encoding from parse metadata.
getClassLoader() - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns a cached classloader for a plugin.
getClazz() - Method in class org.apache.nutch.exchange.ExchangeConfig
getClazz() - Method in class org.apache.nutch.plugin.Extension
  Returns the full class name of the extension point implementation
getClient(URL) - Method in class org.apache.nutch.protocol.okhttp.OkHttp
  Distribute hosts over clients by host name
getCode() - Method in interface org.apache.nutch.net.protocols.Response
  Get the response code.
getCode() - Method in class org.apache.nutch.protocol.file.FileResponse
  Get the response code.
getCode() - Method in class org.apache.nutch.protocol.ftp.FtpResponse
  Get the response code.
getCode() - Method in class org.apache.nutch.protocol.htmlunit.HttpResponse
getCode() - Method in class org.apache.nutch.protocol.http.HttpResponse
getCode() - Method in class org.apache.nutch.protocol.httpclient.HttpResponse
getCode() - Method in class org.apache.nutch.protocol.interactiveselenium.HttpResponse
getCode() - Method in class org.apache.nutch.protocol.okhttp.OkHttpResponse
getCode() - Method in class org.apache.nutch.protocol.ProtocolStatus
getCode() - Method in class org.apache.nutch.protocol.selenium.HttpResponse
getCode(int) - Method in exception org.apache.nutch.protocol.file.FileError
getCode(int) - Method in exception org.apache.nutch.protocol.ftp.FtpError
getCollectionManager(Configuration) - Static method in class org.apache.nutch.collection.CollectionManager
getCommand() - Method in class org.apache.nutch.util.CommandRunner
getCommonCrawlFormat(String, String, Content, Metadata, Configuration, CommonCrawlConfig) - Static method in class org.apache.nutch.tools.CommonCrawlFormatFactory
  Deprecated.
getCommonCrawlFormat(String, Configuration, CommonCrawlConfig) - Static method in class org.apache.nutch.tools.CommonCrawlFormatFactory
getConf() - Method in class org.apache.nutch.analysis.lang.HTMLLanguageParser
getConf() - Method in class org.apache.nutch.analysis.lang.LanguageIndexingFilter
getConf() - Method in class org.apache.nutch.crawl.Generator.Selector
getConf() - Method in class org.apache.nutch.crawl.Signature
getConf() - Method in class org.apache.nutch.crawl.URLPartitioner
getConf() - Method in class org.apache.nutch.exchange.jexl.JexlExchange
getConf() - Method in class org.apache.nutch.indexer.anchor.AnchorIndexingFilter
  Get the `Configuration` object
getConf() - Method in class org.apache.nutch.indexer.arbitrary.ArbitraryIndexingFilter
  Get the `Configuration` object
getConf() - Method in class org.apache.nutch.indexer.basic.BasicIndexingFilter
  Get the `Configuration` object
getConf() - Method in class org.apache.nutch.indexer.CleaningJob
getConf() - Method in class org.apache.nutch.indexer.feed.FeedIndexingFilter
getConf() - Method in class org.apache.nutch.indexer.filter.MimeTypeIndexingFilter
getConf() - Method in class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter
getConf() - Method in class org.apache.nutch.indexer.jexl.JexlIndexingFilter
getConf() - Method in class org.apache.nutch.indexer.links.LinksIndexingFilter
getConf() - Method in class org.apache.nutch.indexer.metadata.MetadataIndexer
getConf() - Method in class org.apache.nutch.indexer.more.MoreIndexingFilter
getConf() - Method in class org.apache.nutch.indexer.replace.ReplaceIndexer
getConf() - Method in class org.apache.nutch.indexer.staticfield.StaticFieldIndexer
  Get the `Configuration` object
getConf() - Method in class org.apache.nutch.indexer.subcollection.SubcollectionIndexingFilter
getConf() - Method in class org.apache.nutch.indexer.tld.TLDIndexingFilter
getConf() - Method in class org.apache.nutch.indexer.urlmeta.URLMetaIndexingFilter
getConf() - Method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
getConf() - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
getConf() - Method in class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
getConf() - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
getConf() - Method in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
getConf() - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
getConf() - Method in class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
getConf() - Method in class org.apache.nutch.indexwriter.solr.SolrIndexWriter
getConf() - Method in class org.apache.nutch.microformats.reltag.RelTagIndexingFilter
getConf() - Method in class org.apache.nutch.microformats.reltag.RelTagParser
getConf() - Method in class org.apache.nutch.net.protocols.ProtocolLogUtil
getConf() - Method in class org.apache.nutch.net.urlnormalizer.ajax.AjaxURLNormalizer
getConf() - Method in class org.apache.nutch.net.urlnormalizer.basic.BasicURLNormalizer
getConf() - Method in class org.apache.nutch.net.urlnormalizer.host.HostURLNormalizer
getConf() - Method in class org.apache.nutch.net.urlnormalizer.pass.PassURLNormalizer
getConf() - Method in class org.apache.nutch.net.urlnormalizer.protocol.ProtocolURLNormalizer
getConf() - Method in class org.apache.nutch.net.urlnormalizer.querystring.QuerystringURLNormalizer
getConf() - Method in class org.apache.nutch.net.urlnormalizer.slash.SlashURLNormalizer
getConf() - Method in class org.apache.nutch.parse.ext.ExtParser
getConf() - Method in class org.apache.nutch.parse.feed.FeedParser
getConf() - Method in class org.apache.nutch.parse.headings.HeadingsParseFilter
getConf() - Method in class org.apache.nutch.parse.html.HtmlParser
getConf() - Method in class org.apache.nutch.parse.js.JSParseFilter
getConf() - Method in class org.apache.nutch.parse.metatags.MetaTagsParser
getConf() - Method in class org.apache.nutch.parse.tika.TikaParser
getConf() - Method in class org.apache.nutch.parse.zip.ZipParser
getConf() - Method in class org.apache.nutch.parsefilter.debug.DebugParseFilter
getConf() - Method in class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
getConf() - Method in class org.apache.nutch.parsefilter.regex.RegexParseFilter
getConf() - Method in class org.apache.nutch.protocol.file.File
  Get the `Configuration` object
getConf() - Method in class org.apache.nutch.protocol.ftp.Ftp
  Get the `Configuration` object
getConf() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getConf() - Method in class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
getConf() - Method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
getConf() - Method in class org.apache.nutch.protocol.RobotRulesParser
  Get the `Configuration` object
getConf() - Method in class org.apache.nutch.publisher.NutchPublishers
getConf() - Method in class org.apache.nutch.publisher.rabbitmq.RabbitMQPublisherImpl
getConf() - Method in class org.apache.nutch.scoring.AbstractScoringFilter
getConf() - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
getConf() - Method in class org.apache.nutch.scoring.similarity.SimilarityScoringFilter
getConf() - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
getConf() - Method in class org.apache.nutch.urlfilter.domain.DomainURLFilter
getConf() - Method in class org.apache.nutch.urlfilter.domaindenylist.DomainDenylistURLFilter
getConf() - Method in class org.apache.nutch.urlfilter.fast.FastURLFilter
getConf() - Method in class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
getConf() - Method in class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
getConf() - Method in class org.apache.nutch.urlfilter.validator.UrlValidator
getConf() - Method in class org.apache.nutch.util.GenericWritableConfigurable
getConf() - Method in class org.creativecommons.nutch.CCIndexingFilter
getConf() - Method in class org.creativecommons.nutch.CCParseFilter
getConnectionFailures() - Method in class org.apache.nutch.hostdb.HostDatum
getContent() - Method in interface org.apache.nutch.net.protocols.Response
  Get the full content of the response.
getContent() - Method in class org.apache.nutch.protocol.Content
  The binary content retrieved.
getContent() - Method in class org.apache.nutch.protocol.file.FileResponse
getContent() - Method in class org.apache.nutch.protocol.ftp.FtpResponse
getContent() - Method in class org.apache.nutch.protocol.htmlunit.HttpResponse
getContent() - Method in class org.apache.nutch.protocol.http.HttpResponse
getContent() - Method in class org.apache.nutch.protocol.httpclient.HttpResponse
getContent() - Method in class org.apache.nutch.protocol.interactiveselenium.HttpResponse
getContent() - Method in class org.apache.nutch.protocol.okhttp.OkHttpResponse
getContent() - Method in class org.apache.nutch.protocol.ProtocolOutput
getContent() - Method in class org.apache.nutch.protocol.selenium.HttpResponse
getContentMeta() - Method in class org.apache.nutch.parse.ParseData
  The original `Metadata` retrieved from content
getContentType() - Method in exception org.apache.nutch.parse.ParserNotFound
getContentType() - Method in class org.apache.nutch.protocol.Content
  The media type of the retrieved content.
getContentType() - Method in class org.apache.nutch.rabbitmq.RabbitMQMessage
getCookie() - Method in class org.apache.nutch.fetcher.FetchItemQueue
getCookie(URL) - Method in class org.apache.nutch.protocol.http.api.HttpBase
  If per-host cookies are configured, this method will look it up for the given url.
getCookiePolicy() - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
getCookies() - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthentication
getCount() - Method in class org.apache.nutch.metrics.LatencyTracker
  Returns the number of recorded samples.
getCount(ErrorTracker.ErrorType) - Method in class org.apache.nutch.metrics.ErrorTracker
  Returns the count for a specific error type.
getCounterName(Throwable) - Static method in class org.apache.nutch.metrics.ErrorTracker
  Gets the counter name for a throwable based on its categorization.
getCounterName(ErrorTracker.ErrorType) - Static method in class org.apache.nutch.metrics.ErrorTracker
  Gets the counter name constant for a given error type.
getCredentials() - Method in interface org.apache.nutch.protocol.httpclient.HttpAuthentication
  Gets the credentials generated by the HttpAuthentication object.
getCredentials() - Method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
  Gets the Basic credentials generated by this HttpBasicAuthentication object
getCurrentKey() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
getCurrentNode() - Method in class org.apache.nutch.parse.html.DOMBuilder
  Get the node currently being processed.
getCurrentNode() - Method in class org.apache.nutch.util.NodeWalker
  Return the current node.
getCurrentValue() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
getCustomMaxInterval(Text) - Method in class org.apache.nutch.crawl.AdaptiveFetchSchedule
  Returns the custom max.
getCustomMinInterval(Text) - Method in class org.apache.nutch.crawl.AdaptiveFetchSchedule
  Returns the custom min.
getCustomRequestHeaders() - Method in class org.apache.nutch.protocol.okhttp.OkHttp
getData() - Method in interface org.apache.nutch.parse.Parse
  Other data extracted from the page.
getData() - Method in class org.apache.nutch.parse.ParseImpl
getDatum() - Method in class org.apache.nutch.fetcher.FetchItem
getDependencies() - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns a array of plugin ids.
getDescriptor() - Method in class org.apache.nutch.plugin.Extension
  Get the plugin descriptor.
getDescriptor() - Method in class org.apache.nutch.plugin.Plugin
  Returns the plugin descriptor
getDnsFailures() - Method in class org.apache.nutch.hostdb.HostDatum
getDocumentMeta() - Method in class org.apache.nutch.indexer.NutchDocument
getDom(InputStream) - Static method in class org.apache.nutch.util.DomUtil
  Returns parsed dom tree or null if any error
getDomainName(String) - Static method in class org.apache.nutch.util.URLUtil
  Returns the domain name of the URL.
getDomainName(URL) - Static method in class org.apache.nutch.util.URLUtil
  Get the domain name of the URL.
getDomainSuffix(String) - Static method in class org.apache.nutch.util.URLUtil
  Returns the domain suffix corresponding to the last public part of the hostname.
getDomainSuffix(URL) - Static method in class org.apache.nutch.util.URLUtil
  Returns the public suffix corresponding to the last public part of the hostname.
getDriverForPage(String, Configuration) - Static method in class org.apache.nutch.protocol.htmlunit.HtmlUnitWebDriver
getDriverForPage(String, Configuration) - Static method in class org.apache.nutch.protocol.selenium.HttpWebClient
getDuplicate(CrawlDatum, CrawlDatum) - Method in class org.apache.nutch.crawl.DeduplicationJob.DedupReducer
getElement(DocumentFragment, String) - Method in class org.apache.nutch.parse.headings.HeadingsParseFilter
  Finds the specified element and returns its value
getEmptyParse(Configuration) - Method in class org.apache.nutch.parse.ParseStatus
  Creates an empty `Parse` instance containing the status
getEmptyParseResult(String, Configuration) - Method in class org.apache.nutch.parse.ParseStatus
  Creates an empty `ParseResult` for a given URL
getEventData() - Method in class org.apache.nutch.fetcher.FetcherThreadEvent
  Get event data
getEventType() - Method in class org.apache.nutch.fetcher.FetcherThreadEvent
  Get type of this event object
getExemptions() - Method in class org.apache.nutch.urlfilter.ignoreexempt.ExemptionUrlFilter
getExitValue() - Method in class org.apache.nutch.util.CommandRunner
getExportedLibUrls() - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns a array of exported libs as URLs
getExtensionInstance() - Method in class org.apache.nutch.plugin.Extension
  Return an instance of the extension implementation.
getExtensionPoint(String) - Method in class org.apache.nutch.plugin.PluginRepository
  Returns a extension point identified by a extension point id.
getExtensions() - Method in class org.apache.nutch.plugin.ExtensionPoint
  Returns a array of extensions that lsiten to this extension point
getExtensions() - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns an array of extensions.
getExtensions(String) - Method in class org.apache.nutch.parse.ParserFactory
  Finds the best-suited parse plugin for a given contentType.
getExtenstionPoints() - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns a array of extension points.
getFetched() - Method in class org.apache.nutch.hostdb.HostDatum
getFetchInterval() - Method in class org.apache.nutch.crawl.CrawlDatum
getFetchItem() - Method in class org.apache.nutch.fetcher.FetchItemQueue
getFetchItem() - Method in class org.apache.nutch.fetcher.FetchItemQueues
getFetchItemQueue(String) - Method in class org.apache.nutch.fetcher.FetchItemQueues
getFetchLatencyTracker() - Method in class org.apache.nutch.fetcher.FetcherThread
  Returns the fetch latency tracker for this thread so the mapper can merge all thread trackers and emit job-level percentiles.
getFetchSchedule(Configuration) - Static method in class org.apache.nutch.crawl.FetchScheduleFactory
  Return the FetchSchedule implementation specified within the given `Configuration`, or `DefaultFetchSchedule` by default.
getFetchTime() - Method in class org.apache.nutch.crawl.CrawlDatum
  Get the fetch time.
getField(String) - Method in class org.apache.nutch.indexer.NutchDocument
getFieldName() - Method in class org.apache.nutch.indexer.replace.FieldReplacer
getFieldNames() - Method in class org.apache.nutch.indexer.NutchDocument
getFieldValue(String) - Method in class org.apache.nutch.indexer.NutchDocument
getFilters() - Method in class org.apache.nutch.net.URLFilters
getFromUrl() - Method in class org.apache.nutch.crawl.Inlink
getGeneralTags() - Method in class org.apache.nutch.parse.HTMLMetaTags
getGone() - Method in class org.apache.nutch.hostdb.HostDatum
getHeader(String) - Method in interface org.apache.nutch.net.protocols.Response
  Get the value of a named header.
getHeader(String) - Method in class org.apache.nutch.protocol.file.FileResponse
  Returns the value of a named header.
getHeader(String) - Method in class org.apache.nutch.protocol.ftp.FtpResponse
  Returns the value of a named header.
getHeader(String) - Method in class org.apache.nutch.protocol.htmlunit.HttpResponse
getHeader(String) - Method in class org.apache.nutch.protocol.http.HttpResponse
getHeader(String) - Method in class org.apache.nutch.protocol.httpclient.HttpResponse
getHeader(String) - Method in class org.apache.nutch.protocol.interactiveselenium.HttpResponse
getHeader(String) - Method in class org.apache.nutch.protocol.okhttp.OkHttpResponse
getHeader(String) - Method in class org.apache.nutch.protocol.selenium.HttpResponse
getHeaders() - Method in interface org.apache.nutch.net.protocols.Response
  Get all the headers.
getHeaders() - Method in class org.apache.nutch.protocol.htmlunit.HttpResponse
getHeaders() - Method in class org.apache.nutch.protocol.http.HttpResponse
getHeaders() - Method in class org.apache.nutch.protocol.httpclient.HttpResponse
getHeaders() - Method in class org.apache.nutch.protocol.interactiveselenium.HttpResponse
getHeaders() - Method in class org.apache.nutch.protocol.okhttp.OkHttpResponse
getHeaders() - Method in class org.apache.nutch.protocol.selenium.HttpResponse
getHeaders() - Method in class org.apache.nutch.rabbitmq.RabbitMQMessage
getHomepageUrl() - Method in class org.apache.nutch.hostdb.HostDatum
getHost(String) - Static method in class org.apache.nutch.util.URLUtil
  Returns the lowercased hostname for the URL or null if the URL is not well-formed.
getHost(URL) - Static method in class org.apache.nutch.util.URLUtil
  Returns the lowercased hostname for the URL.
getHostname(Configuration) - Static method in class org.apache.nutch.tools.WARCUtils
getHostName(String) - Static method in class org.apache.nutch.crawl.AdaptiveFetchSchedule
  Strip a URL, leaving only the hostname.
getHostSegments(String) - Static method in class org.apache.nutch.util.URLUtil
  Partitions of the hostname of the url by "."
getHostSegments(URL) - Static method in class org.apache.nutch.util.URLUtil
  Partitions of the hostname of the url by "."
getHTMLContent(WebDriver, Configuration) - Static method in class org.apache.nutch.protocol.htmlunit.HtmlUnitWebDriver
getHtmlPage(String) - Static method in class org.apache.nutch.protocol.selenium.HttpWebClient
getHtmlPage(String, Configuration) - Static method in class org.apache.nutch.protocol.htmlunit.HtmlUnitWebDriver
  Function for obtaining the HTML BODY using the selected selenium webdriver There are a number of configuration properties within `nutch-site.xml` which determine whether to take screenshots of the rendered pages and persist them as timestamped .png's into HDFS.
getHtmlPage(String, Configuration) - Static method in class org.apache.nutch.protocol.selenium.HttpWebClient
  Function for obtaining the HTML using the selected selenium webdriver There are a number of configuration properties within `nutch-site.xml` which determine whether to take screenshots of the rendered pages and persist them as timestamped .png's into HDFS.
getHttpEquivTags() - Method in class org.apache.nutch.parse.HTMLMetaTags
getId() - Method in class org.apache.nutch.collection.Subcollection
getId() - Method in class org.apache.nutch.exchange.ExchangeConfig
getId() - Method in class org.apache.nutch.plugin.Extension
  Return the unique id of the extension.
getId() - Method in class org.apache.nutch.plugin.ExtensionPoint
  Returns the unique id of the extension point.
getID(String) - Static method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchUtils
  Returns a normalised doc ID based on the URL of a document
getImported() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getInlinks(Text) - Method in class org.apache.nutch.crawl.LinkDbReader
getInLinks() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getInLinks() - Method in interface org.apache.nutch.tools.CommonCrawlFormat
  gets set of inlinks
getInlinkScore() - Method in class org.apache.nutch.scoring.webgraph.Node
getInProgressSize() - Method in class org.apache.nutch.fetcher.FetchItemQueue
getInstance() - Static method in class org.apache.nutch.plugin.URLStreamHandlerFactory
  Get the singleton instance of this class.
getInstance() - Static method in class org.apache.nutch.urlfilter.fast.FastURLFilter.DenyAllRule
getInstance(Element) - Static method in class org.apache.nutch.exchange.ExchangeConfig
getInt(String, int) - Method in class org.apache.nutch.indexer.IndexWriterParams
getIPAddress(Configuration) - Static method in class org.apache.nutch.tools.WARCUtils
getJobFailureLogMessage(String, Job) - Static method in class org.apache.nutch.util.NutchJob
  Method to return job failure log message.
getJsonArray() - Method in class org.apache.nutch.tools.CommonCrawlConfig
getJsonData() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getJsonData() - Method in interface org.apache.nutch.tools.CommonCrawlFormat
  Get a string representation of the JSON structure of the URL content.
getJsonData() - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
getJsonData(String, Content, Metadata) - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getJsonData(String, Content, Metadata) - Method in interface org.apache.nutch.tools.CommonCrawlFormat
  Returns a string representation of the JSON structure of the URL content.
getJsonData(String, Content, Metadata, ParseData) - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getJsonData(String, Content, Metadata, ParseData) - Method in interface org.apache.nutch.tools.CommonCrawlFormat
  Returns a string representation of the JSON structure of the URL content.
getJsonData(String, Content, Metadata, ParseData) - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
getKey() - Method in class org.apache.nutch.collection.Subcollection
getKey() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getKeyPrefix() - Method in class org.apache.nutch.tools.CommonCrawlConfig
getL2Norm() - Method in class org.apache.nutch.scoring.similarity.cosine.DocVector
getLastCheck() - Method in class org.apache.nutch.hostdb.HostDatum
getLastModified() - Method in class org.apache.nutch.protocol.ProtocolStatus
getLinks() - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
getLinkType() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
getLoginFormId() - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
getLoginPostData() - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
getLoginUrl() - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
getLong(String, long) - Method in class org.apache.nutch.indexer.IndexWriterParams
getMajorCode() - Method in class org.apache.nutch.parse.ParseStatus
getMaxContent() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getMaxDuration() - Method in class org.apache.nutch.protocol.http.api.HttpBase
  The time limit to download the entire content, in seconds.
getMessage() - Method in class org.apache.nutch.parse.ParseStatus
getMessage() - Method in class org.apache.nutch.protocol.ProtocolStatus
getMeta(String) - Method in class org.apache.nutch.metadata.MetaWrapper
  Get metadata value for a given key.
getMeta(String) - Method in class org.apache.nutch.parse.ParseData
  Get a metadata single value.
getMetadata() - Method in class org.apache.nutch.crawl.Inlink
getMetadata() - Method in class org.apache.nutch.metadata.MetaWrapper
  Get all metadata.
getMetadata() - Method in class org.apache.nutch.parse.Outlink
getMetadata() - Method in class org.apache.nutch.protocol.Content
  Other protocol-specific data.
getMetadata() - Method in class org.apache.nutch.scoring.webgraph.Node
getMetaData() - Method in class org.apache.nutch.crawl.CrawlDatum
  Get CrawlDatum metadata
getMetaData() - Method in class org.apache.nutch.hostdb.HostDatum
  Get Host metadata.
getMetaTags(HTMLMetaTags, Node, URL) - Static method in class org.apache.nutch.parse.html.HTMLMetaProcessor
  Sets the indicators in `robotsMeta` to appropriate values, based on any META tags found under the given `node`.
getMetaTags(HTMLMetaTags, Node, URL) - Static method in class org.apache.nutch.parse.tika.HTMLMetaProcessor
  Sets the indicators in `robotsMeta` to appropriate values, based on any META tags found under the given `node`.
getMetaValues(String) - Method in class org.apache.nutch.metadata.MetaWrapper
  Get multiple metadata values for a given key.
getMethod() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getMimeType(File) - Method in class org.apache.nutch.util.MimeUtil
  Facade interface to Tika's underlying `MimeTypes.getMimeType(File)` method.
getMimeType(String) - Method in class org.apache.nutch.util.MimeUtil
  Facade interface to Tika's underlying `MimeTypes.getMimeType(String)` method.
getMinorCode() - Method in class org.apache.nutch.parse.ParseStatus
getModifiedTime() - Method in class org.apache.nutch.crawl.CrawlDatum
getName() - Method in class org.apache.nutch.collection.Subcollection
getName() - Method in class org.apache.nutch.plugin.ExtensionPoint
  Returns the name of the extension point.
getName() - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns the name of the plugin.
getName() - Method in class org.apache.nutch.protocol.ProtocolStatus
getNoCache() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Get the current value of `noCache`.
getNode() - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
getNodeValue(Node) - Static method in class org.apache.nutch.parse.headings.HeadingsParseFilter
  Returns the text value of the specified Node and child nodes
getNoFollow() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Get the current value of `noFollow`.
getNoIndex() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Get the current value of `noIndex`.
getNormalizedName(String) - Static method in class org.apache.nutch.metadata.SpellCheckedMetadata
  Get the normalized name of metadata attribute name.
getNotExportedLibUrls() - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns a array of libraries as URLs that are not exported by the plugin.
getNotModified() - Method in class org.apache.nutch.hostdb.HostDatum
getNumInlinks() - Method in class org.apache.nutch.scoring.webgraph.Node
getNumOutlinks() - Method in class org.apache.nutch.scoring.webgraph.Node
getObject(String) - Method in class org.apache.nutch.util.ObjectCache
getOrderedPlugins(Class<?>, String, String) - Method in class org.apache.nutch.plugin.PluginRepository
  Get ordered list of plugins.
getOutlinks() - Method in class org.apache.nutch.parse.ParseData
  Get the outlinks of the page.
getOutlinks(String, String, Configuration) - Static method in class org.apache.nutch.parse.OutlinkExtractor
  Extracts `Outlink` from given plain text and adds anchor to the extracted `Outlink`s
getOutlinks(String, Configuration) - Static method in class org.apache.nutch.parse.OutlinkExtractor
  Extracts `Outlink` from given plain text.
getOutlinks(URL, ArrayList<Outlink>, List<Link>) - Method in class org.apache.nutch.parse.tika.DOMContentUtils
getOutlinks(URL, ArrayList<Outlink>, Node) - Method in class org.apache.nutch.parse.html.DOMContentUtils
  This method finds all anchors below the supplied DOM `node`, and creates appropriate `Outlink` records for each (relative to the supplied `base` URL), and adds them to the `outlinks` `ArrayList`.
getOutlinks(URL, ArrayList<Outlink>, Node) - Method in class org.apache.nutch.parse.tika.DOMContentUtils
  This method finds all anchors below the supplied DOM `node`, and creates appropriate `Outlink` records for each (relative to the supplied `base` URL), and adds them to the `outlinks` `ArrayList`.
getOutlinkScore() - Method in class org.apache.nutch.scoring.webgraph.Node
getOutputCommitter(TaskAttemptContext) - Method in class org.apache.nutch.indexer.IndexerOutputFormat
getOutputCommitter(TaskAttemptContext) - Method in class org.apache.nutch.parse.ParseOutputFormat
getOutputDir() - Method in class org.apache.nutch.tools.CommonCrawlConfig
getPage(String) - Static method in class org.apache.nutch.util.URLUtil
  Returns the page for the url.
getParameters() - Method in class org.apache.nutch.exchange.ExchangeConfig
getParse(Content) - Method in class org.apache.nutch.parse.ext.ExtParser
getParse(Content) - Method in class org.apache.nutch.parse.feed.FeedParser
  Parses the given feed and extracts out and parsers all linked items within the feed, using the underlying ROME feed parsing library.
getParse(Content) - Method in class org.apache.nutch.parse.html.HtmlParser
getParse(Content) - Method in class org.apache.nutch.parse.js.JSParseFilter
  Parse a JavaScript file and extract outlinks
getParse(Content) - Method in interface org.apache.nutch.parse.Parser
  This method parses the given content and returns a map of <key, parse> pairs.
getParse(Content) - Method in class org.apache.nutch.parse.tika.TikaParser
getParse(Content) - Method in class org.apache.nutch.parse.zip.ZipParser
getParseMeta() - Method in class org.apache.nutch.parse.ParseData
  Other content properties.
getParserById(String) - Method in class org.apache.nutch.parse.ParserFactory
  Function returns a `Parser` instance with the specified `extId`, representing its extension ID.
getParsers(String, String) - Method in class org.apache.nutch.parse.ParserFactory
  Function returns an array of `Parser`s for a given content type.
getPartition(FloatWritable, Writable, int) - Method in class org.apache.nutch.crawl.Generator.Selector
  Partition by host / domain or IP.
getPartition(Text, Writable, int) - Method in class org.apache.nutch.crawl.URLPartitioner
  Hash by host or domain name or IP address.
getPartition(Text, NutchWritable, int) - Method in class org.apache.nutch.parse.ParseSegment.ParseSegmentPartitioner
getPassAllFilter() - Static method in class org.apache.nutch.util.HadoopFSUtil
  Get a path filter which allows all paths.
getPassDirectoriesFilter(FileSystem) - Static method in class org.apache.nutch.util.HadoopFSUtil
  Get a path filter which allows all directories.
getPaths(FileStatus[]) - Static method in class org.apache.nutch.util.HadoopFSUtil
  Turns an array of FileStatus into an array of Paths.
getPattern() - Method in class org.apache.nutch.indexer.replace.FieldReplacer
getPercentile(double) - Method in class org.apache.nutch.metrics.LatencyTracker
  Returns the percentile value for the given quantile.
getPluginClass() - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns the fully qualified name of the class which implements the abstarct `Plugin` class.
getPluginDescriptor(String) - Method in class org.apache.nutch.plugin.PluginRepository
  Returns the descriptor of one plugin identified by a plugin id.
getPluginDescriptors() - Method in class org.apache.nutch.plugin.PluginRepository
  Returns all registed plugin descriptors.
getPluginFolder(String) - Method in class org.apache.nutch.plugin.PluginManifestParser
  Return the named plugin folder.
getPluginId() - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns the unique identifier of the plug-in or `null`.
getPluginInstance(PluginDescriptor) - Method in class org.apache.nutch.plugin.PluginRepository
  Returns an instance of a plugin.
getPluginPath() - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns the directory path of the plugin.
getPos() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
  Returns the current position in the file.
getProgress() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
  Returns the percentage of progress in processing the file.
getProgress() - Method in class org.apache.nutch.util.NutchTool
  Get relative progress of the tool.
getPropertyName() - Method in enum class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter.DatabaseType
getProtocol(String) - Method in class org.apache.nutch.protocol.ProtocolFactory
  Returns the appropriate `Protocol` implementation for a url.
getProtocol(String) - Static method in class org.apache.nutch.util.URLUtil
getProtocol(URL) - Method in class org.apache.nutch.protocol.ProtocolFactory
  Returns the appropriate `Protocol` implementation for a url.
getProtocol(URL) - Static method in class org.apache.nutch.util.URLUtil
getProtocolById(String) - Method in class org.apache.nutch.protocol.ProtocolFactory
getProtocolOutput(String, CrawlDatum, boolean) - Method in class org.apache.nutch.util.AbstractChecker
getProtocolOutput(Text, CrawlDatum) - Method in class org.apache.nutch.protocol.file.File
  Creates a `FileResponse` object corresponding to the url and return a `ProtocolOutput` object as per the content received
getProtocolOutput(Text, CrawlDatum) - Method in class org.apache.nutch.protocol.ftp.Ftp
  Creates a `FtpResponse` object corresponding to the url and returns a `ProtocolOutput` object as per the content received
getProtocolOutput(Text, CrawlDatum) - Method in class org.apache.nutch.protocol.http.api.HttpBase
getProtocolOutput(Text, CrawlDatum) - Method in interface org.apache.nutch.protocol.Protocol
  Get the `ProtocolOutput` for a given url and crawldatum
getProviderName() - Method in class org.apache.nutch.plugin.PluginDescriptor
getProxyHost() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getProxyPort() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getQueueCount() - Method in class org.apache.nutch.fetcher.FetchItemQueues
getQueueCountMaxExceptions() - Method in class org.apache.nutch.fetcher.FetchItemQueues
getQueueID() - Method in class org.apache.nutch.fetcher.FetchItem
getQueueSize() - Method in class org.apache.nutch.fetcher.FetchItemQueue
getReaders(Path, Configuration) - Static method in class org.apache.nutch.util.SegmentReaderUtil
getRealm() - Method in interface org.apache.nutch.protocol.httpclient.HttpAuthentication
  Gets the realm used by the HttpAuthentication object during creation.
getRealm() - Method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
  Gets the realm attribute of the HttpBasicAuthentication object.
getReason() - Method in class org.apache.nutch.protocol.okhttp.OkHttpResponse.TruncatedContent
getRecordReader(InputSplit, Job, Mapper.Context) - Method in class org.apache.nutch.segment.ContentAsTextInputFormat
getRecordReader(InputSplit, Job, Mapper.Context) - Method in class org.apache.nutch.tools.arc.ArcInputFormat
  Get the `RecordReader` for reading the arc file.
getRecordWriter(TaskAttemptContext) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumCsvOutputFormat
getRecordWriter(TaskAttemptContext) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumJsonOutputFormat
getRecordWriter(TaskAttemptContext) - Method in class org.apache.nutch.fetcher.FetcherOutputFormat
getRecordWriter(TaskAttemptContext) - Method in class org.apache.nutch.indexer.IndexerOutputFormat
getRecordWriter(TaskAttemptContext) - Method in class org.apache.nutch.parse.ParseOutputFormat
getRecordWriter(TaskAttemptContext) - Method in class org.apache.nutch.segment.SegmentMerger.SegmentOutputFormat
getRecordWriter(TaskAttemptContext) - Method in class org.apache.nutch.segment.SegmentReader.TextOutputFormat
getRedirPerm() - Method in class org.apache.nutch.hostdb.HostDatum
getRedirTemp() - Method in class org.apache.nutch.hostdb.HostDatum
getRefresh() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Get the current value of `refresh`.
getRefreshHref() - Method in class org.apache.nutch.parse.HTMLMetaTags
getRefreshTime() - Method in class org.apache.nutch.parse.HTMLMetaTags
getRemovedFormFields() - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
getReplacement() - Method in class org.apache.nutch.indexer.replace.FieldReplacer
getReprUrl() - Method in class org.apache.nutch.fetcher.FetcherThread
getRequestAccept() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getRequestAcceptEncoding() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getRequestAcceptLanguage() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getRequestContactEmail() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getRequestContactName() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getRequestHostAddress() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getRequestHostName() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getRequestRobots() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getRequestSoftware() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getRequestUserAgent() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getResource(String) - Method in class org.apache.nutch.plugin.PluginClassLoader
getResourceAsStream(String) - Method in class org.apache.nutch.plugin.PluginClassLoader
getResources(String) - Method in class org.apache.nutch.plugin.PluginClassLoader
getResourceString(String, Locale) - Method in class org.apache.nutch.plugin.PluginDescriptor
  Returns a I18N'd resource string.
getResponse(URL, CrawlDatum, boolean) - Method in class org.apache.nutch.protocol.htmlunit.Http
getResponse(URL, CrawlDatum, boolean) - Method in class org.apache.nutch.protocol.http.api.HttpBase
getResponse(URL, CrawlDatum, boolean) - Method in class org.apache.nutch.protocol.http.Http
getResponse(URL, CrawlDatum, boolean) - Method in class org.apache.nutch.protocol.httpclient.Http
  Fetches the `url` with a configured HTTP client and gets the response.
getResponse(URL, CrawlDatum, boolean) - Method in class org.apache.nutch.protocol.interactiveselenium.Http
getResponse(URL, CrawlDatum, boolean) - Method in class org.apache.nutch.protocol.okhttp.OkHttp
getResponse(URL, CrawlDatum, boolean) - Method in class org.apache.nutch.protocol.selenium.Http
getResponseAddress() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getResponseContent() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getResponseContentEncoding() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getResponseContentType() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getResponseDate() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getResponseHostName() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getResponseServer() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getResponseStatus() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getRetriesSinceFetch() - Method in class org.apache.nutch.crawl.CrawlDatum
getReversedHost(String) - Static method in class org.apache.nutch.util.TableUtil
  Given a reversed url, returns the reversed host E.g "com.foo.bar:http:8983/to/index.html?a=b" -> "com.foo.bar"
getReverseKey() - Method in class org.apache.nutch.tools.CommonCrawlConfig
getReverseKeyValue() - Method in class org.apache.nutch.tools.CommonCrawlConfig
getRobotRules(URL, CrawlDatum, List<Content>) - Method in class org.apache.nutch.protocol.file.File
  No robots parsing is done for file protocol.
getRobotRules(URL, CrawlDatum, List<Content>) - Method in class org.apache.nutch.protocol.ftp.Ftp
  Get the robots rules for a given url
getRobotRules(URL, CrawlDatum, List<Content>) - Method in class org.apache.nutch.protocol.http.api.HttpBase
getRobotRules(URL, CrawlDatum, List<Content>) - Method in interface org.apache.nutch.protocol.Protocol
  Retrieve robot rules applicable for this URL.
getRobotRules(Text, CrawlDatum, List<Content>) - Method in class org.apache.nutch.protocol.file.File
  No robots parsing is done for file protocol.
getRobotRules(Text, CrawlDatum, List<Content>) - Method in class org.apache.nutch.protocol.ftp.Ftp
  Get the robots rules for a given url
getRobotRules(Text, CrawlDatum, List<Content>) - Method in class org.apache.nutch.protocol.http.api.HttpBase
getRobotRules(Text, CrawlDatum, List<Content>) - Method in interface org.apache.nutch.protocol.Protocol
  Retrieve robot rules applicable for this URL.
getRobotRulesSet(Protocol, URL, List<Content>) - Method in class org.apache.nutch.protocol.ftp.FtpRobotRulesParser
  The hosts for which the caching of robots rules is yet to be done, it sends a Ftp request to the host corresponding to the `URL` passed, gets robots file, parses the rules and caches the rules object to avoid re-work in future.
getRobotRulesSet(Protocol, URL, List<Content>) - Method in class org.apache.nutch.protocol.http.api.HttpRobotRulesParser
  Get the rules from robots.txt which applies for the given `url`.
getRobotRulesSet(Protocol, URL, List<Content>) - Method in class org.apache.nutch.protocol.RobotRulesParser
  Fetch robots.txt (or it's protocol-specific equivalent) which applies to the given URL, parse it and return the set of robot rules applicable for the configured agent name(s).
getRobotRulesSet(Protocol, Text, List<Content>) - Method in class org.apache.nutch.protocol.RobotRulesParser
  Fetch robots.txt (or it's protocol-specific equivalent) which applies to the given URL, parse it and return the set of robot rules applicable for the configured agent name(s).
getRootNode() - Method in class org.apache.nutch.parse.html.DOMBuilder
  Get the root node of the DOM being created.
getRulesReader(Configuration) - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
  Returns the name of the file of rules to use for a particular implementation.
getRulesReader(Configuration) - Method in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
  Rules specified as a config property will override rules specified as a config file.
getRulesReader(Configuration) - Method in class org.apache.nutch.urlfilter.ignoreexempt.ExemptionUrlFilter
  Gets reader for regex rules
getRulesReader(Configuration) - Method in class org.apache.nutch.urlfilter.regex.RegexURLFilter
  Rules specified as a config property will override rules specified as a config file.
getSchema() - Method in class org.apache.nutch.plugin.ExtensionPoint
  Returns a path to the xml schema of a extension point.
getScopedRules() - Method in class org.apache.nutch.net.urlnormalizer.regex.RegexURLNormalizer
getScore() - Method in class org.apache.nutch.crawl.CrawlDatum
getScore() - Method in class org.apache.nutch.hostdb.HostDatum
getScore() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
getSignature() - Method in class org.apache.nutch.crawl.CrawlDatum
getSignature(Configuration) - Static method in class org.apache.nutch.crawl.SignatureFactory
  Return the `Signature` implementation for a given `Configuration`, or `MD5Signature` by default.
getSimpleDateFormat() - Method in class org.apache.nutch.tools.CommonCrawlConfig
getSplits(JobContext) - Method in class org.apache.nutch.fetcher.Fetcher.InputFormat
  Don't split inputs to keep things polite - a single fetch list must be processed in one fetcher task.
getStats(Path, SegmentReader.SegmentReaderStats) - Method in class org.apache.nutch.segment.SegmentReader
getStatus() - Method in class org.apache.nutch.crawl.CrawlDatum
getStatus() - Method in class org.apache.nutch.parse.ParseData
  Get the status of parsing the page.
getStatus() - Method in class org.apache.nutch.protocol.ProtocolOutput
getStatus() - Method in class org.apache.nutch.util.NutchTool
  Returns current status of the running tool
getStatusByName(String) - Static method in class org.apache.nutch.crawl.CrawlDatum
getStatusName(byte) - Static method in class org.apache.nutch.crawl.CrawlDatum
getStrings(String) - Method in class org.apache.nutch.indexer.IndexWriterParams
getStrings(String, String...) - Method in class org.apache.nutch.indexer.IndexWriterParams
getSubColection(String) - Method in class org.apache.nutch.collection.CollectionManager
  Get the named subcollection
getSubCollections(String) - Method in class org.apache.nutch.collection.CollectionManager
  Return names of collections url is part of
getSum() - Method in class org.apache.nutch.metrics.LatencyTracker
  Returns the sum of all recorded latencies.
getSystemName() - Method in class org.apache.nutch.protocol.ftp.Client
  Fetches the system type name from the server and returns the string.
getTargetPoint() - Method in class org.apache.nutch.plugin.Extension
  Get target point
getText() - Method in interface org.apache.nutch.parse.Parse
  The textual content of the page.
getText() - Method in class org.apache.nutch.parse.ParseImpl
getText() - Method in class org.apache.nutch.parse.ParseText
getText(StringBuffer, Node) - Method in class org.apache.nutch.parse.html.DOMContentUtils
  This is a convinience method, equivalent to `getText(sb, node, false)`.
getText(StringBuffer, Node) - Method in class org.apache.nutch.parse.tika.DOMContentUtils
  This is a convinience method, equivalent to `getText(sb, node, false)`.
getText(StringBuffer, Node, boolean) - Method in class org.apache.nutch.parse.html.DOMContentUtils
  This method takes a `StringBuffer` and a DOM `Node`, and will append all the content text found beneath the DOM node to the `StringBuffer`.
getThrownError() - Method in class org.apache.nutch.util.CommandRunner
getTimeout() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getTimeout() - Method in class org.apache.nutch.util.CommandRunner
getTimestamp() - Method in class org.apache.nutch.fetcher.FetcherThreadEvent
  Get timestamp of current event.
getTimestamp() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
getTimestamp() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getTitle() - Method in class org.apache.nutch.parse.ParseData
  Get the title of the page.
getTitle(StringBuffer, Node) - Method in class org.apache.nutch.parse.html.DOMContentUtils
  This method takes a `StringBuffer` and a DOM `Node`, and will append the content text found beneath the first `title` node to the `StringBuffer`.
getTitle(StringBuffer, Node) - Method in class org.apache.nutch.parse.tika.DOMContentUtils
  This method takes a `StringBuffer` and a DOM `Node`, and will append the content text found beneath the first `title` node to the `StringBuffer`.
getTlsPreferredCipherSuites() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getTlsPreferredProtocols() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getToFieldName() - Method in class org.apache.nutch.indexer.replace.FieldReplacer
getTokenStream() - Method in class org.apache.nutch.scoring.similarity.util.LuceneTokenizer
  get the tokenStream created by `Tokenizer`
getTopLevelDomainName(String) - Static method in class org.apache.nutch.util.URLUtil
  Returns the top-level domain name of the URL.
getTopLevelDomainName(URL) - Static method in class org.apache.nutch.util.URLUtil
  Returns the top-level domain name of the URL.
getTotalCount() - Method in class org.apache.nutch.metrics.ErrorTracker
  Returns the total count of all errors.
getTotalSize() - Method in class org.apache.nutch.fetcher.FetchItemQueues
getToUrl() - Method in class org.apache.nutch.parse.Outlink
getTypes() - Method in class org.apache.nutch.crawl.NutchWritable
getUnfetched() - Method in class org.apache.nutch.hostdb.HostDatum
getUniqueFile(TaskAttemptContext, String) - Method in class org.apache.nutch.parse.ParseOutputFormat
getUrl() - Method in class org.apache.nutch.fetcher.FetcherThreadEvent
  Get URL of this event
getUrl() - Method in class org.apache.nutch.fetcher.FetchItem
getUrl() - Method in interface org.apache.nutch.net.protocols.Response
  Get the URL the protocol actually used when requesting the Response.
getUrl() - Method in exception org.apache.nutch.parse.ParserNotFound
getUrl() - Method in class org.apache.nutch.protocol.Content
  The url fetched.
getUrl() - Method in class org.apache.nutch.protocol.htmlunit.HttpResponse
getUrl() - Method in class org.apache.nutch.protocol.http.HttpResponse
getUrl() - Method in class org.apache.nutch.protocol.httpclient.HttpResponse
getUrl() - Method in class org.apache.nutch.protocol.interactiveselenium.HttpResponse
getUrl() - Method in class org.apache.nutch.protocol.okhttp.OkHttpResponse
getUrl() - Method in exception org.apache.nutch.protocol.ProtocolNotFound
getUrl() - Method in class org.apache.nutch.protocol.selenium.HttpResponse
getUrl() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
getUrl() - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
getUrl() - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
getURL2() - Method in class org.apache.nutch.fetcher.FetchItem
getUrlMD5(String) - Static method in class org.apache.nutch.util.DumpFileUtil
getUseHttp11() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getUserAgent() - Method in class org.apache.nutch.protocol.http.api.HttpBase
getUUID(Configuration) - Static method in class org.apache.nutch.util.NutchConfiguration
  Retrieve a Nutch UUID of this configuration object, or null if the configuration was created elsewhere.
getValues() - Method in class org.apache.nutch.indexer.NutchField
getValues(String) - Method in class org.apache.nutch.metadata.Metadata
  Get the values associated to a metadata name.
getValues(String) - Method in class org.apache.nutch.metadata.SpellCheckedMetadata
getVersion() - Method in class org.apache.nutch.parse.ParseData
getVersion() - Method in class org.apache.nutch.parse.ParseStatus
getVersion() - Method in class org.apache.nutch.plugin.PluginDescriptor
getWaitForExit() - Method in class org.apache.nutch.util.CommandRunner
getWARCInfoContent(Configuration) - Static method in class org.apache.nutch.tools.WARCUtils
getWarcSize() - Method in class org.apache.nutch.tools.CommonCrawlConfig
getWeight() - Method in class org.apache.nutch.indexer.NutchDocument
getWeight() - Method in class org.apache.nutch.indexer.NutchField
getWhiteList() - Method in class org.apache.nutch.collection.Subcollection
  Returns whitelist
getWhiteListString() - Method in class org.apache.nutch.collection.Subcollection
  Returns whitelist String
getWriter() - Method in class org.apache.nutch.parse.html.DOMBuilder
  Return null since there is no Writer for this class.
gone - Variable in class org.apache.nutch.hostdb.HostDatum
GONE - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Resource is gone.
GROUP_CLEANING - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for cleaning job operations.
GROUP_COMMONCRAWL_DUMPER - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for Common Crawl data dumper tool.
GROUP_CRAWLDB - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for CrawlDb operations.
GROUP_CRAWLDB_FILTER - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for CrawlDb filter operations.
GROUP_DEDUP - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for deduplication operations.
GROUP_DOMAIN_STATS - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for domain statistics operations.
GROUP_FETCHER - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for fetcher operations.
GROUP_FETCHER_OUTLINKS - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for fetcher outlink processing.
GROUP_GENERATOR - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for generator operations.
GROUP_HOSTDB - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for HostDb operations.
GROUP_INDEXER - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for indexer operations.
GROUP_INJECTOR - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for injector operations.
GROUP_PARSER - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for parser operations.
GROUP_SITEMAP - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for sitemap processing operations.
GROUP_WARC_EXPORTER - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for WARC export operations.
GROUP_WEBGRAPH - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Counter group for WebGraph operations.
guessEncoding(Content, String) - Method in class org.apache.nutch.util.EncodingDetector
  Guess the encoding with the previously specified list of clues.
GZIPUtils - Class in org.apache.nutch.util
  A collection of utility methods for working on GZIPed data.
GZIPUtils() - Constructor for class org.apache.nutch.util.GZIPUtils

## H

HadoopFSUtil - Class in org.apache.nutch.util
HadoopFSUtil() - Constructor for class org.apache.nutch.util.HadoopFSUtil
hasDbStatus(CrawlDatum) - Static method in class org.apache.nutch.crawl.CrawlDatum
  Validate DB Status (ref.: CrawlDbReducer, IndexerReducer)
hasFetchStatus(CrawlDatum) - Static method in class org.apache.nutch.crawl.CrawlDatum
  Validate Fetch Status (ref.: CrawlDbReducer, IndexerReducer, SegmentMergerReducer)
hashCode() - Method in class org.apache.nutch.crawl.CrawlDatum
hashCode() - Method in class org.apache.nutch.crawl.Inlink
hashCode() - Method in class org.apache.nutch.parse.Outlink
hashCode() - Method in class org.apache.nutch.plugin.PluginClassLoader
hashCode() - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
HashComparator() - Constructor for class org.apache.nutch.crawl.Generator.HashComparator
hasHomepageUrl() - Method in class org.apache.nutch.hostdb.HostDatum
hasHostDomainRules - Variable in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
  Whether there are host- or domain-specific rules.
hasMetaData() - Method in class org.apache.nutch.hostdb.HostDatum
hasNext() - Method in class org.apache.nutch.util.NodeWalker
hasObject(String) - Method in class org.apache.nutch.util.ObjectCache
HeadingsParseFilter - Class in org.apache.nutch.parse.headings
  HtmlParseFilter to retrieve h1 and h2 values from the DOM.
HeadingsParseFilter() - Constructor for class org.apache.nutch.parse.headings.HeadingsParseFilter
homepageUrl - Variable in class org.apache.nutch.hostdb.HostDatum
host - Variable in class org.apache.nutch.hostdb.ResolverThread
host - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
HOST - Static variable in interface org.apache.nutch.indexwriter.kafka.KafkaConstants
hostDatum - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
HostDatum - Class in org.apache.nutch.hostdb
HostDatum() - Constructor for class org.apache.nutch.hostdb.HostDatum
HostDatum(float) - Constructor for class org.apache.nutch.hostdb.HostDatum
HostDatum(float, Date) - Constructor for class org.apache.nutch.hostdb.HostDatum
HostDatum(float, Date, String) - Constructor for class org.apache.nutch.hostdb.HostDatum
HOSTDB_CHECK_FAILED - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_CHECK_KNOWN - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_CHECK_NEW - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_CHECKED_HOSTS_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Hosts checked.
HOSTDB_CRAWLDATUM_PROCESSORS - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_DUMP_HEADER - Static variable in class org.apache.nutch.hostdb.ReadHostDb
HOSTDB_DUMP_HOMEPAGES - Static variable in class org.apache.nutch.hostdb.ReadHostDb
HOSTDB_DUMP_HOSTNAMES - Static variable in class org.apache.nutch.hostdb.ReadHostDb
HOSTDB_EXISTING_KNOWN_HOST_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Existing known hosts.
HOSTDB_EXISTING_UNKNOWN_HOST_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Existing unknown hosts.
HOSTDB_FILTER_EXPRESSION - Static variable in class org.apache.nutch.hostdb.ReadHostDb
HOSTDB_FILTERED_RECORDS_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Records filtered in HostDb.
HOSTDB_FORCE_CHECK - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_NEW_KNOWN_HOST_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  New known hosts discovered.
HOSTDB_NEW_UNKNOWN_HOST_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  New unknown hosts.
HOSTDB_NUM_RESOLVER_THREADS - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_NUMERIC_FIELDS - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_PERCENTILES - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_PURGE_FAILED_HOSTS_THRESHOLD - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_PURGED_UNKNOWN_HOST_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Purged unknown hosts.
HOSTDB_RECHECK_INTERVAL - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_REDISCOVERED_HOST_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Rediscovered hosts.
HOSTDB_SKIPPED_NOT_ELIGIBLE_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Hosts skipped (not eligible).
HOSTDB_STRING_FIELDS - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_TOTAL_HOSTS_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Total hosts processed.
HOSTDB_URL_FILTERING - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_URL_LIMIT - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTDB_URL_LIMIT_NOT_REACHED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Hosts where URL limit was not reached.
HOSTDB_URL_NORMALIZING - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
HOSTNAME - Static variable in class org.apache.nutch.tools.WARCUtils
hostOrDomain() - Method in class org.apache.nutch.urlfilter.api.RegexRule
  Return if this rule is used for filtering-in or out.
hostProtocolMapping - Variable in class org.apache.nutch.protocol.ProtocolFactory
HOSTS - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
HOSTS - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
hostText - Variable in class org.apache.nutch.hostdb.ResolverThread
HostURLNormalizer - Class in org.apache.nutch.net.urlnormalizer.host
  URL normalizer for mapping hosts to their desired form.
HostURLNormalizer() - Constructor for class org.apache.nutch.net.urlnormalizer.host.HostURLNormalizer
HTMLLanguageParser - Class in org.apache.nutch.analysis.lang
HTMLLanguageParser() - Constructor for class org.apache.nutch.analysis.lang.HTMLLanguageParser
HTMLMetaProcessor - Class in org.apache.nutch.parse.html
  Class for parsing META Directives from DOM trees.
HTMLMetaProcessor - Class in org.apache.nutch.parse.tika
  Class for parsing META Directives from DOM trees.
HTMLMetaProcessor() - Constructor for class org.apache.nutch.parse.html.HTMLMetaProcessor
HTMLMetaProcessor() - Constructor for class org.apache.nutch.parse.tika.HTMLMetaProcessor
HTMLMetaTags - Class in org.apache.nutch.parse
  This class holds the information about HTML "meta" tags extracted from a page.
HTMLMetaTags() - Constructor for class org.apache.nutch.parse.HTMLMetaTags
HtmlParseFilter - Interface in org.apache.nutch.parse
  Extension point for DOM-based HTML parsers.
HTMLPARSEFILTER_ORDER - Static variable in class org.apache.nutch.parse.HtmlParseFilters
HtmlParseFilters - Class in org.apache.nutch.parse
  Creates and caches `HtmlParseFilter` implementing plugins.
HtmlParseFilters(Configuration) - Constructor for class org.apache.nutch.parse.HtmlParseFilters
HtmlParser - Class in org.apache.nutch.parse.html
HtmlParser() - Constructor for class org.apache.nutch.parse.html.HtmlParser
HtmlUnitWebDriver - Class in org.apache.nutch.protocol.htmlunit
HtmlUnitWebDriver() - Constructor for class org.apache.nutch.protocol.htmlunit.HtmlUnitWebDriver
HtmlUnitWebWindowListener - Class in org.apache.nutch.protocol.htmlunit
HtmlUnitWebWindowListener() - Constructor for class org.apache.nutch.protocol.htmlunit.HtmlUnitWebWindowListener
HtmlUnitWebWindowListener(int) - Constructor for class org.apache.nutch.protocol.htmlunit.HtmlUnitWebWindowListener
Http - Class in org.apache.nutch.protocol.htmlunit
Http - Class in org.apache.nutch.protocol.http
Http - Class in org.apache.nutch.protocol.httpclient
  This class is a protocol plugin that configures an HTTP client for Basic, Digest and NTLM authentication schemes for web server as well as proxy server.
Http - Class in org.apache.nutch.protocol.interactiveselenium
Http - Class in org.apache.nutch.protocol.selenium
Http() - Constructor for class org.apache.nutch.protocol.htmlunit.Http
  Default constructor.
Http() - Constructor for class org.apache.nutch.protocol.http.Http
  Public default constructor.
Http() - Constructor for class org.apache.nutch.protocol.httpclient.Http
  Constructs this plugin.
Http() - Constructor for class org.apache.nutch.protocol.interactiveselenium.Http
Http() - Constructor for class org.apache.nutch.protocol.selenium.Http
HTTP - Enum constant in enum class org.apache.nutch.protocol.htmlunit.HttpResponse.Scheme
HTTP - Enum constant in enum class org.apache.nutch.protocol.http.HttpResponse.Scheme
HTTP - Enum constant in enum class org.apache.nutch.protocol.interactiveselenium.HttpResponse.Scheme
HTTP - Enum constant in enum class org.apache.nutch.protocol.selenium.HttpResponse.Scheme
HTTP_HEADER_FROM - Static variable in class org.apache.nutch.tools.WARCUtils
HTTP_HEADER_USER_AGENT - Static variable in class org.apache.nutch.tools.WARCUtils
HTTP_LOG_SUPPRESSION - Static variable in class org.apache.nutch.net.protocols.ProtocolLogUtil
HttpAuthentication - Interface in org.apache.nutch.protocol.httpclient
  The base level of services required for Http Authentication
HttpAuthenticationException - Exception in org.apache.nutch.protocol.httpclient
  Can be used to identify problems during creation of Authentication objects.
HttpAuthenticationException() - Constructor for exception org.apache.nutch.protocol.httpclient.HttpAuthenticationException
  Constructs a new exception with null as its detail message.
HttpAuthenticationException(String) - Constructor for exception org.apache.nutch.protocol.httpclient.HttpAuthenticationException
  Constructs a new exception with the specified detail message.
HttpAuthenticationException(String, Throwable) - Constructor for exception org.apache.nutch.protocol.httpclient.HttpAuthenticationException
  Constructs a new exception with the specified message and cause.
HttpAuthenticationException(Throwable) - Constructor for exception org.apache.nutch.protocol.httpclient.HttpAuthenticationException
  Constructs a new exception with the specified cause and detail message from given clause if it is not null.
HttpAuthenticationFactory - Class in org.apache.nutch.protocol.httpclient
  Provides the Http protocol implementation with the ability to authenticate when prompted.
HttpAuthenticationFactory(Configuration) - Constructor for class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
HttpBase - Class in org.apache.nutch.protocol.http.api
HttpBase() - Constructor for class org.apache.nutch.protocol.http.api.HttpBase
  Creates a new instance of HttpBase
HttpBase(Logger) - Constructor for class org.apache.nutch.protocol.http.api.HttpBase
  Creates a new instance of HttpBase
HttpBasicAuthentication - Class in org.apache.nutch.protocol.httpclient
  Implementation of RFC 2617 Basic Authentication.
HttpBasicAuthentication(String, Configuration) - Constructor for class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
  Construct an HttpBasicAuthentication for the given challenge parameters.
HttpDateFormat - Class in org.apache.nutch.net.protocols
  Parse and format HTTP dates in HTTP headers, e.g., used to fill the "If-Modified-Since" request header field.
HttpDateFormat() - Constructor for class org.apache.nutch.net.protocols.HttpDateFormat
HttpException - Exception in org.apache.nutch.protocol.http.api
HttpException() - Constructor for exception org.apache.nutch.protocol.http.api.HttpException
HttpException(String) - Constructor for exception org.apache.nutch.protocol.http.api.HttpException
HttpException(String, Throwable) - Constructor for exception org.apache.nutch.protocol.http.api.HttpException
HttpException(Throwable) - Constructor for exception org.apache.nutch.protocol.http.api.HttpException
HttpFormAuthConfigurer - Class in org.apache.nutch.protocol.httpclient
HttpFormAuthConfigurer() - Constructor for class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
HttpFormAuthentication - Class in org.apache.nutch.protocol.httpclient
HttpFormAuthentication(String, String, Map<String, String>, Map<String, String>, Set<String>) - Constructor for class org.apache.nutch.protocol.httpclient.HttpFormAuthentication
HttpFormAuthentication(HttpFormAuthConfigurer, HttpClient, Http) - Constructor for class org.apache.nutch.protocol.httpclient.HttpFormAuthentication
HttpHeaders - Interface in org.apache.nutch.metadata
  A collection of HTTP header names.
HttpResponse - Class in org.apache.nutch.protocol.htmlunit
  An HTTP response.
HttpResponse - Class in org.apache.nutch.protocol.http
  An HTTP response.
HttpResponse - Class in org.apache.nutch.protocol.httpclient
  An HTTP response.
HttpResponse - Class in org.apache.nutch.protocol.interactiveselenium
HttpResponse - Class in org.apache.nutch.protocol.selenium
HttpResponse(HttpBase, URL, CrawlDatum) - Constructor for class org.apache.nutch.protocol.htmlunit.HttpResponse
  Default public constructor.
HttpResponse(HttpBase, URL, CrawlDatum) - Constructor for class org.apache.nutch.protocol.http.HttpResponse
  Default public constructor.
HttpResponse(Http, URL, CrawlDatum) - Constructor for class org.apache.nutch.protocol.interactiveselenium.HttpResponse
HttpResponse(Http, URL, CrawlDatum) - Constructor for class org.apache.nutch.protocol.selenium.HttpResponse
HttpResponse.Scheme - Enum Class in org.apache.nutch.protocol.htmlunit
HttpResponse.Scheme - Enum Class in org.apache.nutch.protocol.http
HttpResponse.Scheme - Enum Class in org.apache.nutch.protocol.interactiveselenium
HttpResponse.Scheme - Enum Class in org.apache.nutch.protocol.selenium
HttpRobotRulesParser - Class in org.apache.nutch.protocol.http.api
  This class is used for parsing robots for urls belonging to HTTP protocol.
HttpRobotRulesParser(Configuration) - Constructor for class org.apache.nutch.protocol.http.api.HttpRobotRulesParser
HTTPS - Enum constant in enum class org.apache.nutch.protocol.htmlunit.HttpResponse.Scheme
HTTPS - Enum constant in enum class org.apache.nutch.protocol.http.HttpResponse.Scheme
HTTPS - Enum constant in enum class org.apache.nutch.protocol.interactiveselenium.HttpResponse.Scheme
HTTPS - Enum constant in enum class org.apache.nutch.protocol.selenium.HttpResponse.Scheme
HttpWebClient - Class in org.apache.nutch.protocol.selenium
HttpWebClient() - Constructor for class org.apache.nutch.protocol.selenium.HttpWebClient

## I

IDENTIFIER - Static variable in interface org.apache.nutch.metadata.DublinCore
  Recommended best practice is to identify the resource by means of a string or number conforming to a formal identification system.
IF_MODIFIED_SINCE - Static variable in interface org.apache.nutch.metadata.HttpHeaders
ignorableWhitespace(char[], int, int) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive notification of ignorable whitespace in element content.
IGNORE_EXTERNAL_LINKS - Static variable in class org.apache.nutch.crawl.LinkDb
IGNORE_INTERNAL_LINKS - Static variable in class org.apache.nutch.crawl.LinkDb
in - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
INC_RATE - Variable in class org.apache.nutch.crawl.AdaptiveFetchSchedule
incConnectionFailures() - Method in class org.apache.nutch.hostdb.HostDatum
incDnsFailures() - Method in class org.apache.nutch.hostdb.HostDatum
incrementCounters(Throwable) - Method in class org.apache.nutch.metrics.ErrorTracker
  Directly increments cached error counters without local accumulation.
incrementCounters(ErrorTracker.ErrorType) - Method in class org.apache.nutch.metrics.ErrorTracker
  Directly increments cached error counters without local accumulation.
incrementExceptionCounter() - Method in class org.apache.nutch.fetcher.FetchItemQueue
index(Path, Path, List<Path>, boolean) - Method in class org.apache.nutch.indexer.IndexingJob
index(Path, Path, List<Path>, boolean, boolean) - Method in class org.apache.nutch.indexer.IndexingJob
index(Path, Path, List<Path>, boolean, boolean, String) - Method in class org.apache.nutch.indexer.IndexingJob
index(Path, Path, List<Path>, boolean, boolean, String, boolean, boolean) - Method in class org.apache.nutch.indexer.IndexingJob
index(Path, Path, List<Path>, boolean, boolean, String, boolean, boolean, boolean) - Method in class org.apache.nutch.indexer.IndexingJob
index(Path, Path, List<Path>, boolean, boolean, String, boolean, boolean, boolean, boolean) - Method in class org.apache.nutch.indexer.IndexingJob
INDEX - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
INDEX - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
INDEXER_BINARY_AS_BASE64 - Static variable in class org.apache.nutch.indexer.IndexerMapReduce
INDEXER_DELETE - Static variable in class org.apache.nutch.indexer.IndexerMapReduce
INDEXER_DELETE_ROBOTS_NOINDEX - Static variable in class org.apache.nutch.indexer.IndexerMapReduce
INDEXER_DELETE_SKIPPED - Static variable in class org.apache.nutch.indexer.IndexerMapReduce
INDEXER_DELETED_BY_INDEXING_FILTER_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents deleted by indexing filter.
INDEXER_DELETED_DUPLICATES_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents deleted as duplicates.
INDEXER_DELETED_FAILED_PARSE_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents deleted because parsing failed.
INDEXER_DELETED_GONE_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents deleted because they are gone.
INDEXER_DELETED_REDIRECTS_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents deleted due to redirects.
INDEXER_DELETED_ROBOTS_NOINDEX_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents deleted due to robots noindex.
INDEXER_INDEXED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents indexed (added or updated).
INDEXER_LATENCY - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Prefix for indexer latency metrics.
INDEXER_NO_COMMIT - Static variable in class org.apache.nutch.indexer.IndexerMapReduce
INDEXER_PARAMS - Static variable in class org.apache.nutch.indexer.IndexerMapReduce
INDEXER_SKIP_NOTMODIFIED - Static variable in class org.apache.nutch.indexer.IndexerMapReduce
INDEXER_SKIPPED_BY_INDEXING_FILTER_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents skipped by indexing filter.
INDEXER_SKIPPED_NOT_MODIFIED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Documents skipped (not modified).
IndexerMapper() - Constructor for class org.apache.nutch.indexer.IndexerMapReduce.IndexerMapper
IndexerMapReduce - Class in org.apache.nutch.indexer
  This class is typically invoked from within `IndexingJob` and handles all MapReduce functionality required when undertaking indexing.
IndexerMapReduce() - Constructor for class org.apache.nutch.indexer.IndexerMapReduce
IndexerMapReduce.IndexerMapper - Class in org.apache.nutch.indexer
IndexerMapReduce.IndexerReducer - Class in org.apache.nutch.indexer
IndexerOutputFormat - Class in org.apache.nutch.indexer
IndexerOutputFormat() - Constructor for class org.apache.nutch.indexer.IndexerOutputFormat
IndexerReducer() - Constructor for class org.apache.nutch.indexer.IndexerMapReduce.IndexerReducer
indexerScore(Text, NutchDocument, CrawlDatum, CrawlDatum, Parse, Inlinks, float) - Method in class org.apache.nutch.scoring.AbstractScoringFilter
indexerScore(Text, NutchDocument, CrawlDatum, CrawlDatum, Parse, Inlinks, float) - Method in class org.apache.nutch.scoring.depth.DepthScoringFilter
indexerScore(Text, NutchDocument, CrawlDatum, CrawlDatum, Parse, Inlinks, float) - Method in class org.apache.nutch.scoring.link.LinkAnalysisScoringFilter
indexerScore(Text, NutchDocument, CrawlDatum, CrawlDatum, Parse, Inlinks, float) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
  Dampen the boost value by scorePower.
indexerScore(Text, NutchDocument, CrawlDatum, CrawlDatum, Parse, Inlinks, float) - Method in interface org.apache.nutch.scoring.ScoringFilter
  This method calculates a indexed document score/boost.
indexerScore(Text, NutchDocument, CrawlDatum, CrawlDatum, Parse, Inlinks, float) - Method in class org.apache.nutch.scoring.ScoringFilters
INDEXING - Enum constant in enum class org.apache.nutch.metrics.ErrorTracker.ErrorType
  Indexing filter errors
IndexingException - Exception in org.apache.nutch.indexer
IndexingException() - Constructor for exception org.apache.nutch.indexer.IndexingException
IndexingException(String) - Constructor for exception org.apache.nutch.indexer.IndexingException
IndexingException(String, Throwable) - Constructor for exception org.apache.nutch.indexer.IndexingException
IndexingException(Throwable) - Constructor for exception org.apache.nutch.indexer.IndexingException
IndexingFilter - Interface in org.apache.nutch.indexer
  Extension point for indexing.
INDEXINGFILTER_ORDER - Static variable in class org.apache.nutch.indexer.IndexingFilters
IndexingFilters - Class in org.apache.nutch.indexer
  Creates and caches `IndexingFilter` implementing plugins.
IndexingFilters(Configuration) - Constructor for class org.apache.nutch.indexer.IndexingFilters
IndexingFiltersChecker - Class in org.apache.nutch.indexer
  Reads and parses a URL and run the indexers on it.
IndexingFiltersChecker() - Constructor for class org.apache.nutch.indexer.IndexingFiltersChecker
IndexingJob - Class in org.apache.nutch.indexer
  Generic indexer which relies on the plugins implementing IndexWriter
IndexingJob() - Constructor for class org.apache.nutch.indexer.IndexingJob
IndexingJob(Configuration) - Constructor for class org.apache.nutch.indexer.IndexingJob
IndexWriter - Interface in org.apache.nutch.indexer
IndexWriterConfig - Class in org.apache.nutch.indexer
IndexWriterParams - Class in org.apache.nutch.indexer
IndexWriterParams(Map<? extends String, ? extends String>) - Constructor for class org.apache.nutch.indexer.IndexWriterParams
  Fill IndexWriterParams from map.
indexWriters(NutchDocument) - Method in class org.apache.nutch.exchange.Exchanges
  Returns all the indexers where the document must be sent to.
IndexWriters - Class in org.apache.nutch.indexer
  Creates and caches `IndexWriter` implementing plugins.
IndexWriters.IndexWriterWrapper - Class in org.apache.nutch.indexer
IndexWriterWrapper() - Constructor for class org.apache.nutch.indexer.IndexWriters.IndexWriterWrapper
inflate(byte[]) - Static method in class org.apache.nutch.util.DeflateUtils
  Returns an inflated copy of the input array.
inflateBestEffort(byte[]) - Static method in class org.apache.nutch.util.DeflateUtils
  Returns an inflated copy of the input array.
inflateBestEffort(byte[], int) - Static method in class org.apache.nutch.util.DeflateUtils
  Returns an inflated copy of the input array, truncated to `sizeLimit` bytes, if necessary.
init() - Method in class org.apache.nutch.collection.CollectionManager
init(Path) - Method in class org.apache.nutch.crawl.LinkDbReader
initCounters(TaskInputOutputContext<?, ?, ?, ?>) - Method in class org.apache.nutch.metrics.ErrorTracker
  Initializes cached counter references from the Hadoop context.
initialize(InputSplit, TaskAttemptContext) - Method in class org.apache.nutch.tools.arc.ArcRecordReader
initialize(Element) - Method in class org.apache.nutch.collection.Subcollection
  Initialize Subcollection from dom element
initializeSchedule(Text, CrawlDatum) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
  Initialize fetch schedule related data.
initializeSchedule(Text, CrawlDatum) - Method in interface org.apache.nutch.crawl.FetchSchedule
  Initialize fetch schedule related data.
initialScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.AbstractScoringFilter
initialScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.depth.DepthScoringFilter
initialScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.link.LinkAnalysisScoringFilter
initialScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
  Set to 0.0f (unknown value) - inlink contributions will bring it to a correct level.
initialScore(Text, CrawlDatum) - Method in interface org.apache.nutch.scoring.ScoringFilter
  Set an initial score for newly discovered pages.
initialScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.ScoringFilters
  Calculate a new initial score, used when adding newly discovered pages.
initMRJob(Path, Path, Collection<Path>, Job, boolean) - Static method in class org.apache.nutch.indexer.IndexerMapReduce
inject(Path, Path) - Method in class org.apache.nutch.crawl.Injector
inject(Path, Path, boolean, boolean) - Method in class org.apache.nutch.crawl.Injector
inject(Path, Path, boolean, boolean, boolean, boolean, boolean) - Method in class org.apache.nutch.crawl.Injector
injectedScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.AbstractScoringFilter
injectedScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.depth.DepthScoringFilter
injectedScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
injectedScore(Text, CrawlDatum) - Method in interface org.apache.nutch.scoring.ScoringFilter
  Set an initial score for newly injected pages.
injectedScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.ScoringFilters
  Calculate a new initial score, used when injecting new pages.
InjectMapper() - Constructor for class org.apache.nutch.crawl.Injector.InjectMapper
Injector - Class in org.apache.nutch.crawl
  Injector takes a flat text file of URLs (or a folder containing text files) and merges ("injects") these URLs into the CrawlDb.
Injector() - Constructor for class org.apache.nutch.crawl.Injector
Injector(Configuration) - Constructor for class org.apache.nutch.crawl.Injector
INJECTOR_URLS_FILTERED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs filtered during injection.
INJECTOR_URLS_INJECTED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs injected.
INJECTOR_URLS_INJECTED_UNIQUE_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Unique URLs injected.
INJECTOR_URLS_MERGED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs merged with existing CrawlDb entries.
INJECTOR_URLS_PURGED_404_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs purged due to 404 status.
INJECTOR_URLS_PURGED_FILTER_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  URLs purged by filter.
Injector.InjectMapper - Class in org.apache.nutch.crawl
  InjectMapper reads the CrawlDb seeds are injected into the plain-text seed files and parses each line into the URL and metadata.
Injector.InjectReducer - Class in org.apache.nutch.crawl
  Combine multiple new entries for a url.
InjectReducer() - Constructor for class org.apache.nutch.crawl.Injector.InjectReducer
Inlink - Class in org.apache.nutch.crawl
  An incoming link to a page.
Inlink() - Constructor for class org.apache.nutch.crawl.Inlink
Inlink(String, String) - Constructor for class org.apache.nutch.crawl.Inlink
INLINK - Static variable in class org.apache.nutch.scoring.webgraph.LinkDatum
INLINK_DIR - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph
inLinks - Variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
Inlinks - Class in org.apache.nutch.crawl
  A list of `Inlink`s.
Inlinks() - Constructor for class org.apache.nutch.crawl.Inlinks
InputCompatMapper() - Constructor for class org.apache.nutch.segment.SegmentReader.InputCompatMapper
InputCompatReducer() - Constructor for class org.apache.nutch.segment.SegmentReader.InputCompatReducer
InputFormat() - Constructor for class org.apache.nutch.fetcher.Fetcher.InputFormat
install(Job, Path) - Static method in class org.apache.nutch.crawl.CrawlDb
install(Job, Path) - Static method in class org.apache.nutch.crawl.LinkDb
InteractiveSeleniumHandler - Interface in org.apache.nutch.protocol.interactiveselenium.handlers
INTERNAL - Enum constant in enum class org.apache.nutch.net.protocols.Response.TruncatedContentReason
  implementation internal reason
invert(Path, Path[], boolean, boolean, boolean) - Method in class org.apache.nutch.crawl.LinkDb
invert(Path, Path, boolean, boolean, boolean) - Method in class org.apache.nutch.crawl.LinkDb
Inverter() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter
InvertMapper() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter.InvertMapper
InvertReducer() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter.InvertReducer
IP - Static variable in class org.apache.nutch.tools.WARCUtils
IP_ADDRESS - Static variable in interface org.apache.nutch.net.protocols.Response
  Key to hold the IP address the request is sent to if `store.ip.address` is true
IPFilterRules - Class in org.apache.nutch.protocol.okhttp
  Optionally limit or block connections to IP address ranges (localhost/loopback or site-local addresses, subnet ranges given in CIDR notation, or single IP addresses).
IPFilterRules(Configuration) - Constructor for class org.apache.nutch.protocol.okhttp.IPFilterRules
isAllowListed(URL) - Method in class org.apache.nutch.protocol.RobotRulesParser
  Check whether a URL belongs to a allowlisted host.
isAnySuccess() - Method in class org.apache.nutch.parse.ParseResult
  A convenience method which returns true if at least one of the parses is successful.
isAscii(String) - Static method in class org.apache.nutch.util.URLUtil
isCanonical() - Method in interface org.apache.nutch.parse.Parse
  Indicates if the parse is coming from a url or a sub-url
isCanonical() - Method in class org.apache.nutch.parse.ParseImpl
isClientTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.htmlunit.DummyX509TrustManager
isClientTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.http.DummyX509TrustManager
isClientTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
isClientTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.interactiveselenium.DummyX509TrustManager
isClientTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.selenium.DummyX509TrustManager
isCompressed() - Method in class org.apache.nutch.tools.CommonCrawlConfig
isCookieEnabled() - Method in class org.apache.nutch.protocol.http.api.HttpBase
isEligibleForCheck(HostDatum) - Method in class org.apache.nutch.hostdb.UpdateHostDbReducer
  Determines whether a record is eligible for recheck.
isEmpty() - Method in class org.apache.nutch.hostdb.HostDatum
isEmpty() - Method in class org.apache.nutch.parse.ParseResult
  Checks whether the result is empty.
isEmpty() - Method in class org.apache.nutch.protocol.okhttp.IPFilterRules
isEmpty(String) - Static method in class org.apache.nutch.util.StringUtil
  Checks if a string is empty (ie is null or empty).
isExempted(String, String) - Method in class org.apache.nutch.net.URLExemptionFilters
  Run all defined filters.
isFailed() - Method in class org.apache.nutch.parse.ParseStatus
isHalted() - Method in class org.apache.nutch.fetcher.FetcherThread
isHomePageOf(URL, String) - Static method in class org.apache.nutch.util.URLUtil
  Test whether a URL is the home page or root page of a host.
isIfModifiedSinceEnabled() - Method in class org.apache.nutch.protocol.http.api.HttpBase
isIgnoreCase() - Method in class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
isIndexable(Path, FileSystem) - Static method in class org.apache.nutch.segment.SegmentChecker
  Check if the segment is indexable.
isLoginRedirect() - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
isMagic(byte[]) - Static method in class org.apache.nutch.tools.arc.ArcRecordReader
  Returns true if the byte array passed matches the gzip header magic number.
isModeAccept() - Method in class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
isModelCreated - Static variable in class org.apache.nutch.scoring.similarity.cosine.Model
isMultiValued(String) - Method in class org.apache.nutch.metadata.Metadata
  Returns true if named value is multivalued.
ISP - Enum constant in enum class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter.DatabaseType
isParsed(Path, FileSystem) - Static method in class org.apache.nutch.segment.SegmentChecker
  Check the segment to see if it is has been parsed before.
isParsing(Configuration) - Static method in class org.apache.nutch.fetcher.Fetcher
isPermanentFailure() - Method in class org.apache.nutch.protocol.ProtocolStatus
isProxyException(String) - Method in class org.apache.nutch.protocol.http.api.HttpBase
isRedirect() - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthentication
isRedirect() - Method in class org.apache.nutch.protocol.ProtocolStatus
isRemoteVerificationEnabled() - Method in class org.apache.nutch.protocol.ftp.Client
  Return whether or not verification of the remote host participating in data connections is enabled.
isSameDomainName(String, String) - Static method in class org.apache.nutch.util.URLUtil
  Returns whether the given URLs have the same domain name.
isSameDomainName(URL, URL) - Static method in class org.apache.nutch.util.URLUtil
  Returns whether the given URLs have the same domain name.
isServerTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.htmlunit.DummyX509TrustManager
isServerTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.http.DummyX509TrustManager
isServerTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
isServerTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.interactiveselenium.DummyX509TrustManager
isServerTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.selenium.DummyX509TrustManager
isStoreHttpHeaders() - Method in class org.apache.nutch.protocol.http.api.HttpBase
isStoreHttpRequest() - Method in class org.apache.nutch.protocol.http.api.HttpBase
isStoreIPAddress() - Method in class org.apache.nutch.protocol.http.api.HttpBase
isStorePartialAsTruncated() - Method in class org.apache.nutch.protocol.http.api.HttpBase
  Whether to save partial fetches as truncated content, cf.
isStoringContent(Configuration) - Static method in class org.apache.nutch.fetcher.Fetcher
isSuccess() - Method in class org.apache.nutch.parse.ParseResult
  A convenience method which returns true only if all parses are successful.
isSuccess() - Method in class org.apache.nutch.parse.ParseStatus
isSuccess() - Method in class org.apache.nutch.protocol.ProtocolStatus
isTlsCheckCertificates() - Method in class org.apache.nutch.protocol.http.api.HttpBase
isTransientFailure() - Method in class org.apache.nutch.protocol.ProtocolStatus
isTruncated(Content) - Static method in class org.apache.nutch.parse.ParseSegment
  Checks if the page's content is truncated.
isValid() - Method in class org.apache.nutch.indexer.replace.FieldReplacer
  Does this FieldReplacer have a valid fieldname and pattern?
isWhiteSpace(char) - Static method in class org.apache.nutch.parse.html.XMLCharacterRecognizer
  Returns whether the specified ch conforms to the XML 1.0 definition of whitespace.
isWhiteSpace(char[], int, int) - Static method in class org.apache.nutch.parse.html.XMLCharacterRecognizer
  Tell if the string is whitespace.
isWhiteSpace(String) - Static method in class org.apache.nutch.parse.html.XMLCharacterRecognizer
  Tell if the string is whitespace.
isWhiteSpace(StringBuffer) - Static method in class org.apache.nutch.parse.html.XMLCharacterRecognizer
  Tell if the string is whitespace.
iterator() - Method in class org.apache.nutch.crawl.Inlinks
iterator() - Method in class org.apache.nutch.indexer.NutchDocument
  Iterate over all fields.
iterator() - Method in class org.apache.nutch.parse.ParseResult
  Iterate over all entries in the <url, Parse> map.

## J

JexlExchange - Class in org.apache.nutch.exchange.jexl
JexlExchange() - Constructor for class org.apache.nutch.exchange.jexl.JexlExchange
JexlIndexingFilter - Class in org.apache.nutch.indexer.jexl
  An `IndexingFilter` that allows filtering of documents based on a JEXL expression.
JexlIndexingFilter() - Constructor for class org.apache.nutch.indexer.jexl.JexlIndexingFilter
JexlUtil - Class in org.apache.nutch.util
  Utility methods for handling JEXL expressions used in crawl and index pipelines.
jsonArray - Variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
JsonIndenter() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.JsonIndenter
JSParseFilter - Class in org.apache.nutch.parse.js
  This class is a heuristic link extractor for JavaScript files and code snippets.
JSParseFilter() - Constructor for class org.apache.nutch.parse.js.JSParseFilter

## K

KafkaConstants - Interface in org.apache.nutch.indexwriter.kafka
KafkaIndexWriter - Class in org.apache.nutch.indexwriter.kafka
  Sends Nutch documents to a configured Kafka Cluster
KafkaIndexWriter() - Constructor for class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
keepClientCnxOpen - Variable in class org.apache.nutch.util.AbstractChecker
KEY_SERIALIZER - Static variable in interface org.apache.nutch.indexwriter.kafka.KafkaConstants
KEY_STORE_PASSWORD - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
KEY_STORE_PATH - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
KEY_STORE_TYPE - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
keyPrefix - Variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
killJob() - Method in class org.apache.nutch.util.NutchTool
  Kill the job immediately.

## L

LANGUAGE - Static variable in interface org.apache.nutch.metadata.DublinCore
  A language of the intellectual content of the resource.
LanguageIndexingFilter - Class in org.apache.nutch.analysis.lang
  An `IndexingFilter` that add a `lang` (language) field to the document.
LanguageIndexingFilter() - Constructor for class org.apache.nutch.analysis.lang.LanguageIndexingFilter
  Constructs a new Language Indexing Filter.
LAST_MODIFIED - Static variable in interface org.apache.nutch.metadata.HttpHeaders
lastCheck - Variable in class org.apache.nutch.hostdb.HostDatum
LATENCY_KEY - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Special key used in map output to send serialized TDigest bytes to the reducer for job-level percentile merge.
LatencyTracker - Class in org.apache.nutch.metrics
  A utility class for tracking latency metrics using TDigest for percentile calculation.
LatencyTracker(String, String) - Constructor for class org.apache.nutch.metrics.LatencyTracker
  Creates a new LatencyTracker.
leftPad(String, int) - Static method in class org.apache.nutch.util.StringUtil
  Returns a copy of `s` (left padded) with leading spaces so that it's length is `length`.
LENGTH - Enum constant in enum class org.apache.nutch.net.protocols.Response.TruncatedContentReason
  fetch exceeded configured http.content.limit
LICENSE_LOCATION - Static variable in interface org.apache.nutch.metadata.CreativeCommons
LICENSE_URL - Static variable in interface org.apache.nutch.metadata.CreativeCommons
LineRecordWriter(DataOutputStream) - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumCsvOutputFormat.LineRecordWriter
LineRecordWriter(DataOutputStream) - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumJsonOutputFormat.LineRecordWriter
LinkAnalysisScoringFilter - Class in org.apache.nutch.scoring.link
LinkAnalysisScoringFilter() - Constructor for class org.apache.nutch.scoring.link.LinkAnalysisScoringFilter
LinkDatum - Class in org.apache.nutch.scoring.webgraph
  A class for holding link information including the url, anchor text, a score, the timestamp of the link and a link type.
LinkDatum() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDatum
  Default constructor, no url, timestamp, score, or link type.
LinkDatum(String) - Constructor for class org.apache.nutch.scoring.webgraph.LinkDatum
  Creates a LinkDatum with a given url.
LinkDatum(String, String) - Constructor for class org.apache.nutch.scoring.webgraph.LinkDatum
  Creates a LinkDatum with a url and an anchor text.
LinkDatum(String, String, long) - Constructor for class org.apache.nutch.scoring.webgraph.LinkDatum
LinkDb - Class in org.apache.nutch.crawl
  Maintains an inverted link map, listing incoming links for each url.
LinkDb() - Constructor for class org.apache.nutch.crawl.LinkDb
LinkDb(Configuration) - Constructor for class org.apache.nutch.crawl.LinkDb
LinkDb.LinkDbMapper - Class in org.apache.nutch.crawl
LinkDBDumpMapper() - Constructor for class org.apache.nutch.crawl.LinkDbReader.LinkDBDumpMapper
LinkDbFilter - Class in org.apache.nutch.crawl
  This class provides a way to separate the URL normalization and filtering steps from the rest of LinkDb manipulation code.
LinkDbFilter() - Constructor for class org.apache.nutch.crawl.LinkDbFilter
LinkDbMapper() - Constructor for class org.apache.nutch.crawl.LinkDb.LinkDbMapper
LinkDbMerger - Class in org.apache.nutch.crawl
  This tool merges several LinkDb-s into one, optionally filtering URLs through the current URLFilters, to skip prohibited URLs and links.
LinkDbMerger() - Constructor for class org.apache.nutch.crawl.LinkDbMerger
LinkDbMerger(Configuration) - Constructor for class org.apache.nutch.crawl.LinkDbMerger
LinkDbMerger.LinkDbMergeReducer - Class in org.apache.nutch.crawl
LinkDbMergeReducer() - Constructor for class org.apache.nutch.crawl.LinkDbMerger.LinkDbMergeReducer
LinkDbReader - Class in org.apache.nutch.crawl
  Read utility for the LinkDb.
LinkDbReader() - Constructor for class org.apache.nutch.crawl.LinkDbReader
LinkDbReader(Configuration, Path) - Constructor for class org.apache.nutch.crawl.LinkDbReader
LinkDbReader.LinkDBDumpMapper - Class in org.apache.nutch.crawl
LinkDumper - Class in org.apache.nutch.scoring.webgraph
  The LinkDumper tool creates a database of node to inlink information that can be read using the nested Reader class.
LinkDumper() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper
LinkDumper.Inverter - Class in org.apache.nutch.scoring.webgraph
  Inverts outlinks from the WebGraph to inlinks and attaches node information.
LinkDumper.Inverter.InvertMapper - Class in org.apache.nutch.scoring.webgraph
  Wraps all values in ObjectWritables.
LinkDumper.Inverter.InvertReducer - Class in org.apache.nutch.scoring.webgraph
  Inverts outlinks to inlinks while attaching node information to the outlink.
LinkDumper.LinkNode - Class in org.apache.nutch.scoring.webgraph
  Bean class which holds url to node information.
LinkDumper.LinkNodes - Class in org.apache.nutch.scoring.webgraph
  Writable class which holds an array of LinkNode objects.
LinkDumper.Merger - Class in org.apache.nutch.scoring.webgraph
  Merges LinkNode objects into a single array value per url.
LinkDumper.Reader - Class in org.apache.nutch.scoring.webgraph
  Reader class which will print out the url and all of its inlinks to system out.
LinkNode() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
LinkNode(String, Node) - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
LinkNodes() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
LinkNodes(LinkDumper.LinkNode[]) - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
LinkParams(String, String, int) - Constructor for class org.apache.nutch.parse.html.DOMContentUtils.LinkParams
LinkRank - Class in org.apache.nutch.scoring.webgraph
LinkRank() - Constructor for class org.apache.nutch.scoring.webgraph.LinkRank
  Default constructor.
LinkRank(Configuration) - Constructor for class org.apache.nutch.scoring.webgraph.LinkRank
  Configurable constructor.
LINKS_INLINKS_HOST - Static variable in class org.apache.nutch.indexer.links.LinksIndexingFilter
LINKS_ONLY_HOSTS - Static variable in class org.apache.nutch.indexer.links.LinksIndexingFilter
LINKS_OUTLINKS_HOST - Static variable in class org.apache.nutch.indexer.links.LinksIndexingFilter
LinksIndexingFilter - Class in org.apache.nutch.indexer.links
  An `IndexingFilter` that adds `outlinks` and `inlinks` field(s) to the document.
LinksIndexingFilter() - Constructor for class org.apache.nutch.indexer.links.LinksIndexingFilter
list(List<Path>, Writer) - Method in class org.apache.nutch.segment.SegmentReader
loadClass(String, boolean) - Method in class org.apache.nutch.plugin.PluginClassLoader
LOCATION - Static variable in interface org.apache.nutch.metadata.HttpHeaders
lock(Configuration, Path, boolean) - Static method in class org.apache.nutch.crawl.CrawlDb
LOCK_NAME - Static variable in class org.apache.nutch.crawl.CrawlDb
LOCK_NAME - Static variable in class org.apache.nutch.crawl.LinkDb
LOCK_NAME - Static variable in class org.apache.nutch.hostdb.UpdateHostDb
LOCK_NAME - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph
LOCK_NAME - Static variable in class org.apache.nutch.util.SitemapProcessor
LockUtil - Class in org.apache.nutch.util
  Utility methods for handling application-level locking.
LockUtil() - Constructor for class org.apache.nutch.util.LockUtil
LOG - Static variable in class org.apache.nutch.crawl.Generator
LOG - Static variable in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
LOG - Static variable in class org.apache.nutch.plugin.PluginManifestParser
LOG - Static variable in class org.apache.nutch.plugin.PluginRepository
LOG - Static variable in class org.apache.nutch.plugin.URLStreamHandlerFactory
LOG - Static variable in class org.apache.nutch.protocol.file.File
LOG - Static variable in class org.apache.nutch.protocol.ftp.Ftp
LOG - Static variable in class org.apache.nutch.protocol.htmlunit.Http
LOG - Static variable in class org.apache.nutch.protocol.http.Http
LOG - Static variable in class org.apache.nutch.protocol.httpclient.Http
LOG - Static variable in class org.apache.nutch.protocol.interactiveselenium.Http
LOG - Static variable in class org.apache.nutch.protocol.okhttp.IPFilterRules
LOG - Static variable in class org.apache.nutch.protocol.okhttp.OkHttp
LOG - Static variable in class org.apache.nutch.protocol.okhttp.OkHttpResponse
LOG - Static variable in class org.apache.nutch.protocol.selenium.Http
LOG - Static variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
LOG - Static variable in class org.apache.nutch.urlfilter.fast.FastURLFilter
logConf() - Method in class org.apache.nutch.protocol.http.api.HttpBase
logDateFormat - Static variable in class org.apache.nutch.util.TimingUtil
  Formats dates for logging
logDateMillis(long) - Static method in class org.apache.nutch.util.TimingUtil
  Convert epoch milliseconds (`System.currentTimeMillis()`) into date string (local time zone) used for logging
login() - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthentication
login(String, String) - Method in class org.apache.nutch.protocol.ftp.Client
  Login to the FTP server using the provided username and password.
logout() - Method in class org.apache.nutch.protocol.ftp.Client
  Logout of the FTP server by sending the QUIT command.
logShort(Throwable) - Method in class org.apache.nutch.net.protocols.ProtocolLogUtil
  Return true if exception is configured to be logged as short message without stack trace, usually done for frequent exceptions with obvious reasons (e.g., UnknownHostException), configurable by `http.log.exceptions.suppress.stack`
longestMatch(String) - Method in class org.apache.nutch.util.PrefixStringMatcher
  Returns the longest prefix of `input` that is matched, or `null` if no match exists.
longestMatch(String) - Method in class org.apache.nutch.util.SuffixStringMatcher
  Returns the longest suffix of `input` that is matched, or `null` if no match exists.
longestMatch(String) - Method in class org.apache.nutch.util.TrieStringMatcher
  Returns the longest substring of `input` that is matched by a pattern in the trie, or `null` if no match exists.
LuceneAnalyzerUtil - Class in org.apache.nutch.scoring.similarity.util
  Creates a custom analyzer based on user provided inputs
LuceneAnalyzerUtil(LuceneAnalyzerUtil.StemFilterType, boolean) - Constructor for class org.apache.nutch.scoring.similarity.util.LuceneAnalyzerUtil
  Creates an analyzer instance based on Lucene default stopword set if the param useStopFilter is set to true
LuceneAnalyzerUtil(LuceneAnalyzerUtil.StemFilterType, List<String>, boolean) - Constructor for class org.apache.nutch.scoring.similarity.util.LuceneAnalyzerUtil
  Creates an analyzer instance based on user provided stop words.
LuceneAnalyzerUtil.StemFilterType - Enum Class in org.apache.nutch.scoring.similarity.util
LuceneTokenizer - Class in org.apache.nutch.scoring.similarity.util
LuceneTokenizer(String, LuceneTokenizer.TokenizerType, boolean, LuceneAnalyzerUtil.StemFilterType) - Constructor for class org.apache.nutch.scoring.similarity.util.LuceneTokenizer
  Creates a tokenizer based on param values
LuceneTokenizer(String, LuceneTokenizer.TokenizerType, List<String>, boolean, LuceneAnalyzerUtil.StemFilterType) - Constructor for class org.apache.nutch.scoring.similarity.util.LuceneTokenizer
  Creates a tokenizer based on param values
LuceneTokenizer(String, LuceneTokenizer.TokenizerType, LuceneAnalyzerUtil.StemFilterType, int, int) - Constructor for class org.apache.nutch.scoring.similarity.util.LuceneTokenizer
  Creates a tokenizer for the ngram model based on param values
LuceneTokenizer.TokenizerType - Enum Class in org.apache.nutch.scoring.similarity.util

## M

m_currentNode - Variable in class org.apache.nutch.parse.html.DOMBuilder
  Current node
m_doc - Variable in class org.apache.nutch.parse.html.DOMBuilder
  Root document
m_docFrag - Variable in class org.apache.nutch.parse.html.DOMBuilder
  First node of document fragment or null if not a DocumentFragment
m_elemStack - Variable in class org.apache.nutch.parse.html.DOMBuilder
  Vector of element nodes
m_inCData - Variable in class org.apache.nutch.parse.html.DOMBuilder
  Flag indicating that we are processing a CData section
main(String[]) - Static method in class org.apache.nutch.crawl.AdaptiveFetchSchedule
main(String[]) - Static method in class org.apache.nutch.crawl.CrawlDb
main(String[]) - Static method in class org.apache.nutch.crawl.CrawlDbMerger
  Run the tool.
main(String[]) - Static method in class org.apache.nutch.crawl.CrawlDbReader
main(String[]) - Static method in class org.apache.nutch.crawl.DeduplicationJob
main(String[]) - Static method in class org.apache.nutch.crawl.Generator
  Generate a fetchlist from the crawldb.
main(String[]) - Static method in class org.apache.nutch.crawl.Injector
main(String[]) - Static method in class org.apache.nutch.crawl.LinkDb
main(String[]) - Static method in class org.apache.nutch.crawl.LinkDbMerger
  Run the job
main(String[]) - Static method in class org.apache.nutch.crawl.LinkDbReader
main(String[]) - Static method in class org.apache.nutch.crawl.MimeAdaptiveFetchSchedule
main(String[]) - Static method in class org.apache.nutch.crawl.TextProfileSignature
main(String[]) - Static method in class org.apache.nutch.fetcher.Fetcher
  Run the fetcher.
main(String[]) - Static method in class org.apache.nutch.hostdb.ReadHostDb
main(String[]) - Static method in class org.apache.nutch.hostdb.UpdateHostDb
main(String[]) - Static method in class org.apache.nutch.indexer.CleaningJob
main(String[]) - Static method in class org.apache.nutch.indexer.filter.MimeTypeIndexingFilter
  Main method for invoking this tool
main(String[]) - Static method in class org.apache.nutch.indexer.IndexingFiltersChecker
main(String[]) - Static method in class org.apache.nutch.indexer.IndexingJob
main(String[]) - Static method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
main(String[]) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
main(String[]) - Static method in class org.apache.nutch.net.URLFilterChecker
main(String[]) - Static method in class org.apache.nutch.net.urlnormalizer.basic.BasicURLNormalizer
main(String[]) - Static method in class org.apache.nutch.net.urlnormalizer.regex.RegexURLNormalizer
  Spits out patterns and substitutions that are in the configuration file.
main(String[]) - Static method in class org.apache.nutch.net.URLNormalizerChecker
main(String[]) - Static method in class org.apache.nutch.parse.feed.FeedParser
  Runs a command line version of this `Parser`.
main(String[]) - Static method in class org.apache.nutch.parse.html.HtmlParser
main(String[]) - Static method in class org.apache.nutch.parse.js.JSParseFilter
  Main method which can be run from command line with the plugin option.
main(String[]) - Static method in class org.apache.nutch.parse.ParseData
main(String[]) - Static method in class org.apache.nutch.parse.ParserChecker
main(String[]) - Static method in class org.apache.nutch.parse.ParseSegment
main(String[]) - Static method in class org.apache.nutch.parse.ParseText
main(String[]) - Static method in class org.apache.nutch.parse.zip.ZipParser
main(String[]) - Static method in class org.apache.nutch.plugin.PluginRepository
  Loads all necessary dependencies for a selected plugin, and then runs one of the classes' main() method.
main(String[]) - Static method in class org.apache.nutch.protocol.Content
main(String[]) - Static method in class org.apache.nutch.protocol.file.File
  Quick way for running this class.
main(String[]) - Static method in class org.apache.nutch.protocol.ftp.Ftp
  For debugging.
main(String[]) - Static method in class org.apache.nutch.protocol.htmlunit.Http
main(String[]) - Static method in class org.apache.nutch.protocol.http.Http
main(String[]) - Static method in class org.apache.nutch.protocol.httpclient.Http
  Main method.
main(String[]) - Static method in class org.apache.nutch.protocol.interactiveselenium.Http
main(String[]) - Static method in class org.apache.nutch.protocol.okhttp.OkHttp
main(String[]) - Static method in class org.apache.nutch.protocol.RobotRulesParser
main(String[]) - Static method in class org.apache.nutch.protocol.selenium.Http
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.LinkDumper
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.LinkDumper.Reader
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.LinkRank
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.NodeDumper
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.NodeReader
  Runs the NodeReader tool.
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.ScoreUpdater
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.WebGraph
main(String[]) - Static method in class org.apache.nutch.segment.SegmentMerger
main(String[]) - Static method in class org.apache.nutch.segment.SegmentReader
main(String[]) - Static method in class org.apache.nutch.tools.arc.ArcSegmentCreator
main(String[]) - Static method in class org.apache.nutch.tools.CommonCrawlDataDumper
  Main method for invoking this tool
main(String[]) - Static method in class org.apache.nutch.tools.DmozParser
  Command-line access.
main(String[]) - Static method in class org.apache.nutch.tools.FileDumper
  Main method for invoking this tool
main(String[]) - Static method in class org.apache.nutch.tools.FreeGenerator
main(String[]) - Static method in class org.apache.nutch.tools.ResolveUrls
  Runs the resolve urls tool.
main(String[]) - Static method in class org.apache.nutch.tools.ShowProperties
main(String[]) - Static method in class org.apache.nutch.tools.warc.WARCExporter
main(String[]) - Static method in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
main(String[]) - Static method in class org.apache.nutch.urlfilter.ignoreexempt.ExemptionUrlFilter
main(String[]) - Static method in class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
main(String[]) - Static method in class org.apache.nutch.urlfilter.regex.RegexURLFilter
main(String[]) - Static method in class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
main(String[]) - Static method in class org.apache.nutch.util.CommandRunner
main(String[]) - Static method in class org.apache.nutch.util.CrawlCompletionStats
main(String[]) - Static method in class org.apache.nutch.util.DomainStatistics
main(String[]) - Static method in class org.apache.nutch.util.EncodingDetector
main(String[]) - Static method in class org.apache.nutch.util.PrefixStringMatcher
main(String[]) - Static method in class org.apache.nutch.util.ProtocolStatusStatistics
main(String[]) - Static method in class org.apache.nutch.util.SitemapProcessor
main(String[]) - Static method in class org.apache.nutch.util.StringUtil
main(String[]) - Static method in class org.apache.nutch.util.SuffixStringMatcher
main(String[]) - Static method in class org.apache.nutch.util.URLUtil
  For testing
main(HttpBase, String[]) - Static method in class org.apache.nutch.protocol.http.api.HttpBase
main(RegexURLFilterBase, String[]) - Static method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
  Filter the standard input using a RegexURLFilterBase.
majorCodes - Static variable in class org.apache.nutch.parse.ParseStatus
makeClient(IndexWriterParams) - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
  Generates a RestHighLevelClient with the hosts given
makeClient(IndexWriterParams) - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
  Generates a RestHighLevelClient with the hosts given
map(FloatWritable, Generator.SelectorEntry, Mapper.Context) - Method in class org.apache.nutch.crawl.Generator.SelectorInverseMapper
map(Text, BytesWritable, Mapper.Context) - Method in class org.apache.nutch.tools.arc.ArcSegmentCreator.ArcSegmentCreatorMapper
  Runs the Map job to translate an arc record into output for Nutch segments.
map(Text, Writable, Mapper.Context) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter.InvertMapper
map(Text, Writable, Mapper.Context) - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater.ScoreUpdaterMapper
map(Text, Writable, Mapper.Context) - Method in class org.apache.nutch.crawl.Injector.InjectMapper
map(Text, Writable, Mapper.Context) - Method in class org.apache.nutch.hostdb.UpdateHostDbMapper
  Mapper ingesting records from the HostDB, CrawlDB and plaintext host scores file.
map(Text, Writable, Mapper.Context) - Method in class org.apache.nutch.indexer.IndexerMapReduce.IndexerMapper
map(Text, Writable, Mapper.Context) - Method in class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb.OutlinkDbMapper
map(Text, Writable, Mapper.Context) - Method in class org.apache.nutch.tools.warc.WARCExporter.WARCMapReduce.WARCMapper
map(Text, CrawlDatum, Mapper.Context) - Method in class org.apache.nutch.crawl.DeduplicationJob.DBFilter
map(Text, CrawlDatum, Mapper.Context) - Method in class org.apache.nutch.indexer.CleaningJob.DBFilter
map(Text, CrawlDatum, Mapper.Context) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNMapper
map(Text, CrawlDatum, Mapper.Context) - Method in class org.apache.nutch.crawl.Generator.SelectorMapper
map(Text, CrawlDatum, Mapper.Context) - Method in class org.apache.nutch.crawl.CrawlDbFilter
map(Text, CrawlDatum, Mapper.Context) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbDumpMapper
map(Text, CrawlDatum, Mapper.Context) - Method in class org.apache.nutch.crawl.Generator.CrawlDbUpdater.CrawlDbUpdateMapper
map(Text, CrawlDatum, Mapper.Context) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatMapper
map(Text, Inlinks, Mapper.Context) - Method in class org.apache.nutch.crawl.LinkDbFilter
map(Text, Inlinks, Mapper.Context) - Method in class org.apache.nutch.crawl.LinkDbReader.LinkDBDumpMapper
map(Text, MetaWrapper, Mapper.Context) - Method in class org.apache.nutch.segment.SegmentMerger.SegmentMergerMapper
map(Text, ParseData, Mapper.Context) - Method in class org.apache.nutch.crawl.LinkDb.LinkDbMapper
map(Text, Node, Mapper.Context) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter.SorterMapper
map(Text, Node, Mapper.Context) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Dumper.DumperMapper
map(WritableComparable<?>, Text, Mapper.Context) - Method in class org.apache.nutch.tools.FreeGenerator.FG.FGMapper
map(WritableComparable<?>, Writable, Mapper.Context) - Method in class org.apache.nutch.segment.SegmentReader.InputCompatMapper
map(WritableComparable<?>, Content, Mapper.Context) - Method in class org.apache.nutch.parse.ParseSegment.ParseSegmentMapper
mask(String) - Static method in class org.apache.nutch.util.StringUtil
  Mask sensitive strings - passwords, etc.
mask(String, char) - Static method in class org.apache.nutch.util.StringUtil
  Mask sensitive strings - passwords, etc.
mask(String, Pattern, char) - Static method in class org.apache.nutch.util.StringUtil
  Mask sensitive strings - passwords, etc.
match(String) - Method in class org.apache.nutch.urlfilter.api.RegexRule
  Checks if a url matches this rule.
match(URL) - Method in class org.apache.nutch.urlfilter.fast.FastURLFilter.DenyAllRule
match(URL) - Method in class org.apache.nutch.urlfilter.fast.FastURLFilter.DenyPathQueryRule
match(URL) - Method in class org.apache.nutch.urlfilter.fast.FastURLFilter.DenyPathRule
match(URL) - Method in class org.apache.nutch.urlfilter.fast.FastURLFilter.Rule
match(NutchDocument) - Method in interface org.apache.nutch.exchange.Exchange
  Determines if the document must go to the related index writers.
match(NutchDocument) - Method in class org.apache.nutch.exchange.jexl.JexlExchange
  Determines if the document must go to the related index writers.
matchChar(TrieStringMatcher.TrieNode, String, int) - Method in class org.apache.nutch.util.TrieStringMatcher
  Get the next `TrieStringMatcher.TrieNode` visited, given that you are at `node`, and that the next character in the input is the `idx`'th character of `s`.
matches(String) - Method in class org.apache.nutch.util.PrefixStringMatcher
  Returns true if the given `String` is matched by a prefix in the trie
matches(String) - Method in class org.apache.nutch.util.SuffixStringMatcher
  Returns true if the given `String` is matched by a suffix in the trie
matches(String) - Method in class org.apache.nutch.util.TrieStringMatcher
  Returns true if the given `String` is matched by a pattern in the trie
MAX_BULK_DOCS - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
MAX_BULK_DOCS - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
MAX_BULK_LENGTH - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
MAX_BULK_LENGTH - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
MAX_DEPTH_KEY - Static variable in class org.apache.nutch.scoring.depth.DepthScoringFilter
MAX_DEPTH_KEY_W - Static variable in class org.apache.nutch.scoring.depth.DepthScoringFilter
MAX_DOC_COUNT - Static variable in interface org.apache.nutch.indexwriter.kafka.KafkaConstants
MAX_DOCS_BATCH - Static variable in interface org.apache.nutch.indexwriter.cloudsearch.CloudSearchConstants
MAX_WARC_FILE_SIZE - Static variable in class org.apache.nutch.tools.CommonCrawlFormatWARC
maxContent - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The length limit for downloaded content, in bytes.
maxCrawlDelay - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Skip page if Crawl-Delay longer than this value.
maxDuration - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The time limit to download the entire content, in seconds.
maxInterval - Variable in class org.apache.nutch.crawl.AbstractFetchSchedule
maxNumRedirects - Variable in class org.apache.nutch.protocol.RobotRulesParser
MD5Signature - Class in org.apache.nutch.crawl
  Default implementation of a page signature.
MD5Signature() - Constructor for class org.apache.nutch.crawl.MD5Signature
merge(Path, Path[], boolean, boolean) - Method in class org.apache.nutch.crawl.CrawlDbMerger
merge(Path, Path[], boolean, boolean) - Method in class org.apache.nutch.crawl.LinkDbMerger
merge(Path, Path[], boolean, boolean, long) - Method in class org.apache.nutch.segment.SegmentMerger
merge(LatencyTracker) - Method in class org.apache.nutch.metrics.LatencyTracker
  Merges another LatencyTracker's digest and aggregates count/sum into this one.
Merger() - Constructor for class org.apache.nutch.crawl.CrawlDbMerger.Merger
Merger() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.Merger
metadata - Variable in class org.apache.nutch.indexer.IndexingFiltersChecker
metadata - Variable in class org.apache.nutch.metadata.Metadata
  A map of all metadata attributes.
metadata - Variable in class org.apache.nutch.parse.ParserChecker
metadata - Variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
metaData - Variable in class org.apache.nutch.hostdb.HostDatum
Metadata - Class in org.apache.nutch.metadata
  A multi-valued metadata container.
Metadata() - Constructor for class org.apache.nutch.metadata.Metadata
  Constructs a new, empty metadata.
METADATA_CONTENT - Static variable in class org.apache.nutch.scoring.metadata.MetadataScoringFilter
METADATA_DATUM - Static variable in class org.apache.nutch.scoring.metadata.MetadataScoringFilter
METADATA_PARSED - Static variable in class org.apache.nutch.scoring.metadata.MetadataScoringFilter
MetadataIndexer - Class in org.apache.nutch.indexer.metadata
  Indexer which can be configured to extract metadata from the crawldb, parse metadata or content metadata.
MetadataIndexer() - Constructor for class org.apache.nutch.indexer.metadata.MetadataIndexer
MetadataScoringFilter - Class in org.apache.nutch.scoring.metadata
  For documentation: `org.apache.nutch.scoring.metadata`
MetadataScoringFilter() - Constructor for class org.apache.nutch.scoring.metadata.MetadataScoringFilter
metadataSource - Static variable in class org.apache.nutch.indexer.subcollection.SubcollectionIndexingFilter
  Metadata source field name
metadataToJson(Metadata) - Method in class org.apache.nutch.tools.warc.WARCExporter.WARCMapReduce.WARCReducer
  Adds keys/values of a Nuta metadata container to a JsonObject.
MetaTagsParser - Class in org.apache.nutch.parse.metatags
  Parse HTML meta tags (keywords, description) and store them in the parse metadata so that they can be indexed with the index-metadata plugin with the prefix 'metatag.'.
MetaTagsParser() - Constructor for class org.apache.nutch.parse.metatags.MetaTagsParser
MetaWrapper - Class in org.apache.nutch.metadata
  This is a simple decorator that adds metadata to any Writable-s that can be serialized by `NutchWritable`.
MetaWrapper() - Constructor for class org.apache.nutch.metadata.MetaWrapper
MetaWrapper(Writable, Configuration) - Constructor for class org.apache.nutch.metadata.MetaWrapper
MetaWrapper(Metadata, Writable, Configuration) - Constructor for class org.apache.nutch.metadata.MetaWrapper
MimeAdaptiveFetchSchedule - Class in org.apache.nutch.crawl
  Extension of @see AdaptiveFetchSchedule that allows for more flexible configuration of DEC and INC factors for various MIME-types.
MimeAdaptiveFetchSchedule() - Constructor for class org.apache.nutch.crawl.MimeAdaptiveFetchSchedule
MIMEFILTER_REGEX_FILE - Static variable in class org.apache.nutch.indexer.filter.MimeTypeIndexingFilter
MimeTypeIndexingFilter - Class in org.apache.nutch.indexer.filter
  An `IndexingFilter` that allows filtering of documents based on the MIME Type detected by Tika
MimeTypeIndexingFilter() - Constructor for class org.apache.nutch.indexer.filter.MimeTypeIndexingFilter
MimeUtil - Class in org.apache.nutch.util
  This is a facade class to insulate Nutch from its underlying Mime Type substrate library, Apache Tika.
MimeUtil(Configuration) - Constructor for class org.apache.nutch.util.MimeUtil
MIN_CONFIDENCE_KEY - Static variable in class org.apache.nutch.util.EncodingDetector
MissingDependencyException - Exception in org.apache.nutch.plugin
  `MissingDependencyException` will be thrown if a plugin dependency cannot be found.
MissingDependencyException(String) - Constructor for exception org.apache.nutch.plugin.MissingDependencyException
MissingDependencyException(Throwable) - Constructor for exception org.apache.nutch.plugin.MissingDependencyException
Model - Class in org.apache.nutch.scoring.similarity.cosine
  This class creates a model used to store Document vector representation of the corpus.
Model() - Constructor for class org.apache.nutch.scoring.similarity.cosine.Model
MODIFIED - Static variable in interface org.apache.nutch.metadata.DublinCore
  Date on which the resource was changed.
modifyWebClient(WebClient) - Method in class org.apache.nutch.protocol.htmlunit.HtmlUnitWebDriver
MoreIndexingFilter - Class in org.apache.nutch.indexer.more
  Add (or reset) a few metaData properties as respective fields (if they are available), so that they can be accurately used within the search index.
MoreIndexingFilter() - Constructor for class org.apache.nutch.indexer.more.MoreIndexingFilter
MOVED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Resource has moved permanently.

## N

NaiveBayesParseFilter - Class in org.apache.nutch.parsefilter.naivebayes
  Html Parse filter that classifies the outlinks from the parseresult as relevant or irrelevant based on the parseText's relevancy (using a training file where you can give positive and negative example texts see the description of parsefilter.naivebayes.trainfile) and if found irrelevant it gives the link a second chance if it contains any of the words from the list given in parsefilter.naivebayes.wordlist.
NaiveBayesParseFilter() - Constructor for class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
names() - Method in class org.apache.nutch.metadata.Metadata
  Returns an array of the names contained in the metadata.
NETWORK - Enum constant in enum class org.apache.nutch.metrics.ErrorTracker.ErrorType
  Network-related errors (IOException, SocketException, etc.)
next(Text, BytesWritable) - Method in class org.apache.nutch.tools.arc.ArcRecordReader
  Returns true if the next record in the split is read into the key and value pair.
nextKeyValue() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
nextNode() - Method in class org.apache.nutch.util.NodeWalker
  Returns the next `Node` on the stack and pushes all of its children onto the stack, allowing us to walk the node tree without the use of recursion.
NO_THRESHOLD - Static variable in class org.apache.nutch.util.EncodingDetector
Node - Class in org.apache.nutch.scoring.webgraph
  A class which holds the number of inlinks and outlinks for a given url along with an inlink score from a link analysis program and any metadata.
Node() - Constructor for class org.apache.nutch.scoring.webgraph.Node
NODE_DIR - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph
nodeChar - Variable in class org.apache.nutch.util.TrieStringMatcher.TrieNode
NodeDumper - Class in org.apache.nutch.scoring.webgraph
  A tools that dumps out the top urls by number of inlinks, number of outlinks, or by score, to a text file.
NodeDumper() - Constructor for class org.apache.nutch.scoring.webgraph.NodeDumper
NodeDumper.Dumper - Class in org.apache.nutch.scoring.webgraph
  Outputs the hosts or domains with an associated value.
NodeDumper.Dumper.DumperMapper - Class in org.apache.nutch.scoring.webgraph
  Outputs the host or domain as key for this record and numInlinks, numOutlinks or score as the value.
NodeDumper.Dumper.DumperReducer - Class in org.apache.nutch.scoring.webgraph
  Outputs either the sum or the top value for this record.
NodeDumper.Sorter - Class in org.apache.nutch.scoring.webgraph
  Outputs the top urls sorted in descending order.
NodeDumper.Sorter.SorterMapper - Class in org.apache.nutch.scoring.webgraph
  Outputs the url with the appropriate number of inlinks, outlinks, or for score.
NodeDumper.Sorter.SorterReducer - Class in org.apache.nutch.scoring.webgraph
  Flips and collects the url and numeric sort value.
NodeReader - Class in org.apache.nutch.scoring.webgraph
  Reads and prints to system out information for a single node from the NodeDb in the WebGraph.
NodeReader() - Constructor for class org.apache.nutch.scoring.webgraph.NodeReader
NodeReader(Configuration) - Constructor for class org.apache.nutch.scoring.webgraph.NodeReader
NodeWalker - Class in org.apache.nutch.util
  A utility class that allows the walking of any DOM tree using a stack instead of recursion.
NodeWalker(Node) - Constructor for class org.apache.nutch.util.NodeWalker
  Starts the `Node` tree from the root node.
NONE - Enum constant in enum class org.apache.nutch.scoring.similarity.util.LuceneAnalyzerUtil.StemFilterType
NORM_HOST_IDN - Static variable in class org.apache.nutch.net.urlnormalizer.basic.BasicURLNormalizer
NORM_HOST_IDNA_2008 - Static variable in class org.apache.nutch.net.urlnormalizer.basic.BasicURLNormalizer
NORM_HOST_TRIM_TRAILING_DOT - Static variable in class org.apache.nutch.net.urlnormalizer.basic.BasicURLNormalizer
normalize - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
normalize(String, String) - Method in class org.apache.nutch.net.urlnormalizer.ajax.AjaxURLNormalizer
  Attempts to normalize the input URL string
normalize(String, String) - Method in class org.apache.nutch.net.urlnormalizer.basic.BasicURLNormalizer
normalize(String, String) - Method in class org.apache.nutch.net.urlnormalizer.host.HostURLNormalizer
normalize(String, String) - Method in interface org.apache.nutch.net.URLNormalizer
normalize(String, String) - Method in class org.apache.nutch.net.urlnormalizer.pass.PassURLNormalizer
normalize(String, String) - Method in class org.apache.nutch.net.urlnormalizer.protocol.ProtocolURLNormalizer
normalize(String, String) - Method in class org.apache.nutch.net.urlnormalizer.querystring.QuerystringURLNormalizer
normalize(String, String) - Method in class org.apache.nutch.net.urlnormalizer.regex.RegexURLNormalizer
normalize(String, String) - Method in class org.apache.nutch.net.urlnormalizer.slash.SlashURLNormalizer
normalize(String, String) - Method in class org.apache.nutch.net.URLNormalizers
  Normalize
normalizeEscapedFragment(String) - Method in class org.apache.nutch.net.urlnormalizer.ajax.AjaxURLNormalizer
  Returns a normalized input URL.
normalizeHashedFragment(String) - Method in class org.apache.nutch.net.urlnormalizer.ajax.AjaxURLNormalizer
  Returns a normalized input URL.
normalizers - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
normalizers - Variable in class org.apache.nutch.indexer.IndexingFiltersChecker
normalizers - Variable in class org.apache.nutch.parse.ParserChecker
NOT_TRUNCATED - Enum constant in enum class org.apache.nutch.net.protocols.Response.TruncatedContentReason
NOTFETCHING - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Not fetching.
NOTFOUND - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Resource was not found.
notModified - Variable in class org.apache.nutch.hostdb.HostDatum
NOTMODIFIED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Unchanged since the last fetch.
NOTPARSED - Static variable in class org.apache.nutch.parse.ParseStatus
  Parsing was not performed.
now - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
numericFields - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
numericFieldWritables - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
numFailures() - Method in class org.apache.nutch.hostdb.HostDatum
numJobs - Variable in class org.apache.nutch.util.NutchTool
numOverDue - Variable in class org.apache.nutch.hostdb.FetchOverdueCrawlDatumProcessor
numRecords() - Method in class org.apache.nutch.hostdb.HostDatum
numResolverThreads - Variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
Nutch - Interface in org.apache.nutch.metadata
  A collection of Nutch internal metadata constants.
NutchConfiguration - Class in org.apache.nutch.util
  Utility to create Hadoop `Configuration`s that include Nutch-specific resources.
NutchDocument - Class in org.apache.nutch.indexer
  A `NutchDocument` is the unit of indexing.
NutchDocument() - Constructor for class org.apache.nutch.indexer.NutchDocument
nutchFetchIntervalMDName - Static variable in class org.apache.nutch.crawl.Injector
  metadata key reserved for setting a custom fetchInterval for a specific URL
NutchField - Class in org.apache.nutch.indexer
  This class represents a multi-valued field with a weight.
NutchField() - Constructor for class org.apache.nutch.indexer.NutchField
NutchField(Object) - Constructor for class org.apache.nutch.indexer.NutchField
NutchField(Object, float) - Constructor for class org.apache.nutch.indexer.NutchField
nutchFixedFetchIntervalMDName - Static variable in class org.apache.nutch.crawl.Injector
  metadata key reserved for setting a fixed custom fetchInterval for a specific URL
NutchIndexAction - Class in org.apache.nutch.indexer
  A `NutchIndexAction` is the new unit of indexing holding the document and action information.
NutchIndexAction() - Constructor for class org.apache.nutch.indexer.NutchIndexAction
NutchIndexAction(NutchDocument, byte) - Constructor for class org.apache.nutch.indexer.NutchIndexAction
NutchJob - Class in org.apache.nutch.util
  A `Job` for Nutch jobs.
NutchJob(Configuration, String) - Constructor for class org.apache.nutch.util.NutchJob
  Deprecated.
  , use instead `Job.getInstance(Configuration)` or `Job.getInstance(Configuration, String)`.
NutchMetrics - Class in org.apache.nutch.metrics
  Centralized constants for Hadoop metrics counter groups and names.
NutchPublisher - Interface in org.apache.nutch.publisher
  All publisher subscriber model implementations should implement this interface.
NutchPublishers - Class in org.apache.nutch.publisher
NutchPublishers(Configuration) - Constructor for class org.apache.nutch.publisher.NutchPublishers
nutchScoreMDName - Static variable in class org.apache.nutch.crawl.Injector
  metadata key reserved for setting a custom score for a specific URL
NutchTool - Class in org.apache.nutch.util
NutchTool() - Constructor for class org.apache.nutch.util.NutchTool
NutchTool(Configuration) - Constructor for class org.apache.nutch.util.NutchTool
NutchWritable - Class in org.apache.nutch.crawl
NutchWritable() - Constructor for class org.apache.nutch.crawl.NutchWritable
NutchWritable(Writable) - Constructor for class org.apache.nutch.crawl.NutchWritable

## O

ObjectCache - Class in org.apache.nutch.util
ObjectInputFormat() - Constructor for class org.apache.nutch.segment.SegmentMerger.ObjectInputFormat
OkHttp - Class in org.apache.nutch.protocol.okhttp
OkHttp() - Constructor for class org.apache.nutch.protocol.okhttp.OkHttp
OkHttpResponse - Class in org.apache.nutch.protocol.okhttp
OkHttpResponse(OkHttp, URL, CrawlDatum) - Constructor for class org.apache.nutch.protocol.okhttp.OkHttpResponse
OkHttpResponse.TruncatedContent - Class in org.apache.nutch.protocol.okhttp
  Container to store whether and why content has been truncated
OLD_OUTLINK_DIR - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph
open() - Method in class org.apache.nutch.exchange.Exchanges
  Opens each configured exchange.
open(Map<String, String>) - Method in interface org.apache.nutch.exchange.Exchange
  Initializes the internal variables.
open(Map<String, String>) - Method in class org.apache.nutch.exchange.jexl.JexlExchange
  Initializes the internal variables.
open(Configuration, String) - Method in interface org.apache.nutch.indexer.IndexWriter
  Deprecated.
  use `IndexWriter.open(IndexWriterParams)`} instead.
open(Configuration, String) - Method in class org.apache.nutch.indexer.IndexWriters
  Initializes the internal variables of index writers.
open(Configuration, String) - Method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
open(Configuration, String) - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
open(Configuration, String) - Method in class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
open(Configuration, String) - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
open(Configuration, String) - Method in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
open(Configuration, String) - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
open(Configuration, String) - Method in class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
open(Configuration, String) - Method in class org.apache.nutch.indexwriter.solr.SolrIndexWriter
open(IndexWriterParams) - Method in interface org.apache.nutch.indexer.IndexWriter
  Initializes the internal variables from a given index writer configuration.
open(IndexWriterParams) - Method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
open(IndexWriterParams) - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
  Initializes the internal variables from a given index writer configuration.
open(IndexWriterParams) - Method in class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
  Initializes the internal variables from a given index writer configuration.
open(IndexWriterParams) - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
  Initializes the internal variables from a given index writer configuration.
open(IndexWriterParams) - Method in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
open(IndexWriterParams) - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
  Initializes the internal variables from a given index writer configuration.
open(IndexWriterParams) - Method in class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
  Initializes the internal variables from a given index writer configuration.
open(IndexWriterParams) - Method in class org.apache.nutch.indexwriter.solr.SolrIndexWriter
  Initializes the internal variables from a given index writer configuration.
openChannel() - Method in class org.apache.nutch.rabbitmq.RabbitMQClient
  Opens a new channel into the opened connection.
openReaders() - Method in class org.apache.nutch.crawl.LinkDbReader
OpenSearch1xConstants - Interface in org.apache.nutch.indexwriter.opensearch1x
OpenSearch1xIndexWriter - Class in org.apache.nutch.indexwriter.opensearch1x
  Sends NutchDocuments to a configured OpenSearch index.
OpenSearch1xIndexWriter() - Constructor for class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
OPERATOR - Static variable in class org.apache.nutch.tools.WARCUtils
OPICScoringFilter - Class in org.apache.nutch.scoring.opic
  This plugin implements a variant of an Online Page Importance Computation (OPIC) score, described in this paper: Abiteboul, Serge and Preda, Mihai and Cobena, Gregory (2003), Adaptive On-Line Page Importance Computation.
OPICScoringFilter() - Constructor for class org.apache.nutch.scoring.opic.OPICScoringFilter
OPTIONS - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
OPTIONS - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
org.apache.nutch.analysis.lang - package org.apache.nutch.analysis.lang
  Text document language identifier.
org.apache.nutch.collection - package org.apache.nutch.collection
  Subcollection is a subset of an index.
org.apache.nutch.crawl - package org.apache.nutch.crawl
  Crawl control code and tools to run the crawler.
org.apache.nutch.exchange - package org.apache.nutch.exchange
  Control code for exchange component, which acts in indexing job and decides to which index writer a document should be routed, based on plugins behavior.
org.apache.nutch.exchange.jexl - package org.apache.nutch.exchange.jexl
  Plugin of Exchange component based on JEXL expressions.
org.apache.nutch.fetcher - package org.apache.nutch.fetcher
  The Nutch multi-threaded fetching module
org.apache.nutch.hostdb - package org.apache.nutch.hostdb
org.apache.nutch.indexer - package org.apache.nutch.indexer
  Index content, configure and run indexing and cleaning jobs to add, update, and delete documents from an index.
org.apache.nutch.indexer.anchor - package org.apache.nutch.indexer.anchor
  An indexing plugin for inbound anchor text.
org.apache.nutch.indexer.arbitrary - package org.apache.nutch.indexer.arbitrary
  Indexing filter to add document arbitrary data to the index from the output of a user-specified class.
org.apache.nutch.indexer.basic - package org.apache.nutch.indexer.basic
  A basic indexing plugin, adds basic fields: url, host, title, content, etc.
org.apache.nutch.indexer.feed - package org.apache.nutch.indexer.feed
  Indexing filter to index meta data from RSS feeds.
org.apache.nutch.indexer.filter - package org.apache.nutch.indexer.filter
org.apache.nutch.indexer.geoip - package org.apache.nutch.indexer.geoip
  This plugin implements an indexing filter which takes advantage of the GeoIP2-java API.
org.apache.nutch.indexer.jexl - package org.apache.nutch.indexer.jexl
  This plugin implements a dynamic indexing filter which uses JEXL expressions to allow filtering based on the page's metadata
org.apache.nutch.indexer.links - package org.apache.nutch.indexer.links
org.apache.nutch.indexer.metadata - package org.apache.nutch.indexer.metadata
  Indexing filter to add document metadata to the index.
org.apache.nutch.indexer.more - package org.apache.nutch.indexer.more
  A more indexing plugin, adds "more" index fields:last modified date, MIME type, content length.
org.apache.nutch.indexer.replace - package org.apache.nutch.indexer.replace
  Indexing filter to allow pattern replacements on metadata.
org.apache.nutch.indexer.staticfield - package org.apache.nutch.indexer.staticfield
  A simple plugin called at indexing that adds fields with static data.
org.apache.nutch.indexer.subcollection - package org.apache.nutch.indexer.subcollection
  Indexing filter to assign documents to subcollections.
org.apache.nutch.indexer.tld - package org.apache.nutch.indexer.tld
  Top Level Domain Indexing plugin.
org.apache.nutch.indexer.urlmeta - package org.apache.nutch.indexer.urlmeta
  URL Meta Tag Indexing Plugin
org.apache.nutch.indexwriter.cloudsearch - package org.apache.nutch.indexwriter.cloudsearch
org.apache.nutch.indexwriter.csv - package org.apache.nutch.indexwriter.csv
  Index writer plugin to write a plain CSV file.
org.apache.nutch.indexwriter.dummy - package org.apache.nutch.indexwriter.dummy
  Index writer plugin for debugging, writes pairs of <action, url> to a text file, action is one of "add", "update", or "delete".
org.apache.nutch.indexwriter.elastic - package org.apache.nutch.indexwriter.elastic
  Index writer plugin for Elasticsearch.
org.apache.nutch.indexwriter.kafka - package org.apache.nutch.indexwriter.kafka
  Index writer plugin to produce JSON messages to Kafka.
org.apache.nutch.indexwriter.opensearch1x - package org.apache.nutch.indexwriter.opensearch1x
  Index writer plugin for OpenSearch.
org.apache.nutch.indexwriter.rabbit - package org.apache.nutch.indexwriter.rabbit
org.apache.nutch.indexwriter.solr - package org.apache.nutch.indexwriter.solr
  Index writer plugin for Apache Solr.
org.apache.nutch.metadata - package org.apache.nutch.metadata
  A Multi-valued Metadata container, and set of constant fields for Nutch Metadata.
org.apache.nutch.metrics - package org.apache.nutch.metrics
  Metrics infrastructure for Apache Nutch.
org.apache.nutch.microformats.reltag - package org.apache.nutch.microformats.reltag
  A microformats Rel-Tag Parser/Indexer/Querier plugin.
org.apache.nutch.net - package org.apache.nutch.net
  Web-related interfaces: URL `filters` and `normalizers`.
org.apache.nutch.net.protocols - package org.apache.nutch.net.protocols
  Helper classes related to the `Protocol` interface, see also `org.apache.nutch.protocol`.
org.apache.nutch.net.urlnormalizer.ajax - package org.apache.nutch.net.urlnormalizer.ajax
org.apache.nutch.net.urlnormalizer.basic - package org.apache.nutch.net.urlnormalizer.basic
  URL normalizer performing basic normalizations: remove default ports, e.g., port 80 for `http://` URLs remove needless slashes and dot segments in the path component remove anchors use percent-encoding (only) where needed E.g., `https://www.example.org/a/../b//./select%2Dlang.php?lang=español#anchor` is normalized to `https://www.example.org/b/select-lang.php?lang=espa%C3%B1ol` Optional and configurable normalizations are: convert Internationalized Domain Names (IDNs) uniquely either to the ASCII (Punycode) or Unicode representation, see property `urlnormalizer.basic.host.idn` remove a trailing dot from host names, see property `urlnormalizer.basic.host.trim-trailing-dot`
org.apache.nutch.net.urlnormalizer.host - package org.apache.nutch.net.urlnormalizer.host
  URL normalizer renaming hosts to a canonical form listed in the configuration file.
org.apache.nutch.net.urlnormalizer.pass - package org.apache.nutch.net.urlnormalizer.pass
  URL normalizer dummy which does not change URLs.
org.apache.nutch.net.urlnormalizer.protocol - package org.apache.nutch.net.urlnormalizer.protocol
  URL normalizer to normalize the protocol for all URLs of a given host or domain.
org.apache.nutch.net.urlnormalizer.querystring - package org.apache.nutch.net.urlnormalizer.querystring
  URL normalizer which sort the elements in the query part to avoid duplicates by permutations.
org.apache.nutch.net.urlnormalizer.regex - package org.apache.nutch.net.urlnormalizer.regex
  URL normalizer with configurable rules based on regular expressions (`Pattern`).
org.apache.nutch.net.urlnormalizer.slash - package org.apache.nutch.net.urlnormalizer.slash
org.apache.nutch.parse - package org.apache.nutch.parse
  The `Parse` interface and related classes.
org.apache.nutch.parse.ext - package org.apache.nutch.parse.ext
  Parse wrapper to run external command to do the parsing.
org.apache.nutch.parse.feed - package org.apache.nutch.parse.feed
  Parse RSS feeds.
org.apache.nutch.parse.headings - package org.apache.nutch.parse.headings
  Parse filter to extract headings (h1, h2, etc.) from DOM parse tree.
org.apache.nutch.parse.html - package org.apache.nutch.parse.html
  An HTML document parsing plugin.
org.apache.nutch.parse.js - package org.apache.nutch.parse.js
  Parser and parse filter plugin to extract all (possible) links from JavaScript files and embedded JavaScript code snippets.
org.apache.nutch.parse.metatags - package org.apache.nutch.parse.metatags
  Parse filter to extract meta tags: keywords, description, etc.
org.apache.nutch.parse.tika - package org.apache.nutch.parse.tika
  Parse various document formats with help of Apache Tika.
org.apache.nutch.parse.zip - package org.apache.nutch.parse.zip
  Parse ZIP files: embedded files are recursively passed to appropriate parsers.
org.apache.nutch.parsefilter.debug - package org.apache.nutch.parsefilter.debug
  Adds serialized DOM to parse data, useful for debugging, to understand how the parser implementation interprets a document (not only HTML).
org.apache.nutch.parsefilter.naivebayes - package org.apache.nutch.parsefilter.naivebayes
  Html Parse filter that classifies the outlinks from the parseresult as relevant or irrelevant based on the parseText's relevancy (using a training file where you can give positive and negative example texts see the description of parsefilter.naivebayes.trainfile) and if found irrelevent it gives the link a second chance if it contains any of the words from the list given in parsefilter.naivebayes.wordlist.
org.apache.nutch.parsefilter.regex - package org.apache.nutch.parsefilter.regex
  RegexParseFilter.
org.apache.nutch.plugin - package org.apache.nutch.plugin
  The Nutch `Plugin` System.
org.apache.nutch.protocol - package org.apache.nutch.protocol
  Classes related to the `Protocol` interface, see also `org.apache.nutch.net.protocols`.
org.apache.nutch.protocol.file - package org.apache.nutch.protocol.file
  Protocol plugin which supports retrieving local file resources.
org.apache.nutch.protocol.ftp - package org.apache.nutch.protocol.ftp
  Protocol plugin which supports retrieving documents via the ftp protocol.
org.apache.nutch.protocol.htmlunit - package org.apache.nutch.protocol.htmlunit
  Protocol plugin which supports retrieving documents via HTTP/HTTPS using Selenium and the HtmlUnitDriver web driver for the for the HtmlUnit headless browser.
org.apache.nutch.protocol.http - package org.apache.nutch.protocol.http
  Protocol plugin which supports retrieving documents via the http protocol.
org.apache.nutch.protocol.http.api - package org.apache.nutch.protocol.http.api
  Common API used by HTTP plugins (`http`, `httpclient`, etc.)
org.apache.nutch.protocol.httpclient - package org.apache.nutch.protocol.httpclient
  Protocol plugin which supports retrieving documents via the HTTP andHTTPS protocols, optionally with Basic, Digest and NTLM authentication schemes for web server as well as proxy server.
org.apache.nutch.protocol.interactiveselenium - package org.apache.nutch.protocol.interactiveselenium
  Protocol plugin which supports retrieving documents using and interacting with Selenium.
org.apache.nutch.protocol.interactiveselenium.handlers - package org.apache.nutch.protocol.interactiveselenium.handlers
  Handler implementations to interact with Selenium for `org.apache.nutch.protocol.interactiveselenium`.
org.apache.nutch.protocol.okhttp - package org.apache.nutch.protocol.okhttp
  Protocol plugin for HTTP/HTTPS based on okhttp, supports HTTP 1.1 and/or http/2.
org.apache.nutch.protocol.selenium - package org.apache.nutch.protocol.selenium
  Protocol plugin which supports retrieving documents via Selenium.
org.apache.nutch.publisher - package org.apache.nutch.publisher
org.apache.nutch.publisher.rabbitmq - package org.apache.nutch.publisher.rabbitmq
  Publisher package to implement queues
org.apache.nutch.rabbitmq - package org.apache.nutch.rabbitmq
org.apache.nutch.scoring - package org.apache.nutch.scoring
  The `ScoringFilter` interface.
org.apache.nutch.scoring.depth - package org.apache.nutch.scoring.depth
  Scoring filter to stop crawling at a configurable depth (number of "hops" from seed URLs).
org.apache.nutch.scoring.link - package org.apache.nutch.scoring.link
  Scoring filter used in conjunction with `WebGraph`.
org.apache.nutch.scoring.metadata - package org.apache.nutch.scoring.metadata
  Metadata Scoring Plugin
org.apache.nutch.scoring.opic - package org.apache.nutch.scoring.opic
  Scoring filter implementing a variant of the Online Page Importance Computation (OPIC) algorithm.
org.apache.nutch.scoring.orphan - package org.apache.nutch.scoring.orphan
  Scoring filter to modify score or status of orphaned pages (no inlinks found for a configurable amount of time).
org.apache.nutch.scoring.similarity - package org.apache.nutch.scoring.similarity
org.apache.nutch.scoring.similarity.cosine - package org.apache.nutch.scoring.similarity.cosine
  Implements the cosine similarity metric for scoring relevant documents
org.apache.nutch.scoring.similarity.util - package org.apache.nutch.scoring.similarity.util
  Utility package for Lucene functions.
org.apache.nutch.scoring.urlmeta - package org.apache.nutch.scoring.urlmeta
  URL Meta Tag Scoring Plugin
org.apache.nutch.scoring.webgraph - package org.apache.nutch.scoring.webgraph
  Scoring implementation based on link analysis (`LinkRank`), see `WebGraph`.
org.apache.nutch.segment - package org.apache.nutch.segment
  A segment stores all data from on generate/fetch/update cycle: fetch list, protocol status, raw content, parsed content, and extracted outgoing links.
org.apache.nutch.tools - package org.apache.nutch.tools
  Miscellaneous tools.
org.apache.nutch.tools.arc - package org.apache.nutch.tools.arc
  Tools to read the Arc file format.
org.apache.nutch.tools.warc - package org.apache.nutch.tools.warc
  Tools to import / export between Nutch segments and WARC archives.
org.apache.nutch.urlfilter.api - package org.apache.nutch.urlfilter.api
  Generic `URL filter` library, abstracting away from regular expression implementations.
org.apache.nutch.urlfilter.automaton - package org.apache.nutch.urlfilter.automaton
  URL filter plugin based on dk.brics.automaton Finite-State Automata for JavaTM.
org.apache.nutch.urlfilter.domain - package org.apache.nutch.urlfilter.domain
  URL filter plugin to include only URLs which match an element in a given list of domain suffixes, domain names, and/or host names.
org.apache.nutch.urlfilter.domaindenylist - package org.apache.nutch.urlfilter.domaindenylist
  URL filter plugin to exclude URLs by domain suffixes, domain names, and/or host names.
org.apache.nutch.urlfilter.fast - package org.apache.nutch.urlfilter.fast
  URL filter plugin that first does fast exact suffix matches on host/domain names before applying regular expressions to the path component of a URL.
org.apache.nutch.urlfilter.ignoreexempt - package org.apache.nutch.urlfilter.ignoreexempt
  URL filter plugin which identifies exemptions to external urls when when external urls are set to ignore.
org.apache.nutch.urlfilter.prefix - package org.apache.nutch.urlfilter.prefix
  URL filter plugin to include only URLs which match one of a given list of URL prefixes.
org.apache.nutch.urlfilter.regex - package org.apache.nutch.urlfilter.regex
  URL filter plugin to include and/or exclude URLs matching Java regular expressions.
org.apache.nutch.urlfilter.suffix - package org.apache.nutch.urlfilter.suffix
  URL filter plugin to either exclude or include only URLs which match one of the given (path) suffixes.
org.apache.nutch.urlfilter.validator - package org.apache.nutch.urlfilter.validator
  URL filter plugin that validates given urls.
org.apache.nutch.util - package org.apache.nutch.util
  Miscellaneous utility classes.
org.creativecommons.nutch - package org.creativecommons.nutch
  Sample plugins that parse and index Creative Commons metadata.
ORIGINAL_CHAR_ENCODING - Static variable in interface org.apache.nutch.metadata.Nutch
ORPHAN_KEY_WRITABLE - Static variable in class org.apache.nutch.scoring.orphan.OrphanScoringFilter
orphanedScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.orphan.OrphanScoringFilter
orphanedScore(Text, CrawlDatum) - Method in interface org.apache.nutch.scoring.ScoringFilter
  This method may change the score or status of CrawlDatum during CrawlDb update, when the URL is neither fetched nor has any inlinks.
orphanedScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.ScoringFilters
  Calculate orphaned page score during CrawlDb.update().
OrphanScoringFilter - Class in org.apache.nutch.scoring.orphan
  Orphan scoring filter that determines whether a page has become orphaned, e.g.
OrphanScoringFilter() - Constructor for class org.apache.nutch.scoring.orphan.OrphanScoringFilter
OTHER - Enum constant in enum class org.apache.nutch.metrics.ErrorTracker.ErrorType
  Other uncategorized errors
Outlink - Class in org.apache.nutch.parse
  An outgoing link from a page.
Outlink() - Constructor for class org.apache.nutch.parse.Outlink
Outlink(String, String) - Constructor for class org.apache.nutch.parse.Outlink
OUTLINK - Static variable in class org.apache.nutch.scoring.webgraph.LinkDatum
OUTLINK_DIR - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph
OutlinkDb() - Constructor for class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb
  Default constructor.
OutlinkDb(Configuration) - Constructor for class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb
  Configurable constructor.
OutlinkDbMapper() - Constructor for class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb.OutlinkDbMapper
OutlinkDbReducer() - Constructor for class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb.OutlinkDbReducer
OutlinkExtractor - Class in org.apache.nutch.parse
  Extractor to extract `Outlink`s / URLs from plain text using Regular Expressions.
OutlinkExtractor() - Constructor for class org.apache.nutch.parse.OutlinkExtractor
overDueTime - Variable in class org.apache.nutch.hostdb.FetchOverdueCrawlDatumProcessor
overDueTimeLimit - Variable in class org.apache.nutch.hostdb.FetchOverdueCrawlDatumProcessor

## P

parse(InputStream) - Method in class org.apache.nutch.collection.CollectionManager
parse(String) - Static method in class org.apache.nutch.segment.SegmentPart
  Create SegmentPart from a String in format "segmentName/partName".
parse(Path) - Method in class org.apache.nutch.parse.ParseSegment
parse(Content) - Method in class org.apache.nutch.parse.ParseUtil
  Performs a parse by iterating through a List of preferred `Parser`s until a successful parse is performed and a `Parse` object is returned.
Parse - Interface in org.apache.nutch.parse
  The result of parsing a page's raw content.
PARSE_DIR_NAME - Static variable in class org.apache.nutch.crawl.CrawlDatum
PARSE_FORMAT - Static variable in class org.apache.nutch.net.protocols.HttpDateFormat
  Use a less restrictive format for parsing: accept single-digit day-of-month and any timezone
parseArgs(String[], int) - Method in class org.apache.nutch.util.AbstractChecker
parseByExtensionId(String, Content) - Method in class org.apache.nutch.parse.ParseUtil
  Method parses a `Content` object using the `Parser` specified by the parameter `extId`, i.e., the Parser's extension ID.
parseCharacterEncoding(String) - Static method in class org.apache.nutch.util.EncodingDetector
  Parse the character encoding from the specified content type header.
parsed - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
ParseData - Class in org.apache.nutch.parse
  Data extracted from a page's content.
ParseData() - Constructor for class org.apache.nutch.parse.ParseData
ParseData(ParseStatus, String, Outlink[], Metadata) - Constructor for class org.apache.nutch.parse.ParseData
ParseData(ParseStatus, String, Outlink[], Metadata, Metadata) - Constructor for class org.apache.nutch.parse.ParseData
parseDmozFile(File, int, boolean, int, Pattern) - Method in class org.apache.nutch.tools.DmozParser
  Iterate through all the items in this structured DMOZ file.
parseErrors - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
ParseException - Exception in org.apache.nutch.parse
ParseException() - Constructor for exception org.apache.nutch.parse.ParseException
ParseException(String) - Constructor for exception org.apache.nutch.parse.ParseException
ParseException(String, Throwable) - Constructor for exception org.apache.nutch.parse.ParseException
ParseException(Throwable) - Constructor for exception org.apache.nutch.parse.ParseException
parseExpression(String) - Static method in class org.apache.nutch.util.JexlUtil
  Parses a JEXL expression using the default (sandboxed) engine.
parseExpression(Configuration, String) - Static method in class org.apache.nutch.util.JexlUtil
  Parses a JEXL expression.
ParseImpl - Class in org.apache.nutch.parse
  The result of parsing a page's raw content.
ParseImpl() - Constructor for class org.apache.nutch.parse.ParseImpl
ParseImpl(String, ParseData) - Constructor for class org.apache.nutch.parse.ParseImpl
ParseImpl(Parse) - Constructor for class org.apache.nutch.parse.ParseImpl
ParseImpl(ParseText, ParseData) - Constructor for class org.apache.nutch.parse.ParseImpl
ParseImpl(ParseText, ParseData, boolean) - Constructor for class org.apache.nutch.parse.ParseImpl
parseList(List<String>, String) - Method in class org.apache.nutch.collection.Subcollection
  Create a list of patterns from a chunk of text, patterns are separated with a newline
ParseOutputFormat - Class in org.apache.nutch.parse
ParseOutputFormat() - Constructor for class org.apache.nutch.parse.ParseOutputFormat
parsePluginFolder(String[]) - Method in class org.apache.nutch.plugin.PluginManifestParser
  Returns a list of all found plugin descriptors.
Parser - Interface in org.apache.nutch.parse
  A parser for content generated by a `Protocol` implementation.
PARSER_LATENCY - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Prefix for parse latency metrics.
ParserChecker - Class in org.apache.nutch.parse
  Parser checker, useful for testing parser.
ParserChecker() - Constructor for class org.apache.nutch.parse.ParserChecker
ParseResult - Class in org.apache.nutch.parse
  A utility class that stores result of a parse.
ParseResult(String) - Constructor for class org.apache.nutch.parse.ParseResult
  Create a container for parse results.
ParserFactory - Class in org.apache.nutch.parse
  Creates and caches `Parser` plugins.
ParserFactory(Configuration) - Constructor for class org.apache.nutch.parse.ParserFactory
ParserNotFound - Exception in org.apache.nutch.parse
ParserNotFound(String) - Constructor for exception org.apache.nutch.parse.ParserNotFound
ParserNotFound(String, String) - Constructor for exception org.apache.nutch.parse.ParserNotFound
ParserNotFound(String, String, String) - Constructor for exception org.apache.nutch.parse.ParserNotFound
parseRules(String, byte[], String, String) - Method in class org.apache.nutch.protocol.RobotRulesParser
  Deprecated.
parseRules(String, byte[], String, Collection<String>) - Method in class org.apache.nutch.protocol.RobotRulesParser
  Parses the robots content using the `SimpleRobotRulesParser` from crawler-commons
ParseSegment - Class in org.apache.nutch.parse
ParseSegment() - Constructor for class org.apache.nutch.parse.ParseSegment
ParseSegment(Configuration) - Constructor for class org.apache.nutch.parse.ParseSegment
ParseSegment.ParseSegmentMapper - Class in org.apache.nutch.parse
ParseSegment.ParseSegmentPartitioner - Class in org.apache.nutch.parse
  Sends LATENCY_KEY to partition 0 so one reducer merges all TDigests.
ParseSegment.ParseSegmentReducer - Class in org.apache.nutch.parse
ParseSegmentMapper() - Constructor for class org.apache.nutch.parse.ParseSegment.ParseSegmentMapper
ParseSegmentPartitioner() - Constructor for class org.apache.nutch.parse.ParseSegment.ParseSegmentPartitioner
ParseSegmentReducer() - Constructor for class org.apache.nutch.parse.ParseSegment.ParseSegmentReducer
ParseStatus - Class in org.apache.nutch.parse
ParseStatus() - Constructor for class org.apache.nutch.parse.ParseStatus
ParseStatus(int) - Constructor for class org.apache.nutch.parse.ParseStatus
ParseStatus(int, int) - Constructor for class org.apache.nutch.parse.ParseStatus
ParseStatus(int, int, String) - Constructor for class org.apache.nutch.parse.ParseStatus
  Simplified constructor for passing just a text message.
ParseStatus(int, int, String[]) - Constructor for class org.apache.nutch.parse.ParseStatus
ParseStatus(int, String) - Constructor for class org.apache.nutch.parse.ParseStatus
  Simplified constructor for passing just a text message.
ParseStatus(int, String[]) - Constructor for class org.apache.nutch.parse.ParseStatus
ParseStatus(Throwable) - Constructor for class org.apache.nutch.parse.ParseStatus
ParseText - Class in org.apache.nutch.parse
ParseText() - Constructor for class org.apache.nutch.parse.ParseText
ParseText(String) - Constructor for class org.apache.nutch.parse.ParseText
ParseUtil - Class in org.apache.nutch.parse
  A Utility class containing methods to simply perform parsing utilities such as iterating through a preferred list of `Parser`s to obtain `Parse` objects.
ParseUtil(Configuration) - Constructor for class org.apache.nutch.parse.ParseUtil
  Overloaded constructor
PARSING - Enum constant in enum class org.apache.nutch.metrics.ErrorTracker.ErrorType
  Parsing errors (ParseException, ParserNotFound)
partialAsTruncated - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Whether to save partial fetches as truncated content.
PARTITION_MODE_DOMAIN - Static variable in class org.apache.nutch.crawl.URLPartitioner
PARTITION_MODE_HOST - Static variable in class org.apache.nutch.crawl.URLPartitioner
PARTITION_MODE_IP - Static variable in class org.apache.nutch.crawl.URLPartitioner
PARTITION_MODE_KEY - Static variable in class org.apache.nutch.crawl.URLPartitioner
PartitionReducer() - Constructor for class org.apache.nutch.crawl.Generator.PartitionReducer
partName - Variable in class org.apache.nutch.segment.SegmentPart
  Name of the segment part (ie.
passScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.AbstractScoringFilter
passScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.depth.DepthScoringFilter
passScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.link.LinkAnalysisScoringFilter
passScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.metadata.MetadataScoringFilter
  Takes the metadata, which was lumped inside the content, and replicates it within your parse data.
passScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
  Copy the value from Content metadata under Fetcher.SCORE_KEY to parseData.
passScoreAfterParsing(Text, Content, Parse) - Method in interface org.apache.nutch.scoring.ScoringFilter
  Currently a part of score distribution is performed using only data coming from the parsing process.
passScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.ScoringFilters
passScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.similarity.SimilarityScoringFilter
passScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.urlmeta.URLMetaScoringFilter
  Takes the metadata, which was lumped inside the content, and replicates it within your parse data.
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in class org.apache.nutch.scoring.AbstractScoringFilter
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in class org.apache.nutch.scoring.depth.DepthScoringFilter
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in class org.apache.nutch.scoring.link.LinkAnalysisScoringFilter
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in class org.apache.nutch.scoring.metadata.MetadataScoringFilter
  Takes the metadata, specified in your "scoring.db.md" property, from the datum object and injects it into the content.
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
  Store a float value of CrawlDatum.getScore() under Fetcher.SCORE_KEY.
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in interface org.apache.nutch.scoring.ScoringFilter
  This method takes all relevant score information from the current datum (coming from a generated fetchlist) and stores it into `Content` metadata.
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in class org.apache.nutch.scoring.ScoringFilters
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in class org.apache.nutch.scoring.urlmeta.URLMetaScoringFilter
  Takes the metadata, specified in your "urlmeta.tags" property, from the datum object and injects it into the content.
PassURLNormalizer - Class in org.apache.nutch.net.urlnormalizer.pass
  This URLNormalizer doesn't change urls.
PassURLNormalizer() - Constructor for class org.apache.nutch.net.urlnormalizer.pass.PassURLNormalizer
PASSWORD - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
PASSWORD - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
PASSWORD - Static variable in interface org.apache.nutch.indexwriter.solr.SolrConstants
PATH - Static variable in interface org.apache.nutch.indexwriter.dummy.DummyConstants
pattern - Variable in class org.apache.nutch.urlfilter.fast.FastURLFilter.Rule
percentiles - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
PERM_REFRESH_TIME - Static variable in class org.apache.nutch.fetcher.Fetcher
Pluggable - Interface in org.apache.nutch.plugin
  Defines the capability of a class to be plugged into Nutch.
Plugin - Class in org.apache.nutch.plugin
  A nutch-plugin is an container for a set of custom logic that provide extensions to the nutch core functionality or another plugin that provides an API for extending.
Plugin(PluginDescriptor, Configuration) - Constructor for class org.apache.nutch.plugin.Plugin
  Overloaded constructor
PluginClassLoader - Class in org.apache.nutch.plugin
  The `PluginClassLoader` is a child-first classloader that only contains classes of the runtime libraries setuped in the plugin manifest file and exported libraries of plugins that are required plugins.
PluginClassLoader(URL[], ClassLoader) - Constructor for class org.apache.nutch.plugin.PluginClassLoader
  Overloaded constructor
PluginDescriptor - Class in org.apache.nutch.plugin
  The `PluginDescriptor` provide access to all meta information of a nutch-plugin, as well to the internationalizable resources and the plugin own classloader.
PluginDescriptor(String, String, String, String, String, String, Configuration) - Constructor for class org.apache.nutch.plugin.PluginDescriptor
  Overloaded constructor
PluginManifestParser - Class in org.apache.nutch.plugin
  The `PluginManifestParser` provides a mechanism for parsing Nutch plugin manifest files (`plugin.xml`) contained in a `String` of plugin directories.
PluginManifestParser(Configuration, PluginRepository) - Constructor for class org.apache.nutch.plugin.PluginManifestParser
PluginRepository - Class in org.apache.nutch.plugin
  The plugin repository is a registry of all plugins.
PluginRepository(Configuration) - Constructor for class org.apache.nutch.plugin.PluginRepository
PluginRuntimeException - Exception in org.apache.nutch.plugin
  `PluginRuntimeException` will be thrown until a exception in the plugin managemnt occurs.
PluginRuntimeException(String) - Constructor for exception org.apache.nutch.plugin.PluginRuntimeException
PluginRuntimeException(Throwable) - Constructor for exception org.apache.nutch.plugin.PluginRuntimeException
PORT - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
PORT - Static variable in interface org.apache.nutch.indexwriter.kafka.KafkaConstants
PORT - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
PORTERSTEM_FILTER - Enum constant in enum class org.apache.nutch.scoring.similarity.util.LuceneAnalyzerUtil.StemFilterType
pos - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
PrefixStringMatcher - Class in org.apache.nutch.util
  A class for efficiently matching `String`s against a set of prefixes.
PrefixStringMatcher(String[]) - Constructor for class org.apache.nutch.util.PrefixStringMatcher
  Creates a new `PrefixStringMatcher` which will match `String`s with any prefix in the supplied array.
PrefixStringMatcher(Collection<String>) - Constructor for class org.apache.nutch.util.PrefixStringMatcher
  Creates a new `PrefixStringMatcher` which will match `String`s with any prefix in the supplied `Collection`.
PrefixURLFilter - Class in org.apache.nutch.urlfilter.prefix
  Filters URLs based on a file of URL prefixes.
PrefixURLFilter() - Constructor for class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
PrefixURLFilter(String) - Constructor for class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
PrintCommandListener - Class in org.apache.nutch.protocol.ftp
  This is a support class for logging all ftp command/reply traffic.
PrintCommandListener(Logger) - Constructor for class org.apache.nutch.protocol.ftp.PrintCommandListener
PROBLEMATIC_HEADERS - Static variable in class org.apache.nutch.tools.WARCUtils
process(String, StringBuilder) - Method in class org.apache.nutch.crawl.CrawlDbReader
process(String, StringBuilder) - Method in class org.apache.nutch.crawl.LinkDbReader
process(String, StringBuilder) - Method in class org.apache.nutch.indexer.IndexingFiltersChecker
process(String, StringBuilder) - Method in class org.apache.nutch.net.URLFilterChecker
process(String, StringBuilder) - Method in class org.apache.nutch.net.URLNormalizerChecker
process(String, StringBuilder) - Method in class org.apache.nutch.parse.ParserChecker
process(String, StringBuilder) - Method in class org.apache.nutch.util.AbstractChecker
processDeflateEncoded(byte[], URL) - Method in class org.apache.nutch.protocol.http.api.HttpBase
processDriver(WebDriver) - Method in class org.apache.nutch.protocol.interactiveselenium.handlers.DefalultMultiInteractionHandler
processDriver(WebDriver) - Method in class org.apache.nutch.protocol.interactiveselenium.handlers.DefaultClickAllAjaxLinksHandler
processDriver(WebDriver) - Method in class org.apache.nutch.protocol.interactiveselenium.handlers.DefaultHandler
processDriver(WebDriver) - Method in interface org.apache.nutch.protocol.interactiveselenium.handlers.InteractiveSeleniumHandler
processDumpJob(String, String, String) - Method in class org.apache.nutch.crawl.LinkDbReader
processDumpJob(String, String, Configuration, String, String, String, Integer, String, Float) - Method in class org.apache.nutch.crawl.CrawlDbReader
processGzipEncoded(byte[], URL) - Method in class org.apache.nutch.protocol.http.api.HttpBase
processingInstruction(String, String) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive notification of a processing instruction.
processSingle(String) - Method in class org.apache.nutch.util.AbstractChecker
processStatJob(String, Configuration, boolean) - Method in class org.apache.nutch.crawl.CrawlDbReader
processStdin() - Method in class org.apache.nutch.util.AbstractChecker
processTCP(int) - Method in class org.apache.nutch.util.AbstractChecker
processTopNJob(String, long, float, String, Configuration) - Method in class org.apache.nutch.crawl.CrawlDbReader
PROTO_NOT_FOUND - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  This protocol was not found.
PROTO_STATUS_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
Protocol - Interface in org.apache.nutch.protocol
  A retriever of url content.
PROTOCOL - Enum constant in enum class org.apache.nutch.metrics.ErrorTracker.ErrorType
  Protocol errors (ProtocolException, ProtocolNotFound)
PROTOCOL_REDIR - Static variable in class org.apache.nutch.fetcher.Fetcher
PROTOCOL_STATUS_CODE_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
PROTOCOL_VERSIONS - Static variable in interface org.apache.nutch.net.protocols.Response
  Key to hold the HTTP and SSL/TLS protocol versions if `store.protocol.versions` is true.
protocolCommandSent(ProtocolCommandEvent) - Method in class org.apache.nutch.protocol.ftp.PrintCommandListener
ProtocolException - Exception in org.apache.nutch.net.protocols
  Deprecated.
  Use `ProtocolException` instead.
ProtocolException - Exception in org.apache.nutch.protocol
ProtocolException() - Constructor for exception org.apache.nutch.net.protocols.ProtocolException
  Deprecated.
ProtocolException() - Constructor for exception org.apache.nutch.protocol.ProtocolException
ProtocolException(String) - Constructor for exception org.apache.nutch.net.protocols.ProtocolException
  Deprecated.
ProtocolException(String) - Constructor for exception org.apache.nutch.protocol.ProtocolException
ProtocolException(String, Throwable) - Constructor for exception org.apache.nutch.net.protocols.ProtocolException
  Deprecated.
ProtocolException(String, Throwable) - Constructor for exception org.apache.nutch.protocol.ProtocolException
ProtocolException(Throwable) - Constructor for exception org.apache.nutch.net.protocols.ProtocolException
  Deprecated.
ProtocolException(Throwable) - Constructor for exception org.apache.nutch.protocol.ProtocolException
ProtocolFactory - Class in org.apache.nutch.protocol
  Creates and caches `Protocol` plugins.
ProtocolFactory(Configuration) - Constructor for class org.apache.nutch.protocol.ProtocolFactory
ProtocolLogUtil - Class in org.apache.nutch.net.protocols
ProtocolLogUtil() - Constructor for class org.apache.nutch.net.protocols.ProtocolLogUtil
ProtocolNotFound - Exception in org.apache.nutch.protocol
ProtocolNotFound(String) - Constructor for exception org.apache.nutch.protocol.ProtocolNotFound
ProtocolNotFound(String, String) - Constructor for exception org.apache.nutch.protocol.ProtocolNotFound
ProtocolOutput - Class in org.apache.nutch.protocol
  Simple aggregate to pass from protocol plugins both content and protocol status.
ProtocolOutput(Content) - Constructor for class org.apache.nutch.protocol.ProtocolOutput
ProtocolOutput(Content, ProtocolStatus) - Constructor for class org.apache.nutch.protocol.ProtocolOutput
protocolReplyReceived(ProtocolCommandEvent) - Method in class org.apache.nutch.protocol.ftp.PrintCommandListener
ProtocolStatus - Class in org.apache.nutch.protocol
ProtocolStatus() - Constructor for class org.apache.nutch.protocol.ProtocolStatus
ProtocolStatus(int) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
ProtocolStatus(int, long) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
ProtocolStatus(int, Object) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
ProtocolStatus(int, Object, long) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
ProtocolStatus(int, String[]) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
ProtocolStatus(int, String[], long) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
ProtocolStatus(Throwable) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
ProtocolStatusStatistics - Class in org.apache.nutch.util
  Extracts protocol status code information from the crawl database.
ProtocolStatusStatistics() - Constructor for class org.apache.nutch.util.ProtocolStatusStatistics
ProtocolStatusStatistics.ProtocolStatusStatisticsCombiner - Class in org.apache.nutch.util
ProtocolStatusStatisticsCombiner() - Constructor for class org.apache.nutch.util.ProtocolStatusStatistics.ProtocolStatusStatisticsCombiner
ProtocolURLNormalizer - Class in org.apache.nutch.net.urlnormalizer.protocol
  URL normalizer to normalize the protocol for all URLs of a given host or domain, e.g.
ProtocolURLNormalizer() - Constructor for class org.apache.nutch.net.urlnormalizer.protocol.ProtocolURLNormalizer
proxyException - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The proxy exception list.
proxyHost - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The proxy hostname.
proxyPort - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The proxy port.
proxyType - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The proxy port.
publish(Object, Configuration) - Method in interface org.apache.nutch.publisher.NutchPublisher
  This method publishes the event.
publish(Object, Configuration) - Method in class org.apache.nutch.publisher.NutchPublishers
publish(Object, Configuration) - Method in class org.apache.nutch.publisher.rabbitmq.RabbitMQPublisherImpl
publish(String, String, RabbitMQMessage) - Method in class org.apache.nutch.rabbitmq.RabbitMQClient
  Publishes a new message over an exchange.
publish(FetcherThreadEvent, Configuration) - Method in class org.apache.nutch.fetcher.FetcherThreadPublisher
  Publish event to all registered publishers
PUBLISHER - Static variable in interface org.apache.nutch.metadata.DublinCore
  An entity responsible for making the resource available.
purgeFailedHostsThreshold - Variable in class org.apache.nutch.hostdb.ResolverThread
purgeFailedHostsThreshold - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
put(String, ParseText, ParseData) - Method in class org.apache.nutch.parse.ParseResult
  Store a result of parsing.
put(Text, ParseText, ParseData) - Method in class org.apache.nutch.parse.ParseResult
  Store a result of parsing.
putAllMetaData(CrawlDatum) - Method in class org.apache.nutch.crawl.CrawlDatum
  Add all metadata from other CrawlDatum to this CrawlDatum.
putAllMetaData(HostDatum) - Method in class org.apache.nutch.hostdb.HostDatum
  Add all metadata from other HostDatum to this HostDatum.

## Q

query(Map<String, String>, Configuration, String, String) - Method in class org.apache.nutch.crawl.CrawlDbReader
QuerystringURLNormalizer - Class in org.apache.nutch.net.urlnormalizer.querystring
  URL normalizer plugin for normalizing query strings but sorting query string parameters.
QuerystringURLNormalizer() - Constructor for class org.apache.nutch.net.urlnormalizer.querystring.QuerystringURLNormalizer
queue - Variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
QUEUE_MODE_DOMAIN - Static variable in class org.apache.nutch.fetcher.FetchItemQueues
QUEUE_MODE_HOST - Static variable in class org.apache.nutch.fetcher.FetchItemQueues
QUEUE_MODE_IP - Static variable in class org.apache.nutch.fetcher.FetchItemQueues
QueueFeeder - Class in org.apache.nutch.fetcher
  This class feeds the queues with input items, and re-fills them as items are consumed by FetcherThread-s.
QueueFeeder(Mapper.Context, FetchItemQueues, int) - Constructor for class org.apache.nutch.fetcher.QueueFeeder

## R

RabbitIndexWriter - Class in org.apache.nutch.indexwriter.rabbit
RabbitIndexWriter() - Constructor for class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
RabbitMQClient - Class in org.apache.nutch.rabbitmq
  Client for RabbitMQ
RabbitMQClient(String) - Constructor for class org.apache.nutch.rabbitmq.RabbitMQClient
  Builds a new instance of `RabbitMQClient`
RabbitMQClient(String, int, String, String, String) - Constructor for class org.apache.nutch.rabbitmq.RabbitMQClient
  Builds a new instance of `RabbitMQClient`
RabbitMQMessage - Class in org.apache.nutch.rabbitmq
RabbitMQMessage() - Constructor for class org.apache.nutch.rabbitmq.RabbitMQMessage
RabbitMQPublisherImpl - Class in org.apache.nutch.publisher.rabbitmq
RabbitMQPublisherImpl() - Constructor for class org.apache.nutch.publisher.rabbitmq.RabbitMQPublisherImpl
read(DataInput) - Static method in class org.apache.nutch.crawl.CrawlDatum
read(DataInput) - Static method in class org.apache.nutch.crawl.Inlink
read(DataInput) - Static method in class org.apache.nutch.parse.Outlink
read(DataInput) - Static method in class org.apache.nutch.parse.ParseData
read(DataInput) - Static method in class org.apache.nutch.parse.ParseImpl
read(DataInput) - Static method in class org.apache.nutch.parse.ParseStatus
read(DataInput) - Static method in class org.apache.nutch.parse.ParseText
read(DataInput) - Static method in class org.apache.nutch.protocol.Content
read(DataInput) - Static method in class org.apache.nutch.protocol.ProtocolStatus
readConfiguration(Reader) - Method in class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
Reader() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.Reader
readFields(DataInput) - Method in class org.apache.nutch.crawl.CrawlDatum
readFields(DataInput) - Method in class org.apache.nutch.crawl.Generator.SelectorEntry
readFields(DataInput) - Method in class org.apache.nutch.crawl.Inlink
readFields(DataInput) - Method in class org.apache.nutch.crawl.Inlinks
readFields(DataInput) - Method in class org.apache.nutch.hostdb.HostDatum
readFields(DataInput) - Method in class org.apache.nutch.indexer.NutchDocument
readFields(DataInput) - Method in class org.apache.nutch.indexer.NutchField
readFields(DataInput) - Method in class org.apache.nutch.indexer.NutchIndexAction
readFields(DataInput) - Method in class org.apache.nutch.metadata.Metadata
readFields(DataInput) - Method in class org.apache.nutch.metadata.MetaWrapper
readFields(DataInput) - Method in class org.apache.nutch.parse.Outlink
readFields(DataInput) - Method in class org.apache.nutch.parse.ParseData
readFields(DataInput) - Method in class org.apache.nutch.parse.ParseImpl
readFields(DataInput) - Method in class org.apache.nutch.parse.ParseStatus
readFields(DataInput) - Method in class org.apache.nutch.parse.ParseText
readFields(DataInput) - Method in class org.apache.nutch.protocol.Content
readFields(DataInput) - Method in class org.apache.nutch.protocol.ProtocolStatus
readFields(DataInput) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
readFields(DataInput) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
readFields(DataInput) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
readFields(DataInput) - Method in class org.apache.nutch.scoring.webgraph.Node
readFields(DataInput) - Method in class org.apache.nutch.util.GenericWritableConfigurable
readHostDb() - Method in class org.apache.nutch.crawl.Generator.SelectorReducer
ReadHostDb - Class in org.apache.nutch.hostdb
ReadHostDb() - Constructor for class org.apache.nutch.hostdb.ReadHostDb
readingCrawlDb - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
readUrl(String, String, Configuration, StringBuilder) - Method in class org.apache.nutch.crawl.CrawlDbReader
recheckInterval - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
record(long) - Method in class org.apache.nutch.metrics.LatencyTracker
  Records a latency sample.
recordError(Throwable) - Method in class org.apache.nutch.metrics.ErrorTracker
  Records an error with automatic categorization based on the throwable type.
recordError(ErrorTracker.ErrorType) - Method in class org.apache.nutch.metrics.ErrorTracker
  Records an error with explicit category.
REDIR_EXCEEDED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Too many redirects.
redirectIsQueuedRecently(Text) - Method in class org.apache.nutch.fetcher.FetchItemQueues
redirPerm - Variable in class org.apache.nutch.hostdb.HostDatum
redirTemp - Variable in class org.apache.nutch.hostdb.HostDatum
reduce(K, Iterable<CrawlDatum>, Reducer.Context) - Method in class org.apache.nutch.crawl.DeduplicationJob.DedupReducer
reduce(ByteWritable, Iterable<Text>, Reducer.Context) - Method in class org.apache.nutch.indexer.CleaningJob.DeleterReducer
reduce(FloatWritable, Iterable<Text>, Reducer.Context) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNReducer
reduce(FloatWritable, Iterable<Text>, Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter.SorterReducer
reduce(FloatWritable, Iterable<Generator.SelectorEntry>, Reducer.Context) - Method in class org.apache.nutch.crawl.Generator.SelectorReducer
reduce(Text, Iterable<FloatWritable>, Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Dumper.DumperReducer
reduce(Text, Iterable<LongWritable>, Reducer.Context) - Method in class org.apache.nutch.util.CrawlCompletionStats.CrawlCompletionStatsCombiner
reduce(Text, Iterable<LongWritable>, Reducer.Context) - Method in class org.apache.nutch.util.DomainStatistics.DomainStatisticsCombiner
reduce(Text, Iterable<LongWritable>, Reducer.Context) - Method in class org.apache.nutch.util.ProtocolStatusStatistics.ProtocolStatusStatisticsCombiner
reduce(Text, Iterable<ObjectWritable>, Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater.ScoreUpdaterReducer
reduce(Text, Iterable<ObjectWritable>, Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter.InvertReducer
reduce(Text, Iterable<CrawlDatum>, Reducer.Context) - Method in class org.apache.nutch.crawl.CrawlDbMerger.Merger
reduce(Text, Iterable<CrawlDatum>, Reducer.Context) - Method in class org.apache.nutch.crawl.CrawlDbReducer
reduce(Text, Iterable<CrawlDatum>, Reducer.Context) - Method in class org.apache.nutch.crawl.DeduplicationJob.StatusUpdateReducer
reduce(Text, Iterable<CrawlDatum>, Reducer.Context) - Method in class org.apache.nutch.crawl.Generator.CrawlDbUpdater.CrawlDbUpdateReducer
reduce(Text, Iterable<CrawlDatum>, Reducer.Context) - Method in class org.apache.nutch.crawl.Injector.InjectReducer
  Merge the input records of one URL as per rules below :
reduce(Text, Iterable<Generator.SelectorEntry>, Reducer.Context) - Method in class org.apache.nutch.crawl.Generator.PartitionReducer
reduce(Text, Iterable<Generator.SelectorEntry>, Reducer.Context) - Method in class org.apache.nutch.tools.FreeGenerator.FG.FGReducer
reduce(Text, Iterable<Inlinks>, Reducer.Context) - Method in class org.apache.nutch.crawl.LinkDbMerger.LinkDbMergeReducer
reduce(Text, Iterable<NutchWritable>, Reducer.Context) - Method in class org.apache.nutch.tools.warc.WARCExporter.WARCMapReduce.WARCReducer
reduce(Text, Iterable<NutchWritable>, Reducer.Context) - Method in class org.apache.nutch.segment.SegmentReader.InputCompatReducer
reduce(Text, Iterable<NutchWritable>, Reducer.Context) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatReducer
reduce(Text, Iterable<NutchWritable>, Reducer.Context) - Method in class org.apache.nutch.fetcher.Fetcher.FetcherReducer
reduce(Text, Iterable<NutchWritable>, Reducer.Context) - Method in class org.apache.nutch.hostdb.UpdateHostDbReducer
reduce(Text, Iterable<NutchWritable>, Reducer.Context) - Method in class org.apache.nutch.indexer.IndexerMapReduce.IndexerReducer
reduce(Text, Iterable<NutchWritable>, Reducer.Context) - Method in class org.apache.nutch.parse.ParseSegment.ParseSegmentReducer
reduce(Text, Iterable<NutchWritable>, Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb.OutlinkDbReducer
reduce(Text, Iterable<MetaWrapper>, Reducer.Context) - Method in class org.apache.nutch.segment.SegmentMerger.SegmentMergerReducer
reduce(Text, Iterable<LinkDumper.LinkNode>, Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Merger
  Aggregate all LinkNode objects for a given url.
regex() - Method in class org.apache.nutch.urlfilter.api.RegexRule
  Return if this rule's regex.
regexEscape(String) - Method in class org.apache.nutch.indexer.staticfield.StaticFieldIndexer
  Escapes any character that needs escaping so it can be used in a regexp.
regexNormalize(String, String) - Method in class org.apache.nutch.net.urlnormalizer.regex.RegexURLNormalizer
  This function does the replacements by iterating through all the regex patterns.
RegexParseFilter - Class in org.apache.nutch.parsefilter.regex
  RegexParseFilter.
RegexParseFilter() - Constructor for class org.apache.nutch.parsefilter.regex.RegexParseFilter
RegexRule - Class in org.apache.nutch.urlfilter.api
  A generic regular expression rule.
RegexRule(boolean, String) - Constructor for class org.apache.nutch.urlfilter.api.RegexRule
  Constructs a new regular expression rule.
RegexRule(boolean, String, String) - Constructor for class org.apache.nutch.urlfilter.api.RegexRule
  Constructs a new regular expression rule.
RegexURLFilter - Class in org.apache.nutch.urlfilter.regex
  Filters URLs based on a file of regular expressions using the `Java Regex implementation`.
RegexURLFilter() - Constructor for class org.apache.nutch.urlfilter.regex.RegexURLFilter
RegexURLFilter(String) - Constructor for class org.apache.nutch.urlfilter.regex.RegexURLFilter
RegexURLFilterBase - Class in org.apache.nutch.urlfilter.api
  Generic `URLFilter` based on regular expressions.
RegexURLFilterBase() - Constructor for class org.apache.nutch.urlfilter.api.RegexURLFilterBase
  Constructs a new empty RegexURLFilterBase
RegexURLFilterBase(File) - Constructor for class org.apache.nutch.urlfilter.api.RegexURLFilterBase
  Constructs a new RegexURLFilter and init it with a file of rules.
RegexURLFilterBase(Reader) - Constructor for class org.apache.nutch.urlfilter.api.RegexURLFilterBase
  Constructs a new RegexURLFilter and init it with a Reader of rules.
RegexURLFilterBase(String) - Constructor for class org.apache.nutch.urlfilter.api.RegexURLFilterBase
  Constructs a new RegexURLFilter and inits it with a list of rules.
RegexURLNormalizer - Class in org.apache.nutch.net.urlnormalizer.regex
  Allows users to do regex substitutions on all/any URLs that are encountered, which is useful for stripping session IDs from URLs.
RegexURLNormalizer() - Constructor for class org.apache.nutch.net.urlnormalizer.regex.RegexURLNormalizer
  The default constructor which is called from UrlNormalizerFactory (normalizerClass.newInstance()) in method: getNormalizer()*
RegexURLNormalizer(Configuration) - Constructor for class org.apache.nutch.net.urlnormalizer.regex.RegexURLNormalizer
RegexURLNormalizer(Configuration, String) - Constructor for class org.apache.nutch.net.urlnormalizer.regex.RegexURLNormalizer
  Constructor which can be passed the configuration file name, so it doesn't look in other configuration files for it.
REGION - Static variable in interface org.apache.nutch.indexwriter.cloudsearch.CloudSearchConstants
registerPluginRepository(PluginRepository) - Method in class org.apache.nutch.plugin.URLStreamHandlerFactory
  Use this method once a new PluginRepository was created to register it.
REL_TAG - Static variable in class org.apache.nutch.microformats.reltag.RelTagParser
RELATION - Static variable in interface org.apache.nutch.metadata.DublinCore
  A reference to a related resource.
reloadRules() - Method in class org.apache.nutch.urlfilter.fast.FastURLFilter
RelTagIndexingFilter - Class in org.apache.nutch.microformats.reltag
  An `IndexingFilter` that add `tag` field(s) to the document.
RelTagIndexingFilter() - Constructor for class org.apache.nutch.microformats.reltag.RelTagIndexingFilter
RelTagParser - Class in org.apache.nutch.microformats.reltag
  Adds microformat rel-tags of document if found.
RelTagParser() - Constructor for class org.apache.nutch.microformats.reltag.RelTagParser
remove(String) - Method in class org.apache.nutch.metadata.Metadata
  Remove a metadata and all its associated values.
remove(String) - Method in class org.apache.nutch.metadata.SpellCheckedMetadata
removeField(String) - Method in class org.apache.nutch.indexer.NutchDocument
removeLockFile(Configuration, Path) - Static method in class org.apache.nutch.util.LockUtil
  Remove lock file.
removeLockFile(FileSystem, Path) - Static method in class org.apache.nutch.util.LockUtil
  Remove lock file.
replace(String) - Method in class org.apache.nutch.indexer.replace.FieldReplacer
  Return the replacement value for a field value.
replace(FileSystem, Path, Path, boolean) - Static method in class org.apache.nutch.util.FSUtils
  Replaces the current path with the new path and if set removes the old path.
replacefirstoccuranceof(String, String) - Static method in class org.apache.nutch.parsefilter.naivebayes.Train
replaceHost(String, String, String) - Method in class org.apache.nutch.net.urlnormalizer.host.HostURLNormalizer
ReplaceIndexer - Class in org.apache.nutch.indexer.replace
  Do pattern replacements on selected field contents prior to indexing.
ReplaceIndexer() - Constructor for class org.apache.nutch.indexer.replace.ReplaceIndexer
REPORT - Enum constant in enum class org.apache.nutch.fetcher.FetcherThreadEvent.PublishEventType
REPR_URL_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
reprUrl - Variable in class org.apache.nutch.hostdb.UpdateHostDbMapper
REQUEST - Static variable in interface org.apache.nutch.net.protocols.Response
  Key to hold the HTTP request if `store.http.request` is true
reset() - Method in class org.apache.nutch.indexer.NutchField
reset() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets all boolean values to `false`.
resetFailures() - Method in class org.apache.nutch.hostdb.HostDatum
resetStatistics() - Method in class org.apache.nutch.hostdb.HostDatum
resolveEncodingAlias(String) - Static method in class org.apache.nutch.util.EncodingDetector
resolverThread - Variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
ResolverThread - Class in org.apache.nutch.hostdb
  Simple runnable that performs DNS lookup for a single host.
ResolverThread(String, HostDatum, Reducer.Context, int) - Constructor for class org.apache.nutch.hostdb.ResolverThread
  Overloaded constructor.
resolveUrl(URL, String) - Method in class org.apache.nutch.protocol.okhttp.OkHttp
  Resolve a relative URL using OkHttp's `HttpUrl` parser, which is more lenient than Java's `URL` (handles malformed protocol-relative slashes such as `https:////host/path`, IDN→punycode, host case normalization, etc.).
resolveUrl(URL, String) - Method in interface org.apache.nutch.protocol.Protocol
  Resolve a relative URL against a base URL using the protocol's URL library.
resolveURL(URL, String) - Static method in class org.apache.nutch.util.URLUtil
  Resolve relative URL-s and fix a java.net.URL error in handling of URLs with pure query targets.
resolveUrls() - Method in class org.apache.nutch.tools.ResolveUrls
  Creates a thread pool for resolving urls.
ResolveUrls - Class in org.apache.nutch.tools
  A simple tool that will spin up multiple threads to resolve urls to ip addresses.
ResolveUrls(String) - Constructor for class org.apache.nutch.tools.ResolveUrls
  Create a new ResolveUrls with a file from the local file system.
ResolveUrls(String, int) - Constructor for class org.apache.nutch.tools.ResolveUrls
  Create a new ResolveUrls with a urls file and a number of threads for the Thread pool.
Response - Interface in org.apache.nutch.net.protocols
  A response interface.
RESPONSE_HEADERS - Static variable in interface org.apache.nutch.net.protocols.Response
  Key to hold the HTTP response header if `store.http.headers` is true
RESPONSE_TIME - Static variable in class org.apache.nutch.protocol.http.api.HttpBase
Response.TruncatedContentReason - Enum Class in org.apache.nutch.net.protocols
responseTime - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Record response time in CrawlDatum's meta data, see property http.store.responsetime.
results - Variable in class org.apache.nutch.util.NutchTool
retrieveFile(String, OutputStream, int) - Method in class org.apache.nutch.protocol.ftp.Client
  retrieve file for path
retrieveList(String, List<FTPFile>, int, FTPFileEntryParser) - Method in class org.apache.nutch.protocol.ftp.Client
  Retrieve list reply for path
retrieveNgrams(Configuration) - Static method in class org.apache.nutch.scoring.similarity.cosine.Model
  Retrieves mingram and maxgram from configuration
RETRY - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Temporary failure.
reverseHost(String) - Static method in class org.apache.nutch.util.TableUtil
reverseKey - Variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
reverseKeyValue - Variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
reverseUrl(String) - Static method in class org.apache.nutch.tools.CommonCrawlDataDumper
reverseUrl(String) - Static method in class org.apache.nutch.util.TableUtil
  Reverses a url's domain.
reverseUrl(URL) - Static method in class org.apache.nutch.util.TableUtil
  Reverses a url's domain.
rightPad(String, int) - Static method in class org.apache.nutch.util.StringUtil
  Returns a copy of `s` (right padded) with trailing spaces so that it's length is `length`.
RIGHTS - Static variable in interface org.apache.nutch.metadata.DublinCore
  Information about rights held in and over the resource.
RobotRulesParser - Class in org.apache.nutch.protocol
  This class uses crawler-commons for handling the parsing of `robots.txt` files.
RobotRulesParser() - Constructor for class org.apache.nutch.protocol.RobotRulesParser
RobotRulesParser(Configuration) - Constructor for class org.apache.nutch.protocol.RobotRulesParser
ROBOTS - Static variable in class org.apache.nutch.tools.WARCUtils
ROBOTS_DENIED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Access denied by robots.txt rules.
ROBOTS_METATAG - Static variable in interface org.apache.nutch.metadata.Nutch
  Name to store the robots metatag in `ParseData`'s metadata.
root - Variable in class org.apache.nutch.util.TrieStringMatcher
Rule(String) - Constructor for class org.apache.nutch.urlfilter.fast.FastURLFilter.Rule
run() - Method in class org.apache.nutch.fetcher.FetcherThread
run() - Method in class org.apache.nutch.fetcher.QueueFeeder
run() - Method in class org.apache.nutch.hostdb.ResolverThread
run() - Method in class org.apache.nutch.util.AbstractChecker
run(String[]) - Method in class org.apache.nutch.crawl.CrawlDb
run(String[]) - Method in class org.apache.nutch.crawl.CrawlDbMerger
run(String[]) - Method in class org.apache.nutch.crawl.CrawlDbReader
run(String[]) - Method in class org.apache.nutch.crawl.DeduplicationJob
run(String[]) - Method in class org.apache.nutch.crawl.Generator
run(String[]) - Method in class org.apache.nutch.crawl.Injector
run(String[]) - Method in class org.apache.nutch.crawl.LinkDb
run(String[]) - Method in class org.apache.nutch.crawl.LinkDbMerger
run(String[]) - Method in class org.apache.nutch.crawl.LinkDbReader
run(String[]) - Method in class org.apache.nutch.fetcher.Fetcher
run(String[]) - Method in class org.apache.nutch.hostdb.ReadHostDb
run(String[]) - Method in class org.apache.nutch.hostdb.UpdateHostDb
run(String[]) - Method in class org.apache.nutch.indexer.CleaningJob
run(String[]) - Method in class org.apache.nutch.indexer.IndexingFiltersChecker
run(String[]) - Method in class org.apache.nutch.indexer.IndexingJob
run(String[]) - Method in class org.apache.nutch.net.URLFilterChecker
run(String[]) - Method in class org.apache.nutch.net.URLNormalizerChecker
run(String[]) - Method in class org.apache.nutch.parse.ParserChecker
run(String[]) - Method in class org.apache.nutch.parse.ParseSegment
run(String[]) - Method in class org.apache.nutch.protocol.RobotRulesParser
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper
  Runs the LinkDumper tool.
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.LinkRank
  Runs the LinkRank tool.
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper
  Runs the node dumper tool.
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater
  Runs the ScoreUpdater tool.
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.WebGraph
  Parses command link arguments and runs the WebGraph jobs.
run(String[]) - Method in class org.apache.nutch.segment.SegmentMerger
  Run this tool
run(String[]) - Method in class org.apache.nutch.segment.SegmentReader
run(String[]) - Method in class org.apache.nutch.tools.arc.ArcSegmentCreator
run(String[]) - Method in class org.apache.nutch.tools.CommonCrawlDataDumper
run(String[]) - Method in class org.apache.nutch.tools.FreeGenerator
run(String[]) - Method in class org.apache.nutch.tools.ShowProperties
run(String[]) - Method in class org.apache.nutch.tools.warc.WARCExporter
run(String[]) - Method in class org.apache.nutch.util.CrawlCompletionStats
run(String[]) - Method in class org.apache.nutch.util.DomainStatistics
run(String[]) - Method in class org.apache.nutch.util.ProtocolStatusStatistics
run(String[]) - Method in class org.apache.nutch.util.SitemapProcessor
run(Map<String, Object>, String) - Method in class org.apache.nutch.crawl.CrawlDb
  Programmatic entry point for `NutchTool` callers.
run(Map<String, Object>, String) - Method in class org.apache.nutch.crawl.DeduplicationJob
run(Map<String, Object>, String) - Method in class org.apache.nutch.crawl.Generator
run(Map<String, Object>, String) - Method in class org.apache.nutch.crawl.Injector
  Programmatic entry point for `NutchTool` callers.
run(Map<String, Object>, String) - Method in class org.apache.nutch.crawl.LinkDb
  Programmatic entry point for `NutchTool` callers.
run(Map<String, Object>, String) - Method in class org.apache.nutch.fetcher.Fetcher
run(Map<String, Object>, String) - Method in class org.apache.nutch.indexer.IndexingJob
  Programmatic entry point for `NutchTool` callers.
run(Map<String, Object>, String) - Method in class org.apache.nutch.parse.ParseSegment
  Programmatic entry point for `NutchTool` callers.
run(Map<String, Object>, String) - Method in class org.apache.nutch.tools.CommonCrawlDataDumper
  Programmatic entry point for `NutchTool` callers.
run(Map<String, Object>, String) - Method in class org.apache.nutch.util.NutchTool
  Runs the tool, using a map of arguments.
run(Mapper.Context) - Method in class org.apache.nutch.fetcher.Fetcher.FetcherRun

## S

save() - Method in class org.apache.nutch.collection.CollectionManager
  Save collections into file
saveDom(OutputStream, DocumentFragment) - Static method in class org.apache.nutch.util.DomUtil
  Save dom into `OutputStream`
saveDom(OutputStream, Element) - Static method in class org.apache.nutch.util.DomUtil
  Save dom into `OutputStream`
SCHEDULE_DEC_RATE - Static variable in class org.apache.nutch.crawl.MimeAdaptiveFetchSchedule
SCHEDULE_INC_RATE - Static variable in class org.apache.nutch.crawl.MimeAdaptiveFetchSchedule
SCHEDULE_MIME_FILE - Static variable in class org.apache.nutch.crawl.MimeAdaptiveFetchSchedule
SCHEME - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
SCHEME - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
SCOPE_CRAWLDB - Static variable in class org.apache.nutch.net.URLNormalizers
  Scope used when updating the CrawlDb with new URLs.
SCOPE_DEFAULT - Static variable in class org.apache.nutch.net.URLNormalizers
  Default scope.
SCOPE_FETCHER - Static variable in class org.apache.nutch.net.URLNormalizers
  Scope used by `Fetcher` when processing redirect URLs.
SCOPE_GENERATE_HOST_COUNT - Static variable in class org.apache.nutch.net.URLNormalizers
  Scope used by `Generator`.
SCOPE_INDEXER - Static variable in class org.apache.nutch.net.URLNormalizers
  Scope used when indexing URLs.
SCOPE_INJECT - Static variable in class org.apache.nutch.net.URLNormalizers
  Scope used by `Injector`.
SCOPE_LINKDB - Static variable in class org.apache.nutch.net.URLNormalizers
  Scope used when updating the LinkDb with new URLs.
SCOPE_OUTLINK - Static variable in class org.apache.nutch.net.URLNormalizers
  Scope used when constructing new `Outlink` instances.
SCOPE_PARTITION - Static variable in class org.apache.nutch.net.URLNormalizers
  Scope used by `URLPartitioner`.
score - Variable in class org.apache.nutch.hostdb.HostDatum
SCORE_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
ScoreUpdater - Class in org.apache.nutch.scoring.webgraph
  Updates the score from the WebGraph node database into the crawl database.
ScoreUpdater() - Constructor for class org.apache.nutch.scoring.webgraph.ScoreUpdater
ScoreUpdater.ScoreUpdaterMapper - Class in org.apache.nutch.scoring.webgraph
  Changes input into ObjectWritables.
ScoreUpdater.ScoreUpdaterReducer - Class in org.apache.nutch.scoring.webgraph
  Creates new CrawlDatum objects with the updated score from the NodeDb or with a cleared score.
ScoreUpdaterMapper() - Constructor for class org.apache.nutch.scoring.webgraph.ScoreUpdater.ScoreUpdaterMapper
ScoreUpdaterReducer() - Constructor for class org.apache.nutch.scoring.webgraph.ScoreUpdater.ScoreUpdaterReducer
SCORING - Enum constant in enum class org.apache.nutch.metrics.ErrorTracker.ErrorType
  Scoring filter errors
ScoringFilter - Interface in org.apache.nutch.scoring
  A contract defining behavior of scoring plugins.
ScoringFilterException - Exception in org.apache.nutch.scoring
  Specialized exception for errors during scoring.
ScoringFilterException() - Constructor for exception org.apache.nutch.scoring.ScoringFilterException
ScoringFilterException(String) - Constructor for exception org.apache.nutch.scoring.ScoringFilterException
ScoringFilterException(String, Throwable) - Constructor for exception org.apache.nutch.scoring.ScoringFilterException
ScoringFilterException(Throwable) - Constructor for exception org.apache.nutch.scoring.ScoringFilterException
ScoringFilters - Class in org.apache.nutch.scoring
  Creates and caches `ScoringFilter` implementing plugins.
ScoringFilters(Configuration) - Constructor for class org.apache.nutch.scoring.ScoringFilters
sdf - Static variable in class org.apache.nutch.util.SitemapProcessor
SECONDS_PER_DAY - Static variable in interface org.apache.nutch.crawl.FetchSchedule
secondsToDaysHMS(long) - Static method in class org.apache.nutch.util.TimingUtil
  Show time in seconds as days, hours, minutes and seconds (d days, hh:mm:ss)
secondsToHMS(long) - Static method in class org.apache.nutch.util.TimingUtil
  Show time in seconds as hours, minutes and seconds (hh:mm:ss)
SEGMENT_NAME_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
SegmentChecker - Class in org.apache.nutch.segment
  Checks whether a segment is valid, or has a certain status (generated, fetched, parsed), or can be used safely for a certain processing step (e.g., indexing).
SegmentChecker() - Constructor for class org.apache.nutch.segment.SegmentChecker
SegmentMergeFilter - Interface in org.apache.nutch.segment
  Interface used to filter segments during segment merge.
SegmentMergeFilters - Class in org.apache.nutch.segment
  This class wraps all `SegmentMergeFilter` extensions in a single object so it is easier to operate on them.
SegmentMergeFilters(Configuration) - Constructor for class org.apache.nutch.segment.SegmentMergeFilters
SegmentMerger - Class in org.apache.nutch.segment
  This tool takes several segments and merges their data together.
SegmentMerger() - Constructor for class org.apache.nutch.segment.SegmentMerger
SegmentMerger(Configuration) - Constructor for class org.apache.nutch.segment.SegmentMerger
SegmentMerger.ObjectInputFormat - Class in org.apache.nutch.segment
  Wraps inputs in an `MetaWrapper`, to permit merging different types in reduce and use additional metadata.
SegmentMerger.SegmentMergerMapper - Class in org.apache.nutch.segment
SegmentMerger.SegmentMergerReducer - Class in org.apache.nutch.segment
  NOTE: in selecting the latest version we rely exclusively on the segment name (not all segment data contain time information).
SegmentMerger.SegmentOutputFormat - Class in org.apache.nutch.segment
SegmentMergerMapper() - Constructor for class org.apache.nutch.segment.SegmentMerger.SegmentMergerMapper
SegmentMergerReducer() - Constructor for class org.apache.nutch.segment.SegmentMerger.SegmentMergerReducer
segmentName - Variable in class org.apache.nutch.segment.SegmentPart
  Name of the segment (just the last path component).
SegmentOutputFormat() - Constructor for class org.apache.nutch.segment.SegmentMerger.SegmentOutputFormat
SegmentPart - Class in org.apache.nutch.segment
  Utility class for handling information about segment parts.
SegmentPart() - Constructor for class org.apache.nutch.segment.SegmentPart
SegmentPart(String, String) - Constructor for class org.apache.nutch.segment.SegmentPart
SegmentReader - Class in org.apache.nutch.segment
  Dump the content of a segment.
SegmentReader() - Constructor for class org.apache.nutch.segment.SegmentReader
SegmentReader.InputCompatMapper - Class in org.apache.nutch.segment
SegmentReader.InputCompatReducer - Class in org.apache.nutch.segment
SegmentReader.SegmentReaderStats - Class in org.apache.nutch.segment
SegmentReader.TextOutputFormat - Class in org.apache.nutch.segment
  Implements a text output format
SegmentReaderStats() - Constructor for class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
SegmentReaderUtil - Class in org.apache.nutch.util
SegmentReaderUtil() - Constructor for class org.apache.nutch.util.SegmentReaderUtil
segnum - Variable in class org.apache.nutch.crawl.Generator.SelectorEntry
Selector() - Constructor for class org.apache.nutch.crawl.Generator.Selector
SelectorEntry() - Constructor for class org.apache.nutch.crawl.Generator.SelectorEntry
SelectorInverseMapper() - Constructor for class org.apache.nutch.crawl.Generator.SelectorInverseMapper
SelectorMapper() - Constructor for class org.apache.nutch.crawl.Generator.SelectorMapper
SelectorReducer() - Constructor for class org.apache.nutch.crawl.Generator.SelectorReducer
sendNoOp() - Method in class org.apache.nutch.protocol.ftp.Client
  Sends a NOOP command to the FTP server.
Separator(String) - Constructor for class org.apache.nutch.indexwriter.csv.CSVIndexWriter.Separator
sepStr - Variable in class org.apache.nutch.indexwriter.csv.CSVIndexWriter.Separator
serialize(Writable, JsonGenerator, SerializerProvider) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumJsonOutputFormat.WritableSerializer
SERVER_TYPE - Static variable in interface org.apache.nutch.indexwriter.solr.SolrConstants
SERVER_URLS - Static variable in interface org.apache.nutch.indexwriter.solr.SolrConstants
set(String) - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter.Separator
set(String, String) - Method in class org.apache.nutch.metadata.Metadata
  Set metadata name/value.
set(String, String) - Method in class org.apache.nutch.metadata.SpellCheckedMetadata
set(CrawlDatum) - Method in class org.apache.nutch.crawl.CrawlDatum
  Copy the contents of another instance into this instance.
setAdditionalPostHeaders(Map<String, String>) - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
setAll(Properties) - Method in class org.apache.nutch.metadata.Metadata
  Copy All key-value pairs from properties.
setAnchor(String) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
setArgs(String[]) - Method in class org.apache.nutch.parse.ParseStatus
setArgs(String[]) - Method in class org.apache.nutch.protocol.ProtocolStatus
setBaseHref(URL) - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets the `baseHref`.
setBlackList(String) - Method in class org.apache.nutch.collection.Subcollection
  Set contents of blacklist from String
setBody(byte[]) - Method in class org.apache.nutch.rabbitmq.RabbitMQMessage
setCache() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets `noCache` to `false`.
setClazz(String) - Method in class org.apache.nutch.plugin.Extension
  Sets the Class that implement the concret extension and is only used until model creation at system start up.
setCode(int) - Method in class org.apache.nutch.protocol.ProtocolStatus
setCommand(String) - Method in class org.apache.nutch.util.CommandRunner
setCompressed(boolean) - Method in class org.apache.nutch.tools.CommonCrawlConfig
setConf(Configuration) - Method in class org.apache.nutch.analysis.lang.HTMLLanguageParser
setConf(Configuration) - Method in class org.apache.nutch.analysis.lang.LanguageIndexingFilter
setConf(Configuration) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
setConf(Configuration) - Method in class org.apache.nutch.crawl.AdaptiveFetchSchedule
setConf(Configuration) - Method in class org.apache.nutch.crawl.Generator.Selector
setConf(Configuration) - Method in class org.apache.nutch.crawl.MimeAdaptiveFetchSchedule
setConf(Configuration) - Method in class org.apache.nutch.crawl.Signature
setConf(Configuration) - Method in class org.apache.nutch.crawl.TextProfileSignature
setConf(Configuration) - Method in class org.apache.nutch.crawl.URLPartitioner
setConf(Configuration) - Method in class org.apache.nutch.exchange.jexl.JexlExchange
setConf(Configuration) - Method in class org.apache.nutch.indexer.anchor.AnchorIndexingFilter
  Set the `Configuration` object
setConf(Configuration) - Method in class org.apache.nutch.indexer.arbitrary.ArbitraryIndexingFilter
  Set the `Configuration` object
setConf(Configuration) - Method in class org.apache.nutch.indexer.basic.BasicIndexingFilter
  Set the `Configuration` object
setConf(Configuration) - Method in class org.apache.nutch.indexer.CleaningJob
setConf(Configuration) - Method in class org.apache.nutch.indexer.feed.FeedIndexingFilter
  Sets the `Configuration` object used to configure this `IndexingFilter`.
setConf(Configuration) - Method in class org.apache.nutch.indexer.filter.MimeTypeIndexingFilter
setConf(Configuration) - Method in class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter
setConf(Configuration) - Method in class org.apache.nutch.indexer.jexl.JexlIndexingFilter
setConf(Configuration) - Method in class org.apache.nutch.indexer.links.LinksIndexingFilter
setConf(Configuration) - Method in class org.apache.nutch.indexer.metadata.MetadataIndexer
setConf(Configuration) - Method in class org.apache.nutch.indexer.more.MoreIndexingFilter
setConf(Configuration) - Method in class org.apache.nutch.indexer.replace.ReplaceIndexer
setConf(Configuration) - Method in class org.apache.nutch.indexer.staticfield.StaticFieldIndexer
  Set the `Configuration` object
setConf(Configuration) - Method in class org.apache.nutch.indexer.subcollection.SubcollectionIndexingFilter
setConf(Configuration) - Method in class org.apache.nutch.indexer.tld.TLDIndexingFilter
setConf(Configuration) - Method in class org.apache.nutch.indexer.urlmeta.URLMetaIndexingFilter
  handles conf assignment and pulls the value assignment from the "urlmeta.tags" property
setConf(Configuration) - Method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
setConf(Configuration) - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
setConf(Configuration) - Method in class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
setConf(Configuration) - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
setConf(Configuration) - Method in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
setConf(Configuration) - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
setConf(Configuration) - Method in class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
setConf(Configuration) - Method in class org.apache.nutch.indexwriter.solr.SolrIndexWriter
setConf(Configuration) - Method in class org.apache.nutch.microformats.reltag.RelTagIndexingFilter
setConf(Configuration) - Method in class org.apache.nutch.microformats.reltag.RelTagParser
setConf(Configuration) - Method in class org.apache.nutch.net.protocols.ProtocolLogUtil
setConf(Configuration) - Method in class org.apache.nutch.net.urlnormalizer.ajax.AjaxURLNormalizer
setConf(Configuration) - Method in class org.apache.nutch.net.urlnormalizer.basic.BasicURLNormalizer
setConf(Configuration) - Method in class org.apache.nutch.net.urlnormalizer.host.HostURLNormalizer
setConf(Configuration) - Method in class org.apache.nutch.net.urlnormalizer.pass.PassURLNormalizer
setConf(Configuration) - Method in class org.apache.nutch.net.urlnormalizer.protocol.ProtocolURLNormalizer
setConf(Configuration) - Method in class org.apache.nutch.net.urlnormalizer.querystring.QuerystringURLNormalizer
setConf(Configuration) - Method in class org.apache.nutch.net.urlnormalizer.regex.RegexURLNormalizer
setConf(Configuration) - Method in class org.apache.nutch.net.urlnormalizer.slash.SlashURLNormalizer
setConf(Configuration) - Method in class org.apache.nutch.parse.ext.ExtParser
setConf(Configuration) - Method in class org.apache.nutch.parse.feed.FeedParser
  Sets the `Configuration` object for this `Parser`.
setConf(Configuration) - Method in class org.apache.nutch.parse.headings.HeadingsParseFilter
setConf(Configuration) - Method in class org.apache.nutch.parse.html.DOMContentUtils
setConf(Configuration) - Method in class org.apache.nutch.parse.html.HtmlParser
setConf(Configuration) - Method in class org.apache.nutch.parse.js.JSParseFilter
setConf(Configuration) - Method in class org.apache.nutch.parse.metatags.MetaTagsParser
setConf(Configuration) - Method in class org.apache.nutch.parse.tika.DOMContentUtils
setConf(Configuration) - Method in class org.apache.nutch.parse.tika.TikaParser
setConf(Configuration) - Method in class org.apache.nutch.parse.zip.ZipParser
setConf(Configuration) - Method in class org.apache.nutch.parsefilter.debug.DebugParseFilter
setConf(Configuration) - Method in class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
setConf(Configuration) - Method in class org.apache.nutch.parsefilter.regex.RegexParseFilter
setConf(Configuration) - Method in class org.apache.nutch.protocol.file.File
  Set the `Configuration` object
setConf(Configuration) - Method in class org.apache.nutch.protocol.ftp.Ftp
  Set the `Configuration` object
setConf(Configuration) - Method in class org.apache.nutch.protocol.htmlunit.Http
  Set the `Configuration` object.
setConf(Configuration) - Method in class org.apache.nutch.protocol.http.api.HttpBase
setConf(Configuration) - Method in class org.apache.nutch.protocol.http.api.HttpRobotRulesParser
setConf(Configuration) - Method in class org.apache.nutch.protocol.http.Http
  Set the `Configuration` object.
setConf(Configuration) - Method in class org.apache.nutch.protocol.httpclient.Http
  Reads the configuration from the Nutch configuration files and sets the configuration.
setConf(Configuration) - Method in class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
setConf(Configuration) - Method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
setConf(Configuration) - Method in class org.apache.nutch.protocol.interactiveselenium.Http
setConf(Configuration) - Method in class org.apache.nutch.protocol.okhttp.OkHttp
setConf(Configuration) - Method in class org.apache.nutch.protocol.RobotRulesParser
  Set the `Configuration` object
setConf(Configuration) - Method in class org.apache.nutch.publisher.NutchPublishers
setConf(Configuration) - Method in class org.apache.nutch.publisher.rabbitmq.RabbitMQPublisherImpl
setConf(Configuration) - Method in class org.apache.nutch.scoring.AbstractScoringFilter
setConf(Configuration) - Method in class org.apache.nutch.scoring.depth.DepthScoringFilter
setConf(Configuration) - Method in class org.apache.nutch.scoring.link.LinkAnalysisScoringFilter
setConf(Configuration) - Method in class org.apache.nutch.scoring.metadata.MetadataScoringFilter
  handles conf assignment and pulls the value assignment from the "scoring.db.md", "scoring.content.md" and "scoring.parse.md" properties.
setConf(Configuration) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
setConf(Configuration) - Method in class org.apache.nutch.scoring.orphan.OrphanScoringFilter
setConf(Configuration) - Method in class org.apache.nutch.scoring.similarity.cosine.CosineSimilarity
setConf(Configuration) - Method in interface org.apache.nutch.scoring.similarity.SimilarityModel
setConf(Configuration) - Method in class org.apache.nutch.scoring.similarity.SimilarityScoringFilter
setConf(Configuration) - Method in class org.apache.nutch.scoring.urlmeta.URLMetaScoringFilter
  handles conf assignment and pulls the value assignment from the "urlmeta.tags" property
setConf(Configuration) - Method in class org.apache.nutch.segment.SegmentMerger
setConf(Configuration) - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
setConf(Configuration) - Method in class org.apache.nutch.urlfilter.domain.DomainURLFilter
  Sets the configuration.
setConf(Configuration) - Method in class org.apache.nutch.urlfilter.domaindenylist.DomainDenylistURLFilter
  Sets the configuration.
setConf(Configuration) - Method in class org.apache.nutch.urlfilter.fast.FastURLFilter
setConf(Configuration) - Method in class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
setConf(Configuration) - Method in class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
setConf(Configuration) - Method in class org.apache.nutch.urlfilter.validator.UrlValidator
setConf(Configuration) - Method in class org.apache.nutch.util.GenericWritableConfigurable
setConf(Configuration) - Method in class org.apache.nutch.util.NutchTool
setConf(Configuration) - Method in class org.creativecommons.nutch.CCIndexingFilter
setConf(Configuration) - Method in class org.creativecommons.nutch.CCParseFilter
setConfig(Configuration) - Method in interface org.apache.nutch.publisher.NutchPublisher
  Use implementation specific configurations
setConfig(Configuration) - Method in class org.apache.nutch.publisher.NutchPublishers
setConfig(Configuration) - Method in class org.apache.nutch.publisher.rabbitmq.RabbitMQPublisherImpl
setConnectionFailures(Long) - Method in class org.apache.nutch.hostdb.HostDatum
setContent(byte[]) - Method in class org.apache.nutch.protocol.Content
setContent(Content) - Method in class org.apache.nutch.protocol.ProtocolOutput
setContentType(String) - Method in class org.apache.nutch.protocol.Content
setContentType(String) - Method in class org.apache.nutch.rabbitmq.RabbitMQMessage
setCookie(Text) - Method in class org.apache.nutch.fetcher.FetchItemQueue
setCookiePolicy(String) - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
setCookies(String) - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthentication
setDataTimeout(int) - Method in class org.apache.nutch.protocol.ftp.Client
  Sets the timeout in milliseconds to use for data connection.
setDescriptor(PluginDescriptor) - Method in class org.apache.nutch.plugin.Extension
  Sets the plugin descriptor and is only used until model creation at system start up.
setDnsFailures(Long) - Method in class org.apache.nutch.hostdb.HostDatum
setDocumentLocator(Locator) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive an object for locating the origin of SAX document events.
setEventData(Map<String, Object>) - Method in class org.apache.nutch.fetcher.FetcherThreadEvent
  Set metadata to this even
setEventType(FetcherThreadEvent.PublishEventType) - Method in class org.apache.nutch.fetcher.FetcherThreadEvent
  Set event type of this object
setFetched(long) - Method in class org.apache.nutch.hostdb.HostDatum
setFetchInterval(float) - Method in class org.apache.nutch.crawl.CrawlDatum
setFetchInterval(int) - Method in class org.apache.nutch.crawl.CrawlDatum
setFetchSchedule(Text, CrawlDatum, long, long, long, long, int) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
  Sets the `fetchInterval` and `fetchTime` on a successfully fetched page.
setFetchSchedule(Text, CrawlDatum, long, long, long, long, int) - Method in class org.apache.nutch.crawl.AdaptiveFetchSchedule
setFetchSchedule(Text, CrawlDatum, long, long, long, long, int) - Method in class org.apache.nutch.crawl.DefaultFetchSchedule
setFetchSchedule(Text, CrawlDatum, long, long, long, long, int) - Method in interface org.apache.nutch.crawl.FetchSchedule
  Sets the `fetchInterval` and `fetchTime` on a successfully fetched page.
setFetchSchedule(Text, CrawlDatum, long, long, long, long, int) - Method in class org.apache.nutch.crawl.MimeAdaptiveFetchSchedule
setFetchTime(long) - Method in class org.apache.nutch.crawl.CrawlDatum
  Sets either the time of the last fetch or the next fetch time, depending on whether Fetcher or CrawlDbReducer set the time.
setFileType(int) - Method in class org.apache.nutch.protocol.ftp.Client
  Sets the file type to be transferred.
setFilterFromPath(boolean) - Method in class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
setFollow() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets `noFollow` to `false`.
setFollowTalk(boolean) - Method in class org.apache.nutch.protocol.ftp.Ftp
  Set followTalk i.e.
setFromConf(IndexWriterParams, String) - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter.Separator
setFromConf(IndexWriterParams, String, boolean) - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter.Separator
setGone(long) - Method in class org.apache.nutch.hostdb.HostDatum
setHalted(boolean) - Method in class org.apache.nutch.fetcher.FetcherThread
setHeaders(String) - Method in class org.apache.nutch.rabbitmq.RabbitMQMessage
setHeaders(Map<String, Object>) - Method in class org.apache.nutch.rabbitmq.RabbitMQMessage
setHomepageUrl(String) - Method in class org.apache.nutch.hostdb.HostDatum
setId(String) - Method in class org.apache.nutch.plugin.Extension
  Sets the unique extension Id and is only used until model creation at system start up.
setIDAttribute(String, Element) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Set an ID string to node association in the ID table.
setIgnoreCase(boolean) - Method in class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
setIndex() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets `noIndex` to `false`.
setIndexedConf(Configuration, int) - Method in class org.apache.nutch.indexer.arbitrary.ArbitraryIndexingFilter
  Set the `Configuration` object for a specific set of values in the config
setInLinks(List<String>) - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
setInLinks(List<String>) - Method in interface org.apache.nutch.tools.CommonCrawlFormat
  sets inlinks of this document
setInlinkScore(float) - Method in class org.apache.nutch.scoring.webgraph.Node
setInputStream(InputStream) - Method in class org.apache.nutch.util.CommandRunner
setJobLevelCounters(TaskInputOutputContext<?, ?, ?, ?>, String, String, long, long, MergingDigest) - Static method in class org.apache.nutch.metrics.LatencyTracker
  Sets job-level percentile counters from a merged digest (e.g.
setJsonArray(boolean) - Method in class org.apache.nutch.tools.CommonCrawlConfig
setKeepConnection(boolean) - Method in class org.apache.nutch.protocol.ftp.Ftp
  Whether to keep ftp connection.
setKeyPrefix(String) - Method in class org.apache.nutch.tools.CommonCrawlConfig
setLastCheck() - Method in class org.apache.nutch.hostdb.HostDatum
setLastCheck(Date) - Method in class org.apache.nutch.hostdb.HostDatum
setLastModified(long) - Method in class org.apache.nutch.protocol.ProtocolStatus
setLinks(LinkDumper.LinkNode[]) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
setLinkType(byte) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
setLoginFormId(String) - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
setLoginPostData(Map<String, String>) - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
setLoginRedirect(boolean) - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
setLoginUrl(String) - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
setMajorCode(byte) - Method in class org.apache.nutch.parse.ParseStatus
setMaxContentLength(int) - Method in class org.apache.nutch.protocol.file.File
  Set the length after at which content is truncated.
setMaxContentLength(int) - Method in class org.apache.nutch.protocol.ftp.Ftp
  Set the length after at which content is truncated.
setMessage(String) - Method in class org.apache.nutch.parse.ParseStatus
setMessage(String) - Method in class org.apache.nutch.protocol.ProtocolStatus
setMeta(String, String) - Method in class org.apache.nutch.metadata.MetaWrapper
  Set metadata.
setMetadata(MapWritable) - Method in class org.apache.nutch.crawl.Inlink
setMetadata(MapWritable) - Method in class org.apache.nutch.parse.Outlink
setMetadata(Metadata) - Method in class org.apache.nutch.protocol.Content
  Other protocol-specific data.
setMetadata(Metadata) - Method in class org.apache.nutch.scoring.webgraph.Node
setMetaData(MapWritable) - Method in class org.apache.nutch.crawl.CrawlDatum
setMetaData(MapWritable) - Method in class org.apache.nutch.hostdb.HostDatum
setMinorCode(short) - Method in class org.apache.nutch.parse.ParseStatus
setModeAccept(boolean) - Method in class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
setModifiedTime(long) - Method in class org.apache.nutch.crawl.CrawlDatum
setNoCache() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets `noCache` to `true`.
setNode(Node) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
setNoFollow() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets `noFollow` to `true`.
setNoIndex() - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets `noIndex` to `true`.
setNotModified(long) - Method in class org.apache.nutch.hostdb.HostDatum
setNumInlinks(int) - Method in class org.apache.nutch.scoring.webgraph.Node
setNumOutlinks(int) - Method in class org.apache.nutch.scoring.webgraph.Node
setObject(String, Object) - Method in class org.apache.nutch.util.ObjectCache
setOutlinks(Outlink[]) - Method in class org.apache.nutch.parse.ParseData
setOutputDir(String) - Method in class org.apache.nutch.tools.CommonCrawlConfig
setPageGoneSchedule(Text, CrawlDatum, long, long, long) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
  This method specifies how to schedule refetching of pages marked as GONE.
setPageGoneSchedule(Text, CrawlDatum, long, long, long) - Method in interface org.apache.nutch.crawl.FetchSchedule
  This method specifies how to schedule refetching of pages marked as GONE.
setPageRetrySchedule(Text, CrawlDatum, long, long, long) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
  This method adjusts the fetch schedule if fetching needs to be re-tried due to transient errors.
setPageRetrySchedule(Text, CrawlDatum, long, long, long) - Method in interface org.apache.nutch.crawl.FetchSchedule
  This method adjusts the fetch schedule if fetching needs to be re-tried due to transient errors.
setParseMeta(Metadata) - Method in class org.apache.nutch.parse.ParseData
setPoolSize(int) - Static method in class org.apache.nutch.util.MimeUtil
setReason(Response.TruncatedContentReason) - Method in class org.apache.nutch.protocol.okhttp.OkHttpResponse.TruncatedContent
setRedirect(boolean) - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthentication
setRedirPerm(long) - Method in class org.apache.nutch.hostdb.HostDatum
setRedirTemp(long) - Method in class org.apache.nutch.hostdb.HostDatum
setRefresh(boolean) - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets `refresh` to the supplied value.
setRefreshHref(URL) - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets the `refreshHref`.
setRefreshTime(int) - Method in class org.apache.nutch.parse.HTMLMetaTags
  Sets the `refreshTime`.
setRemoteVerificationEnabled(boolean) - Method in class org.apache.nutch.protocol.ftp.Client
  Enable or disable verification that the remote host taking part of a data connection is the same as the host to which the control connection is attached.
setRemovedFormFields(Set<String>) - Method in class org.apache.nutch.protocol.httpclient.HttpFormAuthConfigurer
setRetriesSinceFetch(int) - Method in class org.apache.nutch.crawl.CrawlDatum
setReverseKey(boolean) - Method in class org.apache.nutch.tools.CommonCrawlConfig
setReverseKeyValue(String) - Method in class org.apache.nutch.tools.CommonCrawlConfig
setScore(float) - Method in class org.apache.nutch.crawl.CrawlDatum
setScore(float) - Method in class org.apache.nutch.hostdb.HostDatum
setScore(float) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
setSignature(byte[]) - Method in class org.apache.nutch.crawl.CrawlDatum
setSimpleDateFormat(boolean) - Method in class org.apache.nutch.tools.CommonCrawlConfig
setStatus(int) - Method in class org.apache.nutch.crawl.CrawlDatum
setStatus(ProtocolStatus) - Method in class org.apache.nutch.protocol.ProtocolOutput
setStdErrorStream(OutputStream) - Method in class org.apache.nutch.util.CommandRunner
setStdOutputStream(OutputStream) - Method in class org.apache.nutch.util.CommandRunner
setTermFreqVector(HashMap<String, Integer>) - Method in class org.apache.nutch.scoring.similarity.cosine.DocVector
setTimeout(int) - Method in class org.apache.nutch.protocol.ftp.Ftp
  Set the timeout.
setTimeout(int) - Method in class org.apache.nutch.util.CommandRunner
setTimeoutReached() - Method in class org.apache.nutch.fetcher.FetchItemQueues
  Signal that the hard timeout is reached because new fetches / requests where made during half of the MapReduce task timeout (`mapreduce.task.timeout`, default value: 10 minutes).
setTimestamp(long) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
setTimestamp(Long) - Method in class org.apache.nutch.fetcher.FetcherThreadEvent
  Set timestamp for this event
setUnfetched(long) - Method in class org.apache.nutch.hostdb.HostDatum
setup(Mapper.Context) - Method in class org.apache.nutch.tools.arc.ArcSegmentCreator.ArcSegmentCreatorMapper
  Configures the job mapper.
setup(Mapper.Context) - Method in class org.apache.nutch.crawl.Injector.InjectMapper
setup(Mapper.Context) - Method in class org.apache.nutch.hostdb.UpdateHostDbMapper
setup(Mapper.Context) - Method in class org.apache.nutch.indexer.IndexerMapReduce.IndexerMapper
setup(Mapper.Context) - Method in class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb.OutlinkDbMapper
  Configures the OutlinkDb job mapper.
setup(Mapper.Context) - Method in class org.apache.nutch.crawl.DeduplicationJob.DBFilter
setup(Mapper.Context) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNMapper
setup(Mapper.Context) - Method in class org.apache.nutch.crawl.Generator.SelectorMapper
setup(Mapper.Context) - Method in class org.apache.nutch.crawl.CrawlDbFilter
setup(Mapper.Context) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbDumpMapper
setup(Mapper.Context) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatMapper
setup(Mapper.Context) - Method in class org.apache.nutch.fetcher.Fetcher.FetcherRun
setup(Mapper.Context) - Method in class org.apache.nutch.crawl.LinkDbFilter
setup(Mapper.Context) - Method in class org.apache.nutch.crawl.LinkDbReader.LinkDBDumpMapper
setup(Mapper.Context) - Method in class org.apache.nutch.segment.SegmentMerger.SegmentMergerMapper
setup(Mapper.Context) - Method in class org.apache.nutch.crawl.LinkDb.LinkDbMapper
setup(Mapper.Context) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter.SorterMapper
  Configures the mapper, sets the flag for type of content and the topN number if any.
setup(Mapper.Context) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Dumper.DumperMapper
setup(Mapper.Context) - Method in class org.apache.nutch.tools.FreeGenerator.FG.FGMapper
setup(Mapper.Context) - Method in class org.apache.nutch.parse.ParseSegment.ParseSegmentMapper
setup(Reducer.Context) - Method in class org.apache.nutch.crawl.DeduplicationJob.DedupReducer
setup(Reducer.Context) - Method in class org.apache.nutch.indexer.CleaningJob.DeleterReducer
setup(Reducer.Context) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNReducer
setup(Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter.SorterReducer
  Configures the reducer, sets the flag for type of content and the topN number if any.
setup(Reducer.Context) - Method in class org.apache.nutch.crawl.Generator.SelectorReducer
setup(Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Dumper.DumperReducer
setup(Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater.ScoreUpdaterReducer
setup(Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter.InvertReducer
setup(Reducer.Context) - Method in class org.apache.nutch.crawl.CrawlDbMerger.Merger
setup(Reducer.Context) - Method in class org.apache.nutch.crawl.CrawlDbReducer
setup(Reducer.Context) - Method in class org.apache.nutch.crawl.DeduplicationJob.StatusUpdateReducer
setup(Reducer.Context) - Method in class org.apache.nutch.crawl.Generator.CrawlDbUpdater.CrawlDbUpdateReducer
setup(Reducer.Context) - Method in class org.apache.nutch.crawl.Injector.InjectReducer
setup(Reducer.Context) - Method in class org.apache.nutch.crawl.LinkDbMerger.LinkDbMergeReducer
setup(Reducer.Context) - Method in class org.apache.nutch.tools.warc.WARCExporter.WARCMapReduce.WARCReducer
setup(Reducer.Context) - Method in class org.apache.nutch.segment.SegmentReader.InputCompatReducer
setup(Reducer.Context) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatReducer
setup(Reducer.Context) - Method in class org.apache.nutch.hostdb.UpdateHostDbReducer
  Configures the thread pool and prestarts all resolver threads.
setup(Reducer.Context) - Method in class org.apache.nutch.indexer.IndexerMapReduce.IndexerReducer
setup(Reducer.Context) - Method in class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb.OutlinkDbReducer
  Configures the OutlinkDb job reducer.
setup(Reducer.Context) - Method in class org.apache.nutch.segment.SegmentMerger.SegmentMergerReducer
setUrl(String) - Method in class org.apache.nutch.fetcher.FetcherThreadEvent
  Set URL of this event (fetched page)
setUrl(String) - Method in class org.apache.nutch.parse.Outlink
setUrl(String) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
setUrl(String) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
setURLScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.similarity.cosine.CosineSimilarity
setURLScoreAfterParsing(Text, Content, Parse) - Method in interface org.apache.nutch.scoring.similarity.SimilarityModel
setVectorEntry(int, long) - Method in class org.apache.nutch.scoring.similarity.cosine.DocVector
setWaitForExit(boolean) - Method in class org.apache.nutch.util.CommandRunner
setWarcSize(long) - Method in class org.apache.nutch.tools.CommonCrawlConfig
setWeight(float) - Method in class org.apache.nutch.indexer.NutchDocument
setWeight(float) - Method in class org.apache.nutch.indexer.NutchField
setWhiteList(String) - Method in class org.apache.nutch.collection.Subcollection
  Set contents of whitelist from String
setWhiteList(ArrayList<String>) - Method in class org.apache.nutch.collection.Subcollection
shortestMatch(String) - Method in class org.apache.nutch.util.PrefixStringMatcher
  Returns the shortest prefix of `input` that is matched, or `null` if no match exists.
shortestMatch(String) - Method in class org.apache.nutch.util.SuffixStringMatcher
  Returns the shortest suffix of `input` that is matched, or `null` if no match exists.
shortestMatch(String) - Method in class org.apache.nutch.util.TrieStringMatcher
  Returns the shortest substring of `input` that is matched by a pattern in the trie, or `null` if no match exists.
shouldCheck(HostDatum) - Method in class org.apache.nutch.hostdb.UpdateHostDbReducer
  Determines whether a record should be checked.
shouldFetch(Text, CrawlDatum, long) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
  This method provides information whether the page is suitable for selection in the current fetchlist.
shouldFetch(Text, CrawlDatum, long) - Method in interface org.apache.nutch.crawl.FetchSchedule
  This method provides information whether the page is suitable for selection in the current fetchlist.
shouldProcessURL(String) - Method in class org.apache.nutch.protocol.interactiveselenium.handlers.DefalultMultiInteractionHandler
shouldProcessURL(String) - Method in class org.apache.nutch.protocol.interactiveselenium.handlers.DefaultClickAllAjaxLinksHandler
shouldProcessURL(String) - Method in class org.apache.nutch.protocol.interactiveselenium.handlers.DefaultHandler
shouldProcessURL(String) - Method in interface org.apache.nutch.protocol.interactiveselenium.handlers.InteractiveSeleniumHandler
ShowProperties - Class in org.apache.nutch.tools
  Tool to list properties and their values set by the current Nutch configuration
ShowProperties() - Constructor for class org.apache.nutch.tools.ShowProperties
shutDown() - Method in class org.apache.nutch.plugin.Plugin
  Shutdown the plugin.
Signature - Class in org.apache.nutch.crawl
Signature() - Constructor for class org.apache.nutch.crawl.Signature
SIGNATURE_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
SignatureComparator - Class in org.apache.nutch.crawl
SignatureComparator() - Constructor for class org.apache.nutch.crawl.SignatureComparator
SignatureFactory - Class in org.apache.nutch.crawl
  Factory class, which instantiates a Signature implementation according to the current Configuration configuration.
SimilarityModel - Interface in org.apache.nutch.scoring.similarity
SimilarityScoringFilter - Class in org.apache.nutch.scoring.similarity
SimilarityScoringFilter() - Constructor for class org.apache.nutch.scoring.similarity.SimilarityScoringFilter
simpleDateFormat - Variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
sitemap(Path, Path, Path, boolean, boolean, boolean, int) - Method in class org.apache.nutch.util.SitemapProcessor
SITEMAP_ALWAYS_TRY_SITEMAPXML_ON_ROOT - Static variable in class org.apache.nutch.util.SitemapProcessor
SITEMAP_EXISTING_ENTRIES_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Existing sitemap entries.
SITEMAP_FAILED_FETCHES_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Failed sitemap fetches.
SITEMAP_FILTERED_FROM_HOSTNAME_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Sitemaps filtered from hostname.
SITEMAP_FILTERED_RECORDS_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Filtered records in sitemap processing.
SITEMAP_FROM_HOSTNAME_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Sitemaps discovered from hostname.
SITEMAP_NEW_ENTRIES_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  New sitemap entries.
SITEMAP_OVERWRITE_EXISTING - Static variable in class org.apache.nutch.util.SitemapProcessor
SITEMAP_REDIR_MAX - Static variable in class org.apache.nutch.util.SitemapProcessor
SITEMAP_SEEDS_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Seeds extracted from sitemaps.
SITEMAP_SIZE_MAX - Static variable in class org.apache.nutch.util.SitemapProcessor
SITEMAP_STRICT_PARSING - Static variable in class org.apache.nutch.util.SitemapProcessor
SITEMAP_URL_FILTERING - Static variable in class org.apache.nutch.util.SitemapProcessor
SITEMAP_URL_NORMALIZING - Static variable in class org.apache.nutch.util.SitemapProcessor
SitemapProcessor - Class in org.apache.nutch.util
  Performs sitemap processing by fetching sitemap links, parsing the content and merging the URLs from sitemaps (with the metadata) into the CrawlDb.
SitemapProcessor() - Constructor for class org.apache.nutch.util.SitemapProcessor
size() - Method in class org.apache.nutch.crawl.Inlinks
size() - Method in class org.apache.nutch.metadata.Metadata
  Returns the number of metadata names in this metadata.
size() - Method in class org.apache.nutch.parse.ParseResult
  Return the number of parse outputs (both successful and failed)
skip(DataInput) - Static method in class org.apache.nutch.crawl.Inlink
  Skips over one Inlink in the input.
skip(DataInput) - Static method in class org.apache.nutch.parse.Outlink
  Skips over one Outlink in the input.
SKIP_TRUNCATED - Static variable in class org.apache.nutch.parse.ParseSegment
skipChildren() - Method in class org.apache.nutch.util.NodeWalker
  Skips over and removes from the node stack the children of the last node.
skippedEntity(String) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive notification of a skipped entity.
skippedNotEligibleCounter - Variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
SlashURLNormalizer - Class in org.apache.nutch.net.urlnormalizer.slash
  Configurable normalization of trailing slashes in URLs.
SlashURLNormalizer() - Constructor for class org.apache.nutch.net.urlnormalizer.slash.SlashURLNormalizer
SOFTWARE - Static variable in class org.apache.nutch.tools.WARCUtils
SolrConstants - Interface in org.apache.nutch.indexwriter.solr
SolrIndexWriter - Class in org.apache.nutch.indexwriter.solr
SolrIndexWriter() - Constructor for class org.apache.nutch.indexwriter.solr.SolrIndexWriter
SolrUtils - Class in org.apache.nutch.indexwriter.solr
SolrUtils() - Constructor for class org.apache.nutch.indexwriter.solr.SolrUtils
Sorter() - Constructor for class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter
SorterMapper() - Constructor for class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter.SorterMapper
SorterReducer() - Constructor for class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter.SorterReducer
SOURCE - Static variable in interface org.apache.nutch.metadata.DublinCore
  A reference to a resource from which the present resource is derived.
SpellCheckedMetadata - Class in org.apache.nutch.metadata
  A decorator to Metadata that adds spellchecking capabilities to property names.
SpellCheckedMetadata() - Constructor for class org.apache.nutch.metadata.SpellCheckedMetadata
splitEnd - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
splitLen - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
splitStart - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
STANDARD - Enum constant in enum class org.apache.nutch.scoring.similarity.util.LuceneTokenizer.TokenizerType
start - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
start(String) - Static method in class org.apache.nutch.parsefilter.naivebayes.Train
START - Enum constant in enum class org.apache.nutch.fetcher.FetcherThreadEvent.PublishEventType
startArray(String, boolean, boolean) - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
startArray(String, boolean, boolean) - Method in class org.apache.nutch.tools.CommonCrawlFormatJackson
startArray(String, boolean, boolean) - Method in class org.apache.nutch.tools.CommonCrawlFormatJettinson
startArray(String, boolean, boolean) - Method in class org.apache.nutch.tools.CommonCrawlFormatSimple
startArray(String, boolean, boolean) - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
startCDATA() - Method in class org.apache.nutch.parse.html.DOMBuilder
  Report the start of a CDATA section.
startDocument() - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive notification of the beginning of a document.
startDTD(String, String, String) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Report the start of DTD declarations, if any.
startElement(String, String, String, Attributes) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Receive notification of the beginning of an element.
startEntity(String) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Report the beginning of an entity.
startObject(String) - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
startObject(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatJackson
startObject(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatJettinson
startObject(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatSimple
startObject(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
startPrefixMapping(String, String) - Method in class org.apache.nutch.parse.html.DOMBuilder
  Begin the scope of a prefix-URI Namespace mapping.
startUp() - Method in class org.apache.nutch.plugin.Plugin
  Will be invoked until plugin start up.
STAT_PROGRESS - Static variable in interface org.apache.nutch.metadata.Nutch
  For progress of job (programmatic / tooling).
StaticFieldIndexer - Class in org.apache.nutch.indexer.staticfield
  A simple plugin called at indexing that adds fields with static data.
StaticFieldIndexer() - Constructor for class org.apache.nutch.indexer.staticfield.StaticFieldIndexer
statNames - Static variable in class org.apache.nutch.crawl.CrawlDatum
status - Variable in class org.apache.nutch.util.NutchTool
STATUS_BLOCKED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
STATUS_DB_DUPLICATE - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page was marked as being a duplicate of another page
STATUS_DB_FETCHED - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page was successfully fetched.
STATUS_DB_GONE - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page no longer exists.
STATUS_DB_MAX - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Maximum value of DB-related status.
STATUS_DB_NOTMODIFIED - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page was successfully fetched and found not modified.
STATUS_DB_ORPHAN - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page was marked as orphan, e.g.
STATUS_DB_PARSE_FAILED - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page parsing failed
STATUS_DB_REDIR_PERM - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page permanently redirects to other page.
STATUS_DB_REDIR_TEMP - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page temporarily redirects to other page.
STATUS_DB_UNFETCHED - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page was not fetched yet.
STATUS_FAILED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
STATUS_FAILURE - Static variable in class org.apache.nutch.parse.ParseStatus
STATUS_FETCH_GONE - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Fetching unsuccessful - page is gone.
STATUS_FETCH_MAX - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Maximum value of fetch-related status.
STATUS_FETCH_NOTMODIFIED - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Fetching successful - page is not modified.
STATUS_FETCH_REDIR_PERM - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Fetching permanently redirected to other page.
STATUS_FETCH_REDIR_TEMP - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Fetching temporarily redirected to other page.
STATUS_FETCH_RETRY - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Fetching unsuccessful, needs to be retried (transient errors).
STATUS_FETCH_SUCCESS - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Fetching was successful.
STATUS_GONE - Static variable in class org.apache.nutch.protocol.ProtocolStatus
STATUS_INJECTED - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page was newly injected.
STATUS_LINKED - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page discovered through a link.
STATUS_MODIFIED - Static variable in interface org.apache.nutch.crawl.FetchSchedule
  Page is known to have been modified since our last visit.
STATUS_NOTFETCHING - Static variable in class org.apache.nutch.protocol.ProtocolStatus
STATUS_NOTFOUND - Static variable in class org.apache.nutch.protocol.ProtocolStatus
STATUS_NOTMODIFIED - Static variable in interface org.apache.nutch.crawl.FetchSchedule
  Page is known to remain unmodified since our last visit.
STATUS_NOTMODIFIED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
STATUS_NOTPARSED - Static variable in class org.apache.nutch.parse.ParseStatus
STATUS_PARSE_FAILED - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page parse failed
STATUS_PARSE_META - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page got metadata from a parser
STATUS_REDIR_EXCEEDED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
STATUS_RETRY - Static variable in class org.apache.nutch.protocol.ProtocolStatus
STATUS_ROBOTS_DENIED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
STATUS_SIGNATURE - Static variable in class org.apache.nutch.crawl.CrawlDatum
  Page signature.
STATUS_SUCCESS - Static variable in class org.apache.nutch.parse.ParseStatus
STATUS_SUCCESS - Static variable in class org.apache.nutch.protocol.ProtocolStatus
STATUS_UNKNOWN - Static variable in interface org.apache.nutch.crawl.FetchSchedule
  It is unknown whether page was changed since our last visit.
STATUS_WOULDBLOCK - Static variable in class org.apache.nutch.protocol.ProtocolStatus
StatusUpdateReducer() - Constructor for class org.apache.nutch.crawl.DeduplicationJob.StatusUpdateReducer
stdin - Variable in class org.apache.nutch.util.AbstractChecker
stopJob() - Method in class org.apache.nutch.util.NutchTool
  Stop the job with the possibility to resume.
storeHttpHeaders - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Record the HTTP response header in the metadata, see property `store.http.headers`.
storeHttpRequest - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Record the HTTP request in the metadata, see property `store.http.request`.
storeIPAddress - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Record the IP address of the responding server, see property `store.ip.address`.
storeProtocolVersions - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Record the HTTP and SSL/TLS protocol versions and the SSL/TLS cipher suites, see property `store.protocol.versions`.
stringFields - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
stringFieldWritables - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
StringUtil - Class in org.apache.nutch.util
  A collection of String processing utility methods.
StringUtil() - Constructor for class org.apache.nutch.util.StringUtil
stripNonCharCodepoints(String) - Static method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchUtils
Subcollection - Class in org.apache.nutch.collection
  SubCollection represents a subset of index, you can define url patterns that will indicate that particular page (url) is part of SubCollection.
Subcollection(String, String, String, Configuration) - Constructor for class org.apache.nutch.collection.Subcollection
  public Constructor
Subcollection(String, String, Configuration) - Constructor for class org.apache.nutch.collection.Subcollection
  public Constructor
Subcollection(Configuration) - Constructor for class org.apache.nutch.collection.Subcollection
SubcollectionIndexingFilter - Class in org.apache.nutch.indexer.subcollection
SubcollectionIndexingFilter() - Constructor for class org.apache.nutch.indexer.subcollection.SubcollectionIndexingFilter
SubcollectionIndexingFilter(Configuration) - Constructor for class org.apache.nutch.indexer.subcollection.SubcollectionIndexingFilter
SUBJECT - Static variable in interface org.apache.nutch.metadata.DublinCore
  The topic of the content of the resource.
SUCCESS - Static variable in class org.apache.nutch.parse.ParseStatus
  Parsing succeeded.
SUCCESS - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Content was retrieved without errors.
SUCCESS_REDIRECT - Static variable in class org.apache.nutch.parse.ParseStatus
  Parsed content contains a directive to redirect to another URL.
SUFFIX_COUNT_TOTAL - Static variable in class org.apache.nutch.metrics.LatencyTracker
  Counter name suffix for total sample count.
SUFFIX_P50_MS - Static variable in class org.apache.nutch.metrics.LatencyTracker
  Counter name suffix for 50th percentile latency in milliseconds.
SUFFIX_P95_MS - Static variable in class org.apache.nutch.metrics.LatencyTracker
  Counter name suffix for 95th percentile latency in milliseconds.
SUFFIX_P99_MS - Static variable in class org.apache.nutch.metrics.LatencyTracker
  Counter name suffix for 99th percentile latency in milliseconds.
SUFFIX_SUM_MS - Static variable in class org.apache.nutch.metrics.LatencyTracker
  Counter name suffix for sum of latencies in milliseconds.
SuffixStringMatcher - Class in org.apache.nutch.util
  A class for efficiently matching `String`s against a set of suffixes.
SuffixStringMatcher(String[]) - Constructor for class org.apache.nutch.util.SuffixStringMatcher
  Creates a new `PrefixStringMatcher` which will match `String`s with any suffix in the supplied array.
SuffixStringMatcher(Collection<String>) - Constructor for class org.apache.nutch.util.SuffixStringMatcher
  Creates a new `PrefixStringMatcher` which will match `String`s with any suffix in the supplied `Collection`
SuffixURLFilter - Class in org.apache.nutch.urlfilter.suffix
  Filters URLs based on a file of URL suffixes.
SuffixURLFilter() - Constructor for class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
SuffixURLFilter(Reader) - Constructor for class org.apache.nutch.urlfilter.suffix.SuffixURLFilter
SYSTEM_PROTOCOLS - Static variable in class org.apache.nutch.plugin.URLStreamHandlerFactory
  Protocols covered by standard JVM URL handlers.

## T

TAB_CHARACTER - Static variable in class org.apache.nutch.crawl.Injector.InjectMapper
TableUtil - Class in org.apache.nutch.util
TableUtil() - Constructor for class org.apache.nutch.util.TableUtil
TAG_BLACKLIST - Static variable in class org.apache.nutch.collection.Subcollection
TAG_COLLECTION - Static variable in class org.apache.nutch.collection.Subcollection
TAG_COLLECTIONS - Static variable in class org.apache.nutch.collection.Subcollection
TAG_ID - Static variable in class org.apache.nutch.collection.Subcollection
TAG_KEY - Static variable in class org.apache.nutch.collection.Subcollection
TAG_NAME - Static variable in class org.apache.nutch.collection.Subcollection
TAG_WHITELIST - Static variable in class org.apache.nutch.collection.Subcollection
tcpPort - Variable in class org.apache.nutch.util.AbstractChecker
TEMP_MOVED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Resource has moved temporarily.
TEMPLATE - Static variable in class org.apache.nutch.tools.CommonCrawlFormatWARC
termFreqVector - Variable in class org.apache.nutch.scoring.similarity.cosine.DocVector
terminal - Variable in class org.apache.nutch.util.TrieStringMatcher.TrieNode
termVector - Variable in class org.apache.nutch.scoring.similarity.cosine.DocVector
TEXT_PLAIN_CONTENT_TYPE - Static variable in class org.apache.nutch.parse.feed.FeedParser
TextMD5Signature - Class in org.apache.nutch.crawl
  Implementation of a page signature.
TextMD5Signature() - Constructor for class org.apache.nutch.crawl.TextMD5Signature
TextOutputFormat() - Constructor for class org.apache.nutch.segment.SegmentReader.TextOutputFormat
TextProfileSignature - Class in org.apache.nutch.crawl
  An implementation of a page signature.
TextProfileSignature() - Constructor for class org.apache.nutch.crawl.TextProfileSignature
TikaParser - Class in org.apache.nutch.parse.tika
  Wrapper for Tika parsers.
TikaParser() - Constructor for class org.apache.nutch.parse.tika.TikaParser
TIME - Enum constant in enum class org.apache.nutch.net.protocols.Response.TruncatedContentReason
  fetch exceeded configured http.time.limit
timelimitExceeded() - Method in class org.apache.nutch.fetcher.FetchItemQueues
timeout - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The network timeout in millisecond
TIMEOUT - Enum constant in enum class org.apache.nutch.metrics.ErrorTracker.ErrorType
  Timeout errors (SocketTimeoutException)
TimingUtil - Class in org.apache.nutch.util
TimingUtil() - Constructor for class org.apache.nutch.util.TimingUtil
TITLE - Static variable in interface org.apache.nutch.metadata.DublinCore
  A name given to the resource.
TLDIndexingFilter - Class in org.apache.nutch.indexer.tld
  Adds the public suffix (aka.
TLDIndexingFilter() - Constructor for class org.apache.nutch.indexer.tld.TLDIndexingFilter
tlsCheckCertificate - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Whether to check TLS/SSL certificates
tlsPreferredCipherSuites - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Which TLS/SSL cipher suites to support
tlsPreferredProtocols - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Which TLS/SSL protocols to support
toASCII(String) - Static method in class org.apache.nutch.util.URLUtil
  Convert URL with IDN host/domain name into the ASCII representation.
toByteArray(HttpHeaders) - Static method in class org.apache.nutch.tools.WARCUtils
toBytes() - Method in class org.apache.nutch.metrics.LatencyTracker
  Serializes the digest to bytes for transmission to a reducer or side file.
toContent() - Method in class org.apache.nutch.protocol.file.FileResponse
toContent() - Method in class org.apache.nutch.protocol.ftp.FtpResponse
toDate(String) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
toHexString(byte[]) - Static method in class org.apache.nutch.util.StringUtil
  Convenience call for `StringUtil.toHexString(byte[], String, int)`, where `sep = null; lineLen = Integer.MAX_VALUE`.
toHexString(byte[], String, int) - Static method in class org.apache.nutch.util.StringUtil
  Get a text representation of a byte[] as hexadecimal String, where each pair of hexadecimal digits corresponds to consecutive bytes in the array.
toLong(String) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
TOPIC - Static variable in interface org.apache.nutch.indexwriter.kafka.KafkaConstants
toString() - Method in class org.apache.nutch.crawl.CrawlDatum
toString() - Method in class org.apache.nutch.crawl.Generator.SelectorEntry
toString() - Method in class org.apache.nutch.crawl.Inlink
toString() - Method in class org.apache.nutch.crawl.Inlinks
toString() - Method in class org.apache.nutch.hostdb.HostDatum
toString() - Method in class org.apache.nutch.indexer.IndexWriterConfig
toString() - Method in class org.apache.nutch.indexer.NutchDocument
toString() - Method in class org.apache.nutch.indexer.NutchField
toString() - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter.Separator
toString() - Method in class org.apache.nutch.metadata.Metadata
toString() - Method in class org.apache.nutch.parse.html.DOMContentUtils.LinkParams
toString() - Method in class org.apache.nutch.parse.HTMLMetaTags
toString() - Method in class org.apache.nutch.parse.Outlink
toString() - Method in class org.apache.nutch.parse.ParseData
toString() - Method in class org.apache.nutch.parse.ParseStatus
toString() - Method in class org.apache.nutch.parse.ParseText
toString() - Method in class org.apache.nutch.plugin.Extension
toString() - Method in class org.apache.nutch.protocol.Content
toString() - Method in class org.apache.nutch.protocol.okhttp.CIDR
toString() - Method in class org.apache.nutch.protocol.ProtocolStatus
toString() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
toString() - Method in class org.apache.nutch.scoring.webgraph.Node
toString() - Method in class org.apache.nutch.segment.SegmentPart
  Return a String representation of this class, in the form "segmentName/partName".
toString() - Method in class org.apache.nutch.urlfilter.fast.FastURLFilter.Rule
toString(long) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
toString(CharSequence) - Static method in class org.apache.nutch.util.TableUtil
  Convert given Utf8 instance to String and and cleans out any offending "�" from the String.
toString(String) - Method in class org.apache.nutch.protocol.Content
toString(String, String) - Method in class org.apache.nutch.metadata.Metadata
toString(Charset) - Method in class org.apache.nutch.protocol.Content
toString(Calendar) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
toString(Date) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
  Get the HTTP format of the specified date.
totalHostsCounter - Variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
toUNICODE(String) - Static method in class org.apache.nutch.util.URLUtil
  Convert URL with IDN host/domain name to the Unicode representation.
toZonedDateTime(String) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
train() - Method in class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
Train - Class in org.apache.nutch.parsefilter.naivebayes
Train() - Constructor for class org.apache.nutch.parsefilter.naivebayes.Train
TRAINFILE_MODELFILTER - Static variable in class org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter
TRANSFER_ENCODING - Static variable in interface org.apache.nutch.metadata.HttpHeaders
TrieStringMatcher - Class in org.apache.nutch.util
  TrieStringMatcher is a base class for simple tree-based string matching.
TrieStringMatcher() - Constructor for class org.apache.nutch.util.TrieStringMatcher
TrieStringMatcher.TrieNode - Class in org.apache.nutch.util
  Node class for the character tree.
TRUNCATED_CONTENT - Static variable in interface org.apache.nutch.net.protocols.Response
  Key to hold boolean whether content has been truncated, e.g., because it exceeds `http.content.limit`
TRUNCATED_CONTENT_REASON - Static variable in interface org.apache.nutch.net.protocols.Response
  Key to hold reason why content has been truncated, see `Response.TruncatedContentReason`
TruncatedContent() - Constructor for class org.apache.nutch.protocol.okhttp.OkHttpResponse.TruncatedContent
TRUST_STORE_PASSWORD - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
TRUST_STORE_PATH - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
TRUST_STORE_TYPE - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
TYPE - Static variable in interface org.apache.nutch.metadata.DublinCore
  The nature or genre of the content of the resource.

## U

unescape(String) - Method in class org.apache.nutch.net.urlnormalizer.ajax.AjaxURLNormalizer
  Unescape some exotic characters in the fragment part
unfetched - Variable in class org.apache.nutch.hostdb.HostDatum
unflattenToHashmap(String) - Static method in class org.apache.nutch.parsefilter.naivebayes.Classify
unreverseHost(String) - Static method in class org.apache.nutch.util.TableUtil
unreverseUrl(String) - Static method in class org.apache.nutch.util.TableUtil
UNSPECIFIED - Enum constant in enum class org.apache.nutch.net.protocols.Response.TruncatedContentReason
  unknown reason
unzip(byte[]) - Static method in class org.apache.nutch.util.GZIPUtils
  Returns an gunzipped copy of the input array.
unzipBestEffort(byte[]) - Static method in class org.apache.nutch.util.GZIPUtils
  Returns an gunzipped copy of the input array.
unzipBestEffort(byte[], int) - Static method in class org.apache.nutch.util.GZIPUtils
  Returns an gunzipped copy of the input array, truncated to `sizeLimit` bytes, if necessary.
update(Path, Path) - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater
  Updates the inlink score in the web graph node databsae into the crawl database.
update(Path, Path[], boolean, boolean) - Method in class org.apache.nutch.crawl.CrawlDb
update(Path, Path[], boolean, boolean, boolean, boolean) - Method in class org.apache.nutch.crawl.CrawlDb
update(NutchDocument) - Method in interface org.apache.nutch.indexer.IndexWriter
update(NutchDocument) - Method in class org.apache.nutch.indexer.IndexWriters
update(NutchDocument) - Method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
update(NutchDocument) - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
update(NutchDocument) - Method in class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
update(NutchDocument) - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
update(NutchDocument) - Method in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
update(NutchDocument) - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
update(NutchDocument) - Method in class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
update(NutchDocument) - Method in class org.apache.nutch.indexwriter.solr.SolrIndexWriter
UPDATE - Static variable in class org.apache.nutch.indexer.NutchIndexAction
updateDbScore(Text, CrawlDatum, CrawlDatum, List<CrawlDatum>) - Method in class org.apache.nutch.scoring.AbstractScoringFilter
updateDbScore(Text, CrawlDatum, CrawlDatum, List<CrawlDatum>) - Method in class org.apache.nutch.scoring.depth.DepthScoringFilter
updateDbScore(Text, CrawlDatum, CrawlDatum, List<CrawlDatum>) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
  Increase the score by a sum of inlinked scores.
updateDbScore(Text, CrawlDatum, CrawlDatum, List<CrawlDatum>) - Method in class org.apache.nutch.scoring.orphan.OrphanScoringFilter
  Used for orphan control.
updateDbScore(Text, CrawlDatum, CrawlDatum, List<CrawlDatum>) - Method in interface org.apache.nutch.scoring.ScoringFilter
  This method calculates a new score of CrawlDatum during CrawlDb update, based on the initial value of the original CrawlDatum, and also score values contributed by inlinked pages.
updateDbScore(Text, CrawlDatum, CrawlDatum, List<CrawlDatum>) - Method in class org.apache.nutch.scoring.ScoringFilters
  Calculate updated page score during CrawlDb.update().
updateHashMap(HashMap<String, Integer>, String) - Static method in class org.apache.nutch.parsefilter.naivebayes.Train
UpdateHostDb - Class in org.apache.nutch.hostdb
  Tool to create a HostDB from the CrawlDB.
UpdateHostDb() - Constructor for class org.apache.nutch.hostdb.UpdateHostDb
UpdateHostDbMapper - Class in org.apache.nutch.hostdb
  Mapper ingesting HostDB and CrawlDB entries.
UpdateHostDbMapper() - Constructor for class org.apache.nutch.hostdb.UpdateHostDbMapper
UpdateHostDbReducer - Class in org.apache.nutch.hostdb
UpdateHostDbReducer() - Constructor for class org.apache.nutch.hostdb.UpdateHostDbReducer
url - Variable in class org.apache.nutch.crawl.Generator.SelectorEntry
url - Variable in class org.apache.nutch.tools.AbstractCommonCrawlFormat
URL - Enum constant in enum class org.apache.nutch.metrics.ErrorTracker.ErrorType
  URL-related errors (MalformedURLException, URLFilterException)
URL_FILTER_NORMALIZE_ALL - Static variable in class org.apache.nutch.crawl.Injector
  property to pass value of command-line option -filterNormalizeAll to mapper
URL_FILTERING - Static variable in class org.apache.nutch.crawl.CrawlDbFilter
URL_FILTERING - Static variable in class org.apache.nutch.crawl.LinkDbFilter
URL_FILTERING - Static variable in class org.apache.nutch.indexer.IndexerMapReduce
URL_FILTERING - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb
URL_NORMALIZING - Static variable in class org.apache.nutch.crawl.CrawlDbFilter
URL_NORMALIZING - Static variable in class org.apache.nutch.crawl.LinkDbFilter
URL_NORMALIZING - Static variable in class org.apache.nutch.indexer.IndexerMapReduce
URL_NORMALIZING - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb
URL_NORMALIZING_SCOPE - Static variable in class org.apache.nutch.crawl.CrawlDbFilter
URL_NORMALIZING_SCOPE - Static variable in class org.apache.nutch.crawl.Injector.InjectMapper
URL_NORMALIZING_SCOPE - Static variable in class org.apache.nutch.crawl.LinkDbFilter
URL_VERSION - Static variable in class org.apache.nutch.tools.arc.ArcSegmentCreator.ArcSegmentCreatorMapper
URL_VERSION - Static variable in class org.apache.nutch.tools.arc.ArcSegmentCreator
URLExemptionFilter - Interface in org.apache.nutch.net
  Interface used to allow exemptions to external domain resources by overriding `db.ignore.external.links`.
URLExemptionFilters - Class in org.apache.nutch.net
  Creates and caches `URLExemptionFilter` implementing plugins.
URLExemptionFilters(Configuration) - Constructor for class org.apache.nutch.net.URLExemptionFilters
URLFilter - Interface in org.apache.nutch.net
  Interface used to limit which URLs enter Nutch.
URLFILTER_AUTOMATON_FILE - Static variable in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
URLFILTER_AUTOMATON_RULES - Static variable in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
URLFILTER_FAST_FILE - Static variable in class org.apache.nutch.urlfilter.fast.FastURLFilter
URLFILTER_FAST_MAX_LENGTH - Static variable in class org.apache.nutch.urlfilter.fast.FastURLFilter
URLFILTER_FAST_PATH_MAX_LENGTH - Static variable in class org.apache.nutch.urlfilter.fast.FastURLFilter
URLFILTER_FAST_QUERY_MAX_LENGTH - Static variable in class org.apache.nutch.urlfilter.fast.FastURLFilter
URLFILTER_ORDER - Static variable in class org.apache.nutch.net.URLFilters
URLFILTER_REGEX_FILE - Static variable in class org.apache.nutch.urlfilter.regex.RegexURLFilter
URLFILTER_REGEX_RULES - Static variable in class org.apache.nutch.urlfilter.regex.RegexURLFilter
URLFilterChecker - Class in org.apache.nutch.net
  Checks one given filter or all filters.
URLFilterChecker() - Constructor for class org.apache.nutch.net.URLFilterChecker
URLFilterException - Exception in org.apache.nutch.net
URLFilterException() - Constructor for exception org.apache.nutch.net.URLFilterException
URLFilterException(String) - Constructor for exception org.apache.nutch.net.URLFilterException
URLFilterException(String, Throwable) - Constructor for exception org.apache.nutch.net.URLFilterException
URLFilterException(Throwable) - Constructor for exception org.apache.nutch.net.URLFilterException
URLFilters - Class in org.apache.nutch.net
  Creates and caches plugins implementing `URLFilter`.
URLFilters(Configuration) - Constructor for class org.apache.nutch.net.URLFilters
urlKey - Static variable in class org.apache.nutch.crawl.DeduplicationJob
urlLimit - Static variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
urlLimitNotReachedCounter - Variable in class org.apache.nutch.hostdb.UpdateHostDbReducer
URLMetaIndexingFilter - Class in org.apache.nutch.indexer.urlmeta
  This is part of the URL Meta plugin.
URLMetaIndexingFilter() - Constructor for class org.apache.nutch.indexer.urlmeta.URLMetaIndexingFilter
URLMetaScoringFilter - Class in org.apache.nutch.scoring.urlmeta
  For documentation: `org.apache.nutch.scoring.urlmeta`
URLMetaScoringFilter() - Constructor for class org.apache.nutch.scoring.urlmeta.URLMetaScoringFilter
URLNormalizer - Interface in org.apache.nutch.net
  Interface used to convert URLs to normal form and optionally perform substitutions
URLNormalizerChecker - Class in org.apache.nutch.net
  Checks one given normalizer or all normalizers.
URLNormalizerChecker() - Constructor for class org.apache.nutch.net.URLNormalizerChecker
URLNormalizers - Class in org.apache.nutch.net
  This class uses a "chained filter" pattern to run defined normalizers.
URLNormalizers(Configuration, String) - Constructor for class org.apache.nutch.net.URLNormalizers
URLPartitioner - Class in org.apache.nutch.crawl
  Partition urls by host, domain name or IP depending on the value of the parameter 'partition.url.mode' which can be 'byHost', 'byDomain' or 'byIP'
URLPartitioner() - Constructor for class org.apache.nutch.crawl.URLPartitioner
URLStreamHandlerFactory - Class in org.apache.nutch.plugin
  This URLStreamHandlerFactory knows about all the plugins in use and thus can create the correct URLStreamHandler even if it comes from a plugin classpath.
URLUtil - Class in org.apache.nutch.util
  Utility class for URL analysis
URLUtil() - Constructor for class org.apache.nutch.util.URLUtil
UrlValidator - Class in org.apache.nutch.urlfilter.validator
  Validates URLs.
UrlValidator() - Constructor for class org.apache.nutch.urlfilter.validator.UrlValidator
usage - Variable in class org.apache.nutch.util.AbstractChecker
usage() - Method in class org.apache.nutch.crawl.Injector
usage() - Static method in class org.apache.nutch.util.SitemapProcessor
USE_AUTH - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
USE_AUTH - Static variable in interface org.apache.nutch.indexwriter.solr.SolrConstants
useHttp11 - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Do we use HTTP/1.1?
useHttp2 - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Whether to use HTTP/2
useProxy - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  Indicates if a proxy is used
useProxy(String) - Method in class org.apache.nutch.protocol.http.api.HttpBase
useProxy(URI) - Method in class org.apache.nutch.protocol.http.api.HttpBase
useProxy(URL) - Method in class org.apache.nutch.protocol.http.api.HttpBase
USER - Static variable in interface org.apache.nutch.indexwriter.elastic.ElasticConstants
USER - Static variable in interface org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xConstants
USER_AGENT - Static variable in interface org.apache.nutch.metadata.HttpHeaders
userAgent - Variable in class org.apache.nutch.protocol.http.api.HttpBase
  The Nutch 'User-Agent' request header
USERNAME - Static variable in interface org.apache.nutch.indexwriter.solr.SolrConstants
UTF_8 - Static variable in class org.apache.nutch.crawl.DeduplicationJob
UUID_KEY - Static variable in class org.apache.nutch.util.NutchConfiguration

## V

VAL_RESULT - Static variable in interface org.apache.nutch.metadata.Nutch
  Name of the key used in the result map from `NutchTool.run(java.util.Map<java.lang.String, java.lang.Object>, java.lang.String)`.
VALUE_SERIALIZER - Static variable in interface org.apache.nutch.indexwriter.kafka.KafkaConstants
valueOf(String) - Static method in enum class org.apache.nutch.fetcher.FetcherThreadEvent.PublishEventType
  Returns the enum constant of this class with the specified name.
valueOf(String) - Static method in enum class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter.DatabaseType
  Returns the enum constant of this class with the specified name.
valueOf(String) - Static method in enum class org.apache.nutch.metrics.ErrorTracker.ErrorType
  Returns the enum constant of this class with the specified name.
valueOf(String) - Static method in enum class org.apache.nutch.net.protocols.Response.TruncatedContentReason
  Returns the enum constant of this class with the specified name.
valueOf(String) - Static method in enum class org.apache.nutch.protocol.htmlunit.HttpResponse.Scheme
  Returns the enum constant of this class with the specified name.
valueOf(String) - Static method in enum class org.apache.nutch.protocol.http.HttpResponse.Scheme
  Returns the enum constant of this class with the specified name.
valueOf(String) - Static method in enum class org.apache.nutch.protocol.interactiveselenium.HttpResponse.Scheme
  Returns the enum constant of this class with the specified name.
valueOf(String) - Static method in enum class org.apache.nutch.protocol.selenium.HttpResponse.Scheme
  Returns the enum constant of this class with the specified name.
valueOf(String) - Static method in enum class org.apache.nutch.scoring.similarity.util.LuceneAnalyzerUtil.StemFilterType
  Returns the enum constant of this class with the specified name.
valueOf(String) - Static method in enum class org.apache.nutch.scoring.similarity.util.LuceneTokenizer.TokenizerType
  Returns the enum constant of this class with the specified name.
values() - Static method in enum class org.apache.nutch.fetcher.FetcherThreadEvent.PublishEventType
  Returns an array containing the constants of this enum class, in the order they are declared.
values() - Static method in enum class org.apache.nutch.indexer.geoip.GeoIPIndexingFilter.DatabaseType
  Returns an array containing the constants of this enum class, in the order they are declared.
values() - Static method in enum class org.apache.nutch.metrics.ErrorTracker.ErrorType
  Returns an array containing the constants of this enum class, in the order they are declared.
values() - Static method in enum class org.apache.nutch.net.protocols.Response.TruncatedContentReason
  Returns an array containing the constants of this enum class, in the order they are declared.
values() - Static method in enum class org.apache.nutch.protocol.htmlunit.HttpResponse.Scheme
  Returns an array containing the constants of this enum class, in the order they are declared.
values() - Static method in enum class org.apache.nutch.protocol.http.HttpResponse.Scheme
  Returns an array containing the constants of this enum class, in the order they are declared.
values() - Static method in enum class org.apache.nutch.protocol.interactiveselenium.HttpResponse.Scheme
  Returns an array containing the constants of this enum class, in the order they are declared.
values() - Static method in enum class org.apache.nutch.protocol.selenium.HttpResponse.Scheme
  Returns an array containing the constants of this enum class, in the order they are declared.
values() - Static method in enum class org.apache.nutch.scoring.similarity.util.LuceneAnalyzerUtil.StemFilterType
  Returns an array containing the constants of this enum class, in the order they are declared.
values() - Static method in enum class org.apache.nutch.scoring.similarity.util.LuceneTokenizer.TokenizerType
  Returns an array containing the constants of this enum class, in the order they are declared.
VERSION - Static variable in class org.apache.nutch.indexer.NutchDocument

## W

walk(Node, URL, Metadata, Configuration) - Static method in class org.creativecommons.nutch.CCParseFilter.Walker
  Scan the document adding attributes to metadata.
WARC_MISSING_CONTENT_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Missing content in WARC export.
WARC_MISSING_METADATA_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Missing metadata in WARC export.
WARC_OMITTED_EMPTY_RESPONSE_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Omitted empty responses in WARC export.
WARC_RECORDS_GENERATED_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  WARC records generated.
WARCExporter - Class in org.apache.nutch.tools.warc
  MapReduce job to exports Nutch segments as WARC files.
WARCExporter() - Constructor for class org.apache.nutch.tools.warc.WARCExporter
WARCExporter(Configuration) - Constructor for class org.apache.nutch.tools.warc.WARCExporter
WARCExporter.WARCMapReduce - Class in org.apache.nutch.tools.warc
WARCExporter.WARCMapReduce.WARCMapper - Class in org.apache.nutch.tools.warc
WARCExporter.WARCMapReduce.WARCReducer - Class in org.apache.nutch.tools.warc
WARCMapper() - Constructor for class org.apache.nutch.tools.warc.WARCExporter.WARCMapReduce.WARCMapper
WARCMapReduce() - Constructor for class org.apache.nutch.tools.warc.WARCExporter.WARCMapReduce
WARCReducer() - Constructor for class org.apache.nutch.tools.warc.WARCExporter.WARCMapReduce.WARCReducer
WARCUtils - Class in org.apache.nutch.tools
WARCUtils() - Constructor for class org.apache.nutch.tools.WARCUtils
WebGraph - Class in org.apache.nutch.scoring.webgraph
  Creates three databases, one for inlinks, one for outlinks, and a node database that holds the number of in and outlinks to a url and the current score for the url.
WebGraph() - Constructor for class org.apache.nutch.scoring.webgraph.WebGraph
WEBGRAPH_ADDED_LINKS_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Links added to WebGraph.
WEBGRAPH_REMOVED_LINKS_TOTAL - Static variable in class org.apache.nutch.metrics.NutchMetrics
  Links removed from WebGraph.
WebGraph.OutlinkDb - Class in org.apache.nutch.scoring.webgraph
  The OutlinkDb creates a database of all outlinks.
WebGraph.OutlinkDb.OutlinkDbMapper - Class in org.apache.nutch.scoring.webgraph
  Passes through existing LinkDatum objects from an existing OutlinkDb and maps out new LinkDatum objects from new crawls ParseData.
WebGraph.OutlinkDb.OutlinkDbReducer - Class in org.apache.nutch.scoring.webgraph
webWindowClosed(WebWindowEvent) - Method in class org.apache.nutch.protocol.htmlunit.HtmlUnitWebWindowListener
webWindowContentChanged(WebWindowEvent) - Method in class org.apache.nutch.protocol.htmlunit.HtmlUnitWebWindowListener
webWindowOpened(WebWindowEvent) - Method in class org.apache.nutch.protocol.htmlunit.HtmlUnitWebWindowListener
WEIGHT_FIELD - Static variable in interface org.apache.nutch.indexwriter.solr.SolrConstants
whitespacePattern - Static variable in class org.apache.nutch.parse.headings.HeadingsParseFilter
  Pattern used to strip surpluss whitespace
WORK_TYPE - Static variable in interface org.apache.nutch.metadata.CreativeCommons
WOULDBLOCK - Static variable in class org.apache.nutch.protocol.ProtocolStatus
  Deprecated.
WRITABLE_CONTENT_TYPE - Static variable in interface org.apache.nutch.metadata.HttpHeaders
WRITABLE_FIXED_INTERVAL_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
WRITABLE_GENERATE_TIME_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
WRITABLE_PROTO_STATUS_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
WRITABLE_REPR_URL_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
WritableSerializer() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumJsonOutputFormat.WritableSerializer
write(DataOutput) - Method in class org.apache.nutch.crawl.CrawlDatum
write(DataOutput) - Method in class org.apache.nutch.crawl.Generator.SelectorEntry
write(DataOutput) - Method in class org.apache.nutch.crawl.Inlink
write(DataOutput) - Method in class org.apache.nutch.crawl.Inlinks
write(DataOutput) - Method in class org.apache.nutch.hostdb.HostDatum
write(DataOutput) - Method in class org.apache.nutch.indexer.NutchDocument
write(DataOutput) - Method in class org.apache.nutch.indexer.NutchField
write(DataOutput) - Method in class org.apache.nutch.indexer.NutchIndexAction
write(DataOutput) - Method in class org.apache.nutch.metadata.Metadata
write(DataOutput) - Method in class org.apache.nutch.metadata.MetaWrapper
write(DataOutput) - Method in class org.apache.nutch.parse.Outlink
write(DataOutput) - Method in class org.apache.nutch.parse.ParseData
write(DataOutput) - Method in class org.apache.nutch.parse.ParseImpl
write(DataOutput) - Method in class org.apache.nutch.parse.ParseStatus
write(DataOutput) - Method in class org.apache.nutch.parse.ParseText
write(DataOutput) - Method in class org.apache.nutch.protocol.Content
write(DataOutput) - Method in class org.apache.nutch.protocol.ProtocolStatus
write(DataOutput) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
write(DataOutput) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
write(DataOutput) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
write(DataOutput) - Method in class org.apache.nutch.scoring.webgraph.Node
write(Text, CrawlDatum) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumCsvOutputFormat.LineRecordWriter
write(Text, CrawlDatum) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumJsonOutputFormat.LineRecordWriter
write(NutchDocument) - Method in interface org.apache.nutch.indexer.IndexWriter
write(NutchDocument) - Method in class org.apache.nutch.indexer.IndexWriters
write(NutchDocument) - Method in class org.apache.nutch.indexwriter.cloudsearch.CloudSearchIndexWriter
write(NutchDocument) - Method in class org.apache.nutch.indexwriter.csv.CSVIndexWriter
write(NutchDocument) - Method in class org.apache.nutch.indexwriter.dummy.DummyIndexWriter
write(NutchDocument) - Method in class org.apache.nutch.indexwriter.elastic.ElasticIndexWriter
write(NutchDocument) - Method in class org.apache.nutch.indexwriter.kafka.KafkaIndexWriter
write(NutchDocument) - Method in class org.apache.nutch.indexwriter.opensearch1x.OpenSearch1xIndexWriter
write(NutchDocument) - Method in class org.apache.nutch.indexwriter.rabbit.RabbitIndexWriter
write(NutchDocument) - Method in class org.apache.nutch.indexwriter.solr.SolrIndexWriter
writeArrayValue(String) - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
writeArrayValue(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatJackson
writeArrayValue(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatJettinson
writeArrayValue(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatSimple
writeArrayValue(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
writeKeyNull(String) - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
writeKeyNull(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatJackson
writeKeyNull(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatJettinson
writeKeyNull(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatSimple
writeKeyNull(String) - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
writeKeyValue(String, String) - Method in class org.apache.nutch.tools.AbstractCommonCrawlFormat
writeKeyValue(String, String) - Method in class org.apache.nutch.tools.CommonCrawlFormatJackson
writeKeyValue(String, String) - Method in class org.apache.nutch.tools.CommonCrawlFormatJettinson
writeKeyValue(String, String) - Method in class org.apache.nutch.tools.CommonCrawlFormatSimple
writeKeyValue(String, String) - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
writeObjectEntrySeparator(JsonGenerator) - Method in class org.apache.nutch.crawl.CrawlDbReader.JsonIndenter
writeObjectFieldValueSeparator(JsonGenerator) - Method in class org.apache.nutch.crawl.CrawlDbReader.JsonIndenter
writeOutAsDuplicate(CrawlDatum, Reducer.Context) - Method in class org.apache.nutch.crawl.DeduplicationJob.DedupReducer
writeRequest(URI) - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
writeResponse() - Method in class org.apache.nutch.tools.CommonCrawlFormatWARC
WWW_AUTHENTICATE - Static variable in class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
  The HTTP Authentication (WWW-Authenticate) header which is returned by a webserver requiring authentication.

## X

X_HIDE_HEADER - Static variable in class org.apache.nutch.tools.WARCUtils
X_POINT_ID - Static variable in interface org.apache.nutch.exchange.Exchange
  The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.indexer.IndexingFilter
  The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.indexer.IndexWriter
  The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.net.URLExemptionFilter
  The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.net.URLFilter
  The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.net.URLNormalizer
X_POINT_ID - Static variable in interface org.apache.nutch.parse.HtmlParseFilter
  The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.parse.Parser
  The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.protocol.Protocol
  The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.publisher.NutchPublisher
X_POINT_ID - Static variable in interface org.apache.nutch.scoring.ScoringFilter
  The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.segment.SegmentMergeFilter
  The name of the extension point.
XMLCharacterRecognizer - Class in org.apache.nutch.parse.html
  Class used to verify whether the specified ch conforms to the XML 1.0 definition of whitespace.
XMLCharacterRecognizer() - Constructor for class org.apache.nutch.parse.html.XMLCharacterRecognizer

## Y

YES_VAL - Static variable in class org.apache.nutch.util.TableUtil

## Z

zip(byte[]) - Static method in class org.apache.nutch.util.GZIPUtils
  Returns an gzipped copy of the input array.
ZipParser - Class in org.apache.nutch.parse.zip
  ZipParser class based on MSPowerPointParser class by Stephan Strittmatter.
ZipParser() - Constructor for class org.apache.nutch.parse.zip.ZipParser
  Creates a new instance of ZipParser
ZipTextExtractor - Class in org.apache.nutch.parse.zip
ZipTextExtractor(Configuration) - Constructor for class org.apache.nutch.parse.zip.ZipTextExtractor
  Creates a new instance of ZipTextExtractor

## _

__openPassiveDataConnection(int, String) - Method in class org.apache.nutch.protocol.ftp.Client
  Open a passive data connection socket
_compare(byte[], int, int, byte[], int, int) - Static method in class org.apache.nutch.crawl.SignatureComparator
_compare(Object, Object) - Static method in class org.apache.nutch.crawl.SignatureComparator

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form