Package org.apache.nutch.parsefilter.naivebayes

# Class NaiveBayesParseFilter

java.lang.Object
org.apache.nutch.parsefilter.naivebayes.NaiveBayesParseFilter

All Implemented Interfaces:
  `Configurable`, `HtmlParseFilter`, `Pluggable`

public class NaiveBayesParseFilter extends Object implements HtmlParseFilter
Html Parse filter that classifies the outlinks from the parseresult as relevant or irrelevant based on the parseText's relevancy (using a training file where you can give positive and negative example texts see the description of parsefilter.naivebayes.trainfile) and if found irrelevant it gives the link a second chance if it contains any of the words from the list given in parsefilter.naivebayes.wordlist. CAUTION: Set the parser.timeout to -1 or a bigger value than 30, when using this classifier.

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final String`
  `DICTFILE_MODELFILTER`
  `static final String`
  `TRAINFILE_MODELFILTER`

  ### Fields inherited from interface org.apache.nutch.parse.HtmlParseFilter

  `X_POINT_ID`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `NaiveBayesParseFilter()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `boolean`
  `classify(String text)`
  `boolean`
  `containsWord(String url, ArrayList<String> wordlist)`
  `ParseResult`
  `filter(Content content, ParseResult parseResult, HTMLMetaTags metaTags, DocumentFragment doc)`
  Adds metadata or otherwise modifies a parse of HTML content, given the DOM tree of a page.
  `boolean`
  `filterParse(String text)`
  `boolean`
  `filterUrl(String url)`
  `Configuration`
  `getConf()`
  `void`
  `setConf(Configuration conf)`
  `void`
  `train()`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### TRAINFILE_MODELFILTER

    public static final String TRAINFILE_MODELFILTER
    See Also:
      - Constant Field Values

  - ### DICTFILE_MODELFILTER

    public static final String DICTFILE_MODELFILTER
    See Also:
      - Constant Field Values

- ## Constructor Details

  - ### NaiveBayesParseFilter

    public NaiveBayesParseFilter()

- ## Method Details

  - ### filterParse

    public boolean filterParse(String text)

  - ### filterUrl

    public boolean filterUrl(String url)

  - ### classify

    public boolean classify(String text) throws IOException
    Throws:
      `IOException`

  - ### train

    public void train() throws Exception
    Throws:
      `Exception`

  - ### containsWord

    public boolean containsWord(String url, ArrayList<String> wordlist)

  - ### setConf

    public void setConf(Configuration conf)
    Specified by:
      `setConf` in interface `Configurable`

  - ### getConf

    public Configuration getConf()
    Specified by:
      `getConf` in interface `Configurable`

  - ### filter

    public ParseResult filter(Content content, ParseResult parseResult, HTMLMetaTags metaTags, DocumentFragment doc)
    Description copied from interface: `HtmlParseFilter`
    Adds metadata or otherwise modifies a parse of HTML content, given the DOM tree of a page.
    Specified by:
      `filter` in interface `HtmlParseFilter`
    Parameters:
      `content` - the `Content` for a given response
      `parseResult` - the result of running on or more `Parser`'s on the content.
      `metaTags` - a populated `HTMLMetaTags` object
      `doc` - a `DocumentFragment` (DOM) which can be processed in the filtering process.
    Returns:
      a filtered `ParseResult`
    See Also:
      - `Parser.getParse(Content)`