Package org.apache.nutch.indexwriter.csv

Class CSVIndexWriter

java.lang.Object

org.apache.nutch.indexwriter.csv.CSVIndexWriter

All Implemented Interfaces:

Configurable, IndexWriter, Pluggable

public class CSVIndexWriter
extends Object
implements IndexWriter

Write Nutch documents to a CSV file (comma separated values), i.e., dump index as CSV or tab-separated plain text table. Format (encoding, separators, etc.) is configurable by a couple of options, see output of describe().

Note: works only in local mode, to be used with index option -noCommit.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	CSVIndexWriter.Separator	represent separators (also quote and escape characters) as char(s) and byte(s) in the output encoding for efficiency.

Field Summary

Fields

Modifier and Type	Field	Description
protected FSDataOutputStream	csvout
protected Charset	encoding	encoding of CSV file

Fields inherited from interface org.apache.nutch.indexer.IndexWriter

X_POINT_ID

Constructor Summary

Constructors

Constructor	Description
CSVIndexWriter()

Method Summary

Modifier and Type	Method	Description
void	close()
void	commit()	(nothing to commit)
void	delete(String key)	(deletion of documents is not supported)
Map<String,Map.Entry<String,Object>>	describe()	Returns Map with the specific parameters the IndexWriter instance can take.
Configuration	getConf()
static void	main(String[] args)
void	open(Configuration conf, String name)
void	open(IndexWriterParams parameters)	Initializes the internal variables from a given index writer configuration.
void	setConf(Configuration conf)
void	update(NutchDocument doc)
void	write(NutchDocument doc)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

encoding

protected Charset encoding

encoding of CSV file

csvout

protected FSDataOutputStream csvout

Constructor Details

CSVIndexWriter

public CSVIndexWriter()

Method Details

open

public void open(Configuration conf,
 String name)
          throws IOException

Specified by:

open in interface IndexWriter

Parameters:

conf - Nutch configuration

name - target name of the IndexWriter to be opened

Throws:

IOException - Some exception thrown by some writer.

open

public void open(IndexWriterParams parameters)
          throws IOException

Initializes the internal variables from a given index writer configuration.

Specified by:

open in interface IndexWriter

Parameters:

parameters - Params from the index writer configuration.

Throws:

IOException - Some exception thrown by writer.

write

public void write(NutchDocument doc)
           throws IOException

Specified by:

write in interface IndexWriter

Throws:

IOException

delete

public void delete(String key)

(deletion of documents is not supported)

Specified by:

delete in interface IndexWriter

update

public void update(NutchDocument doc)
            throws IOException

Specified by:

update in interface IndexWriter

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface IndexWriter

Throws:

IOException

commit

public void commit()

(nothing to commit)

Specified by:

commit in interface IndexWriter

getConf

public Configuration getConf()

Specified by:

getConf in interface Configurable

describe

public Map<String,Map.Entry<String,Object>> describe()

Returns Map with the specific parameters the IndexWriter instance can take.

Specified by:

describe in interface IndexWriter

Returns:

The values of each row. It must have the form <KEY,<DESCRIPTION,VALUE>>.

setConf

public void setConf(Configuration conf)

Specified by:

setConf in interface Configurable

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception