Package org.apache.nutch.indexwriter.kafka

Class KafkaIndexWriter

java.lang.Object

org.apache.nutch.indexwriter.kafka.KafkaIndexWriter

All Implemented Interfaces:

Configurable, IndexWriter, Pluggable

public class KafkaIndexWriter
extends Object
implements IndexWriter

Sends Nutch documents to a configured Kafka Cluster

Field Summary

Fields

Modifier and Type	Field	Description
static org.slf4j.Logger	LOG

Fields inherited from interface org.apache.nutch.indexer.IndexWriter

X_POINT_ID

Constructor Summary

Constructors

Constructor	Description
KafkaIndexWriter()

Method Summary

Modifier and Type	Method	Description
void	close()
void	commit()
void	delete(String key)
Map<String,Map.Entry<String,Object>>	describe()	Returns Map with the specific parameters the IndexWriter instance can take.
Configuration	getConf()
void	open(Configuration job, String name)
void	open(IndexWriterParams params)	Initializes the internal variables from a given index writer configuration.
void	setConf(Configuration conf)
void	update(NutchDocument doc)
void	write(NutchDocument doc)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOG

public static org.slf4j.Logger LOG

Constructor Details

KafkaIndexWriter

public KafkaIndexWriter()

Method Details

open

public void open(Configuration job,
 String name)
          throws IOException

Specified by:

open in interface IndexWriter

Parameters:

job - Nutch configuration

name - target name of the IndexWriter to be opened

Throws:

IOException - Some exception thrown by some writer.

open

public void open(IndexWriterParams params)
          throws IOException

Description copied from interface: IndexWriter

Initializes the internal variables from a given index writer configuration.

Specified by:

open in interface IndexWriter

Parameters:

params - Params from the index writer configuration.

Throws:

IOException - Some exception thrown by writer.

write

public void write(NutchDocument doc)
           throws IOException

Specified by:

write in interface IndexWriter

Throws:

IOException

delete

public void delete(String key)
            throws IOException

Specified by:

delete in interface IndexWriter

Throws:

IOException

update

public void update(NutchDocument doc)
            throws IOException

Specified by:

update in interface IndexWriter

Throws:

IOException

commit

public void commit()
            throws IOException

Specified by:

commit in interface IndexWriter

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface IndexWriter

Throws:

IOException

describe

public Map<String,Map.Entry<String,Object>> describe()

Description copied from interface: IndexWriter

Returns Map with the specific parameters the IndexWriter instance can take.

Specified by:

describe in interface IndexWriter

Returns:

The values of each row. It must have the form <KEY,<DESCRIPTION,VALUE>>.

setConf

public void setConf(Configuration conf)

Specified by:

setConf in interface Configurable

getConf

public Configuration getConf()

Specified by:

getConf in interface Configurable