Package org.apache.nutch.util

Class TableUtil

java.lang.Object
org.apache.nutch.util.TableUtil
public class TableUtil extends Object

  • Field Details

    • YES_VAL

      public static final ByteBuffer YES_VAL

  • Constructor Details

    • TableUtil

      public TableUtil()

  • Method Details

    • reverseUrl

      public static String reverseUrl(String urlString) throws MalformedURLException
      Reverses a url's domain. This form is better for storing in hbase. Because scans within the same domain are faster.

      E.g. "http://bar.foo.com:8983/to/index.html?a=b" becomes "com.foo.bar:8983:http/to/index.html?a=b".

      Parameters:
      urlString - url to be reversed
      Returns:
      Reversed url
      Throws:
      MalformedURLException - if the input urlString is malformed

    • reverseUrl

      public static String reverseUrl(URL url)
      Reverses a url's domain. This form is better for storing in hbase. Because scans within the same domain are faster.

      E.g. "http://bar.foo.com:8983/to/index.html?a=b" becomes "com.foo.bar:http:8983/to/index.html?a=b".

      Parameters:
      url - url to be reversed
      Returns:
      Reversed url

    • unreverseUrl

      public static String unreverseUrl(String reversedUrl)

    • getReversedHost

      public static String getReversedHost(String reversedUrl)
      Given a reversed url, returns the reversed host E.g "com.foo.bar:http:8983/to/index.html?a=b" -> "com.foo.bar"
      Parameters:
      reversedUrl - Reversed url
      Returns:
      Reversed host

    • reverseHost

      public static String reverseHost(String hostName)

    • unreverseHost

      public static String unreverseHost(String reversedHostName)

    • toString

      public static String toString(CharSequence utf8)
      Convert given Utf8 instance to String and and cleans out any offending "�" from the String.
      Parameters:
      utf8 - Utf8 object
      Returns:
      string-ifed Utf8 object or null if Utf8 instance is null