Package weka.core.tokenizers

package weka.core.tokenizers
  • Classes
    Class
    Description
    AlphabeticTokenizer
    Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.
    CharacterDelimitedTokenizer
    Abstract superclass for tokenizers that take characters as delimiters.
    CharacterNGramTokenizer
    Splits a string into an n-gram with min and max grams.
    NGramTokenizer
    Splits a string into an n-gram with min and max grams.
    Tokenizer
    A superclass for all tokenizer algorithms.
    WordTokenizer
    A simple tokenizer that is using the java.util.StringTokenizer class to tokenize the strings.