net.sourceforge.pmd.cpd

## Class AnyTokenizer

- java.lang.Object
- - net.sourceforge.pmd.cpd.AnyTokenizer

All Implemented Interfaces:
  Tokenizer

    public class AnyTokenizer
    extends Object
    implements Tokenizer

This class does a best-guess try-anything tokenization.

Author:
  jheintz

### Field Summary

Fields
`static String` `TOKENS`

### Constructor Summary

Constructors
Constructor and Description
`AnyTokenizer()`

### Method Summary

Methods
`void` `tokenize(SourceCode sourceCode, Tokens tokenEntries)`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### TOKENS

    public static final String TOKENS

See Also:
  Constant Field Values

### Constructor Detail

#### AnyTokenizer

    public AnyTokenizer()

### Method Detail

#### tokenize

    public void tokenize(SourceCode sourceCode,
                Tokens tokenEntries)

Specified by:
  `tokenize` in interface `Tokenizer`