net.sourceforge.pmd.cpd

## Interface Tokenizer

All Known Implementing Classes:
  AbstractTokenizer, AnyTokenizer, CPPTokenizer, CsTokenizer, EcmascriptTokenizer, FortranTokenizer, JavaTokenizer, JSPTokenizer, PHPTokenizer, RubyTokenizer

    public interface Tokenizer

### Method Summary

Methods
`void` `tokenize(SourceCode sourceCode, Tokens tokenEntries)`

### Method Detail

#### tokenize

    void tokenize(SourceCode sourceCode,
                Tokens tokenEntries)
                  throws IOException

Throws:
  `IOException`