During indexing, Net Search Extender processes document text in the following way, breaking the text up into tokens.