class RMMSeg::Ferret::Analyzer
Public Class Methods
new(&brk)
click to toggle source
Construct an Analyzer. Optional block can be used to add more +TokenFilter+s. e.g.
analyzer = RMMSeg::Ferret::Analyzer.new { |tokenizer| Ferret::Analysis::LowerCaseFilter.new(tokenizer) }
# File lib/rmmseg/ferret.rb, line 17 def initialize(&brk) @brk = brk end
Public Instance Methods
token_stream(field, text)
click to toggle source
# File lib/rmmseg/ferret.rb, line 21 def token_stream(field, text) t = Tokenizer.new(text) if @brk @brk.call(t) else t end end