Overview | Package | Class | Use | Tree | Deprecated | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.apache.lucene.search.similarities
Class BM25Similarity

    java.lang.Object
      org.apache.lucene.search.similarities.Similarity
          org.apache.lucene.search.similarities.BM25Similarity

    public class BM25Similarity

extends Similarity

BM25 Similarity. Introduced in Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. Okapi at TREC-3. In Proceedings of the Third Text REtrieval Conference (TREC 1994). Gaithersburg, USA, November 1994.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.Similarity
`Similarity.ExactSimScorer, Similarity.SimWeight, Similarity.SloppySimScorer`

Field Summary
`protected boolean` | `discountOverlaps` True if overlap tokens (tokens with a position of increment of zero) are discounted from the document's length.

Constructor Summary
`BM25Similarity()` BM25 with these default values: `k1 = 1.2`, `b = 0.75`.
`BM25Similarity(float k1, float b)` BM25 with the supplied parameter values.

Method Summary
`protected float` | `avgFieldLength(CollectionStatistics collectionStats)` The default implementation computes the average as `sumTotalTermFreq / maxDoc`, or returns `1` if the index does not store sumTotalTermFreq (Lucene 3.x indexes or any field that omits frequency information).
`long` | `computeNorm(FieldInvertState state)` Computes the normalization value for a field, given the accumulated state of term processing for this field (see `FieldInvertState`).
`Similarity.SimWeight` | `computeWeight(float queryBoost, CollectionStatistics collectionStats, TermStatistics... termStats)` Compute any collection-level weight (e.g.
`protected float` | `decodeNormValue(byte b)` The default implementation returns `1 / f2` where `f` is `SmallFloat.byte315ToFloat(byte)`.
`protected byte` | `encodeNormValue(float boost, int fieldLength)` The default implementation encodes `boost / sqrt(length)` with `SmallFloat.floatToByte315(float)`.
`Similarity.ExactSimScorer` | `exactSimScorer(Similarity.SimWeight stats, AtomicReaderContext context)` Creates a new `Similarity.ExactSimScorer` to score matching documents from a segment of the inverted index.
`float` | `getB()` Returns the `b` parameter
`boolean` | `getDiscountOverlaps()` Returns true if overlap tokens are discounted from the document's length.
`float` | `getK1()` Returns the `k1` parameter
`protected float` | `idf(long docFreq, long numDocs)` Implemented as `log(1 + (numDocs - docFreq + 0.5)/(docFreq + 0.5))`.
`Explanation` | `idfExplain(CollectionStatistics collectionStats, TermStatistics termStats)` Computes a score factor for a simple term and returns an explanation for that score factor.
`Explanation` | `idfExplain(CollectionStatistics collectionStats, TermStatistics[] termStats)` Computes a score factor for a phrase.
`protected float` | `scorePayload(int doc, int start, int end, BytesRef payload)` The default implementation returns `1`
`void` | `setDiscountOverlaps(boolean v)` Sets whether overlap tokens (Tokens with 0 position increment) are ignored when computing norm.
`protected float` | `sloppyFreq(int distance)` Implemented as `1 / (distance + 1)`.
`Similarity.SloppySimScorer` | `sloppySimScorer(Similarity.SimWeight stats, AtomicReaderContext context)` Creates a new `Similarity.SloppySimScorer` to score matching documents from a segment of the inverted index.
`String` | `toString()`

Methods inherited from class org.apache.lucene.search.similarities.Similarity
`coord, queryNorm`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

### discountOverlaps

    protected boolean discountOverlaps

  True if overlap tokens (tokens with a position of increment of zero) are discounted from the document's length.

Constructor Detail

### BM25Similarity

    public BM25Similarity(float k1,
                          float b)

  BM25 with the supplied parameter values.

  Parameters:
    `k1` - Controls non-linear term frequency normalization (saturation).
    `b` - Controls to what degree document length normalizes tf values.

### BM25Similarity

    public BM25Similarity()

  BM25 with these default values:
  - `k1 = 1.2`,
  - `b = 0.75`.

Method Detail

### idf

    protected float idf(long docFreq,
                        long numDocs)

  Implemented as `log(1 + (numDocs - docFreq + 0.5)/(docFreq + 0.5))`.

### sloppyFreq

    protected float sloppyFreq(int distance)

  Implemented as `1 / (distance + 1)`.

### scorePayload

    protected float scorePayload(int doc,
                                 int start,
                                 int end,
                                 BytesRef payload)

  The default implementation returns `1`

### avgFieldLength

    protected float avgFieldLength(CollectionStatistics collectionStats)

  The default implementation computes the average as `sumTotalTermFreq / maxDoc`, or returns `1` if the index does not store sumTotalTermFreq (Lucene 3.x indexes or any field that omits frequency information).

### encodeNormValue

    protected byte encodeNormValue(float boost,
                                   int fieldLength)

  The default implementation encodes `boost / sqrt(length)` with `SmallFloat.floatToByte315(float)`. This is compatible with Lucene's default implementation. If you change this, then you should change `decodeNormValue(byte)` to match.

### decodeNormValue

    protected float decodeNormValue(byte b)

  The default implementation returns `1 / f2` where `f` is `SmallFloat.byte315ToFloat(byte)`.

### setDiscountOverlaps

    public void setDiscountOverlaps(boolean v)

  Sets whether overlap tokens (Tokens with 0 position increment) are ignored when computing norm. By default this is true, meaning overlap tokens do not count when computing norms.

### getDiscountOverlaps

    public boolean getDiscountOverlaps()

  Returns true if overlap tokens are discounted from the document's length.

  See Also:
    `setDiscountOverlaps(boolean)`

### computeNorm

    public final long computeNorm(FieldInvertState state)

  Description copied from class: `Similarity`
  Computes the normalization value for a field, given the accumulated state of term processing for this field (see `FieldInvertState`).

  Matches in longer fields are less precise, so implementations of this method usually set smaller values when `state.getLength()` is large, and larger values when `state.getLength()` is small.

  Specified by:
    `computeNorm` in class `Similarity`
  Parameters:
    `state` - current processing state for this field
  Returns:
    computed norm value

### idfExplain

    public Explanation idfExplain(CollectionStatistics collectionStats,
                                  TermStatistics termStats)

  Computes a score factor for a simple term and returns an explanation for that score factor.

  The default implementation uses:

       idf(docFreq, searcher.maxDoc());

  Note that `CollectionStatistics.maxDoc()` is used instead of `IndexReader#numDocs()` because also `TermStatistics.docFreq()` is used, and when the latter is inaccurate, so is `CollectionStatistics.maxDoc()`, and in the same direction. In addition, `CollectionStatistics.maxDoc()` is more efficient to compute

  Parameters:
    `collectionStats` - collection-level statistics
    `termStats` - term-level statistics for the term
  Returns:
    an Explain object that includes both an idf score factor and an explanation for the term.

### idfExplain

    public Explanation idfExplain(CollectionStatistics collectionStats,
                                  TermStatistics[] termStats)

  Computes a score factor for a phrase.

  The default implementation sums the idf factor for each term in the phrase.

  Parameters:
    `collectionStats` - collection-level statistics
    `termStats` - term-level statistics for the terms in the phrase
  Returns:
    an Explain object that includes both an idf score factor for the phrase and an explanation for each term.

### computeWeight

    public final Similarity.SimWeight computeWeight(float queryBoost,
                                                    CollectionStatistics collectionStats,
                                                    TermStatistics... termStats)

  Description copied from class: `Similarity`
  Compute any collection-level weight (e.g. IDF, average document length, etc) needed for scoring a query.

  Specified by:
    `computeWeight` in class `Similarity`
  Parameters:
    `queryBoost` - the query-time boost.
    `collectionStats` - collection-level statistics, such as the number of tokens in the collection.
    `termStats` - term-level statistics, such as the document frequency of a term across the collection.
  Returns:
    SimWeight object with the information this Similarity needs to score a query.

### exactSimScorer

    public final Similarity.ExactSimScorer exactSimScorer(Similarity.SimWeight stats,
                                                          AtomicReaderContext context)
                                                   throws IOException

  Description copied from class: `Similarity`
  Creates a new `Similarity.ExactSimScorer` to score matching documents from a segment of the inverted index.

  Specified by:
    `exactSimScorer` in class `Similarity`
  Parameters:
    `stats` - collection information from `Similarity.computeWeight(float, CollectionStatistics, TermStatistics...)`
    `context` - segment of the inverted index to be scored.
  Returns:
    ExactSimScorer for scoring documents across `context`
  Throws:
    `IOException` - if there is a low-level I/O error

### sloppySimScorer

    public final Similarity.SloppySimScorer sloppySimScorer(Similarity.SimWeight stats,
                                                            AtomicReaderContext context)
                                                     throws IOException

  Description copied from class: `Similarity`
  Creates a new `Similarity.SloppySimScorer` to score matching documents from a segment of the inverted index.

  Specified by:
    `sloppySimScorer` in class `Similarity`
  Parameters:
    `stats` - collection information from `Similarity.computeWeight(float, CollectionStatistics, TermStatistics...)`
    `context` - segment of the inverted index to be scored.
  Returns:
    SloppySimScorer for scoring documents across `context`
  Throws:
    `IOException` - if there is a low-level I/O error

### toString

    public String toString()

  Overrides:
    `toString` in class `Object`

### getK1

    public float getK1()

  Returns the `k1` parameter

  See Also:
    `BM25Similarity(float, float)`

### getB

    public float getB()

  Returns the `b` parameter

  See Also:
    `BM25Similarity(float, float)`

Overview | Package | Class | Use | Tree | Deprecated | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.