org.apache.lucene.search.similarities

## Class BM25Similarity

- java.lang.Object
- - org.apache.lucene.search.similarities.Similarity
  - - org.apache.lucene.search.similarities.BM25Similarity

-     public class BM25Similarity
      extends Similarity

  BM25 Similarity. Introduced in Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. Okapi at TREC-3. In Proceedings of the Third Text REtrieval Conference (TREC 1994). Gaithersburg, USA, November 1994.
  WARNING: This API is experimental and might change in incompatible ways in the next release.

- - ### Nested Class Summary

    - ### Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.Similarity

      `Similarity.ExactSimScorer, Similarity.SimWeight, Similarity.SloppySimScorer`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected boolean` | `discountOverlaps` True if overlap tokens (tokens with a position of increment of zero) are discounted from the document's length.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `BM25Similarity()` BM25 with these default values: `k1 = 1.2`, `b = 0.75`.
    `BM25Similarity(float k1, float b)` BM25 with the supplied parameter values.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected float` | `avgFieldLength(CollectionStatistics collectionStats)` The default implementation computes the average as `sumTotalTermFreq / maxDoc`, or returns `1` if the index does not store sumTotalTermFreq (Lucene 3.x indexes or any field that omits frequency information).
    `long` | `computeNorm(FieldInvertState state)` Computes the normalization value for a field, given the accumulated state of term processing for this field (see `FieldInvertState`).
    `Similarity.SimWeight` | `computeWeight(float queryBoost, CollectionStatistics collectionStats, TermStatistics... termStats)` Compute any collection-level weight (e.g.
    `protected float` | `decodeNormValue(byte b)` The default implementation returns `1 / f2` where `f` is `SmallFloat.byte315ToFloat(byte)`.
    `protected byte` | `encodeNormValue(float boost, int fieldLength)` The default implementation encodes `boost / sqrt(length)` with `SmallFloat.floatToByte315(float)`.
    `Similarity.ExactSimScorer` | `exactSimScorer(Similarity.SimWeight stats, AtomicReaderContext context)` Creates a new `Similarity.ExactSimScorer` to score matching documents from a segment of the inverted index.
    `float` | `getB()` Returns the `b` parameter
    `boolean` | `getDiscountOverlaps()` Returns true if overlap tokens are discounted from the document's length.
    `float` | `getK1()` Returns the `k1` parameter
    `protected float` | `idf(long docFreq, long numDocs)` Implemented as `log(1 + (numDocs - docFreq + 0.5)/(docFreq + 0.5))`.
    `Explanation` | `idfExplain(CollectionStatistics collectionStats, TermStatistics termStats)` Computes a score factor for a simple term and returns an explanation for that score factor.
    `Explanation` | `idfExplain(CollectionStatistics collectionStats, TermStatistics[] termStats)` Computes a score factor for a phrase.
    `protected float` | `scorePayload(int doc, int start, int end, BytesRef payload)` The default implementation returns `1`
    `void` | `setDiscountOverlaps(boolean v)` Sets whether overlap tokens (Tokens with 0 position increment) are ignored when computing norm.
    `protected float` | `sloppyFreq(int distance)` Implemented as `1 / (distance + 1)`.
    `Similarity.SloppySimScorer` | `sloppySimScorer(Similarity.SimWeight stats, AtomicReaderContext context)` Creates a new `Similarity.SloppySimScorer` to score matching documents from a segment of the inverted index.
    `String` | `toString()`

    - ### Methods inherited from class org.apache.lucene.search.similarities.Similarity

      `coord, queryNorm`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### discountOverlaps

          protected boolean discountOverlaps

      True if overlap tokens (tokens with a position of increment of zero) are discounted from the document's length.

  - ### Constructor Detail

    - #### BM25Similarity

          public BM25Similarity(float k1,
                        float b)

      BM25 with the supplied parameter values.
      Parameters:
        `k1` - Controls non-linear term frequency normalization (saturation).
        `b` - Controls to what degree document length normalizes tf values.

    - #### BM25Similarity

          public BM25Similarity()

      BM25 with these default values:
      - `k1 = 1.2`,
      - `b = 0.75`.

  - ### Method Detail

    - #### idf

          protected float idf(long docFreq,
                  long numDocs)

      Implemented as `log(1 + (numDocs - docFreq + 0.5)/(docFreq + 0.5))`.

    - #### sloppyFreq

          protected float sloppyFreq(int distance)

      Implemented as `1 / (distance + 1)`.

    - #### scorePayload

          protected float scorePayload(int doc,
                           int start,
                           int end,
                           BytesRef payload)

      The default implementation returns `1`

    - #### avgFieldLength

          protected float avgFieldLength(CollectionStatistics collectionStats)

      The default implementation computes the average as `sumTotalTermFreq / maxDoc`, or returns `1` if the index does not store sumTotalTermFreq (Lucene 3.x indexes or any field that omits frequency information).

    - #### encodeNormValue

          protected byte encodeNormValue(float boost,
                             int fieldLength)

      The default implementation encodes `boost / sqrt(length)` with `SmallFloat.floatToByte315(float)`. This is compatible with Lucene's default implementation. If you change this, then you should change `decodeNormValue(byte)` to match.

    - #### decodeNormValue

          protected float decodeNormValue(byte b)

      The default implementation returns `1 / f2` where `f` is `SmallFloat.byte315ToFloat(byte)`.

    - #### setDiscountOverlaps

          public void setDiscountOverlaps(boolean v)

      Sets whether overlap tokens (Tokens with 0 position increment) are ignored when computing norm. By default this is true, meaning overlap tokens do not count when computing norms.

    - #### getDiscountOverlaps

          public boolean getDiscountOverlaps()

      Returns true if overlap tokens are discounted from the document's length.
      See Also:
        `setDiscountOverlaps(boolean)`

    - #### computeNorm

          public final long computeNorm(FieldInvertState state)

      Description copied from class: `Similarity`
      Computes the normalization value for a field, given the accumulated state of term processing for this field (see `FieldInvertState`).

      Matches in longer fields are less precise, so implementations of this method usually set smaller values when `state.getLength()` is large, and larger values when `state.getLength()` is small.

      Specified by:
        `computeNorm` in class `Similarity`
      Parameters:
        `state` - current processing state for this field
      Returns:
        computed norm value

    - #### idfExplain

          public Explanation idfExplain(CollectionStatistics collectionStats,
                               TermStatistics termStats)

      Computes a score factor for a simple term and returns an explanation for that score factor.

      The default implementation uses:

           idf(docFreq, searcher.maxDoc());

      Note that `CollectionStatistics.maxDoc()` is used instead of `IndexReader#numDocs()` because also `TermStatistics.docFreq()` is used, and when the latter is inaccurate, so is `CollectionStatistics.maxDoc()`, and in the same direction. In addition, `CollectionStatistics.maxDoc()` is more efficient to compute
      Parameters:
        `collectionStats` - collection-level statistics
        `termStats` - term-level statistics for the term
      Returns:
        an Explain object that includes both an idf score factor and an explanation for the term.

    - #### idfExplain

          public Explanation idfExplain(CollectionStatistics collectionStats,
                               TermStatistics[] termStats)

      Computes a score factor for a phrase.

      The default implementation sums the idf factor for each term in the phrase.

      Parameters:
        `collectionStats` - collection-level statistics
        `termStats` - term-level statistics for the terms in the phrase
      Returns:
        an Explain object that includes both an idf score factor for the phrase and an explanation for each term.

    - #### computeWeight

          public final Similarity.SimWeight computeWeight(float queryBoost,
                                           CollectionStatistics collectionStats,
                                           TermStatistics... termStats)

      Description copied from class: `Similarity`
      Compute any collection-level weight (e.g. IDF, average document length, etc) needed for scoring a query.
      Specified by:
        `computeWeight` in class `Similarity`
      Parameters:
        `queryBoost` - the query-time boost.
        `collectionStats` - collection-level statistics, such as the number of tokens in the collection.
        `termStats` - term-level statistics, such as the document frequency of a term across the collection.
      Returns:
        SimWeight object with the information this Similarity needs to score a query.

    - #### exactSimScorer

          public final Similarity.ExactSimScorer exactSimScorer(Similarity.SimWeight stats,
                                                 AtomicReaderContext context)
                                                         throws IOException

      Description copied from class: `Similarity`
      Creates a new `Similarity.ExactSimScorer` to score matching documents from a segment of the inverted index.
      Specified by:
        `exactSimScorer` in class `Similarity`
      Parameters:
        `stats` - collection information from `Similarity.computeWeight(float, CollectionStatistics, TermStatistics...)`
        `context` - segment of the inverted index to be scored.
      Returns:
        ExactSimScorer for scoring documents across `context`
      Throws:
        `IOException` - if there is a low-level I/O error

    - #### sloppySimScorer

          public final Similarity.SloppySimScorer sloppySimScorer(Similarity.SimWeight stats,
                                                   AtomicReaderContext context)
                                                           throws IOException

      Description copied from class: `Similarity`
      Creates a new `Similarity.SloppySimScorer` to score matching documents from a segment of the inverted index.
      Specified by:
        `sloppySimScorer` in class `Similarity`
      Parameters:
        `stats` - collection information from `Similarity.computeWeight(float, CollectionStatistics, TermStatistics...)`
        `context` - segment of the inverted index to be scored.
      Returns:
        SloppySimScorer for scoring documents across `context`
      Throws:
        `IOException` - if there is a low-level I/O error

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `Object`

    - #### getK1

          public float getK1()

      Returns the `k1` parameter
      See Also:
        `BM25Similarity(float, float)`

    - #### getB

          public float getB()

      Returns the `b` parameter
      See Also:
        `BM25Similarity(float, float)`

Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.