Apache Solr DisMaxQParserPlugin

DisMaxQParserPlugin은 사용자가 입력한 간단한 문구들(어려운 문장이 없는)을 처리하거나 각각의 필드의 의미들에 기반한 다른 가중치(boosts)를 사용하는 여러개의 필드들의 단어 하나하나들의 검색을 위해 고안되었다. 추가 옵션으로 각 사용 사례(독립적인 사용자 입력)에 대한 특정 규칙에 기반한 점수에 영향을 줄 수 있게 되어 있다.

알림: Solr3.1 부터 문장부호 제어, 연관성 연산, 전체 질의 구문, 다른 필드간의 예명이 개선된 ExtendedDisMax 라는 더 진보된 분석기를 사용할 수 있다. 환경 설정 예제에에 edismax로 등록되어 있다.​

Contents

1. Overview
2. Query Syntax
3. Query Structure
4. Parameters
   1. q.alt
   2. qf (Query Fields)
   3. mm (Minimum 'Should' Match)
   4. pf (Phrase Fields)
   5. ps (Phrase Slop)
   6. qs (Query Phrase Slop)
   7. tie (Tie breaker)
   8. bq (Boost Query)
   9. bf (Boost Functions)
5. Examples

Overview

이 질의어 분석기는 Lucene 질의어 분석기 구문의 극도로 단순화 된 일부를 지원한다. 인용부호들은 구문들을 묶어주고, +/-는 필수 그리고 부가적인 경우들을 나타낼때 사용할 수 있다. 하지만 모든 다른 Lucene 질의어 분석기의 특수 문자들은 사용자가 간단히 이스케이프(escape​)함으로 바로 사용하면 된다. 핸들러(handler)는 당신이 지정한 필드들과  증폭(boost)들 간의 DisjunctionMax(dismax의 오타가 아닐까 싶음.)질의어들을 포함하는 Boolean ​질의어들을 사용한 사용자의 입력으로부터 멋진 질의어를 만드는 역활을 한다. 이것 또한, 증폭(boosting) 질의어들,  증폭 함수들 그리고 모든 검색들의 결과에 인위적으로 영향을 주는 필터 질의어들을 추가적으로 제공한다. 이 옵션들은 solrconfig.xml의 핸들러에 기본 값으로 설정을 할 수도 있고 Solr 질의어 URL에 같이 포함해서 사용 할 수도 있다.

Query Syntax

이것은 사용자가 입력​한 특수문자 이스케이프가 되지 않은 순수 입력 문자들을 지원하도록 설계되었다. '+'와 '-' 문자들은 하위 용어들에서 "필수"와 "금지"를 수정하는 용도로 취급된다. 인용부호 "로 감싼 텍스트는  구문들로 취급되고, 한개의 인용부호가 질의문에 포함되어 있으면 전혀 사용 안 된 것으로 동작한다.

지원하지 않던 구문의 예제:

• 와일드카드(Wildcards)
• 범위(Ranges)

숫자와 날짜 필드들에 대해선,  부적절한 질의 조건은 동작이 안됐다. 예를들어, "my_int"라는 숫자 필드가 있고 다음처럼 입력했다고 하자:
defType=dismax&qf=my_int textfield&q=bonkers 78
그러면 my_int의 숫자값 "78"이 아닌, 문자필드에서 "bonkers"와 "78"를 검색하는 질의에 대한 결과가 나오게 된다.

Query Structure

질의문의 각각의 "단어"에 사용하기 위해 dismax는 (적당한 가충치와 tie 매개변수에서 설정된 교차차단(tiebreaker) 값이 포함된) qf 매개 변수의 모든 필드들의 단어를 위해 DisjunctionMaxQuery 객체를 만든다. 이 DisjunctionMaxQuery 객체들은 mm 매개변수에 의해 설정되는 minNumberShouldMatch 선택값으로 BooleanQuery 안에 위치하게 된다. 만약 다른 매개변수들이 정의된다면, 더 큰 BooleanQuery는 qf 선택값의 첫번째 BooleanQuery로 감싸여지고, 다른 매개변수들(bf, bq, pf)은 부가항목으로 추가된다. pf 필드들 각각에서 전체 쿼리 '구문'을 포함하는 하나의 DisjuntionMaxQuery는, pf 매개변수에서 시작되어 한개의 복합항목에서 발생한다.

 :TODO:  Need more detail on the query structure generated based on input ... a picture would be nice.

Parameters

다음과 같은 매개변수들이 일반적인 요청 매개변수나 지역 매개변수 형태로 지원된다

 :TODO:  document which params are multivalue

q.alt

If specified, this query will be used (and parsed by default using standard query parsing syntax) when the main query string is not specified or blank. This comes in handy when you need something like a match-all-docs query (don't forget &rows=0 for that one!) in order to get collection-wise faceting counts.

qf (Query Fields)

List of fields and the "boosts" to associate with each of them when building DisjunctionMaxQueries from the user's query. The format supported is fieldOne^2.3 fieldTwo fieldThree^0.4, which indicates that fieldOne has a boost of 2.3, fieldTwo has the default boost, and fieldThree has a boost of 0.4 ... this indicates that matches in fieldOne are much more significant than matches in fieldTwo, which are more significant than matches in fieldThree.

mm (Minimum 'Should' Match)

When dealing with queries there are 3 types of "clauses" that Lucene knows about: mandatory, prohibited, and 'optional' (aka: "SHOULD") By default all words or phrases specified in the "q" param are treated as "optional" clauses unless they are preceeded by a "+" or a "-". When dealing with these "optional" clauses, the "mm" option makes it possible to say that a certain minimum number of those clauses must match (mm). Specifying this minimum number can be done in complex ways, equating to ideas like...

• At least 2 of the optional clauses must match, regardless of how many clauses there are: "2"
• At least 75% of the optional clauses must match, rounded down: "75%"
• If there are less than 3 optional clauses, they all must match; if there are 3 or more, then 75% must match, rounded up: "2<-25 tt="">"
• If there are less than 3 optional clauses, they all must match; for 3 to 5 clauses, one less than the number of clauses must match, for 6 or more clauses, 80% must match, rounded down: "2<-1 nbsp="" tt="">"

Full details on the variety of complex expressions supported are explained in detail here.

In Solr 1.4 and prior, you should basically set mm=0 if you want the equivilent of q.op=OR, and mm=100% if you want the equivilent of q.op=AND. In 3.x and trunk the default value of mm is dictated by the q.op param (q.op=AND => mm=100%; q.op=OR => mm=0%). Keep in mind the default operator is effected by your schema.xml entry. In older versions of Solr the default value is 100% (all clauses must match)

pf (Phrase Fields)

Once the list of matching documents has been identified using the "fq" and "qf" params, the "pf" param can be used to "boost" the score of documents in cases where all of the terms in the "q" param appear in close proximity.

The format is the same as the "qf" param: a list of fields and "boosts" to associate with each of them when making phrase queries out of the entire "q" param.

ps (Phrase Slop)

Amount of slop on phrase queries built for "pf" fields (affects boosting).

qs (Query Phrase Slop)

Amount of slop on phrase queries explicitly included in the user's query string (in qf fields; affects matching). " height="16" src="http://wiki.apache.org/wiki/modernized/img/attention.png" title="<!>" width="16" style="border: 0px;"> Solr1.3

tie (Tie breaker)

Float value to use as tiebreaker in DisjunctionMaxQueries (should be something much less than 1)

When a term from the users input is tested against multiple fields, more than one field may match and each field will generate a different score based on how common that word is in that field (for each document relative to all other documents). By default the score from the field with the maximum score is used. If two documents both have a matching score, the tie parameter has the effect of breaking the tie. When a tie parameter is specified the scores from other matching fields are added to the score of the maximum scoring field:

(score of matching clause with the highest score) + ( (tie paramenter) * (scores of any other matching clauses) )

The "tie" param let's you configure how much the final score of the query will be influenced by the scores of the lower scoring fields compared to the highest scoring field.

A value of "0.0" makes the query a pure "disjunction max query" -- only the maximum scoring sub query contributes to the final score. A value of "1.0" makes the query a pure "disjunction sum query" where it doesn't matter what the maximum scoring sub query is, the final score is the sum of the sub scores. Typically a low value (ie: 0.1) is useful.

bq (Boost Query)

A raw query string (in the SolrQuerySyntax) that will be included with the user's query to influence the score. If this is a BooleanQuery with a default boost (1.0f) then the individual clauses will be added directly to the main query. Otherwise, the query will be included as is.

 :TODO:  That latter part is deprecated behavior but still works. It can be problematic so avoid it.

bf (Boost Functions)

Functions (with optional boosts) that will be included in the user's query to influence the score. Any function supported natively by Solr can be used, along with a boost value, e.g.: recip(rord(myfield),1,2,3)^1.5

Specifying functions with the "bf" param is just shorthand for using the _val_:"...function..." syntax in a "bq" param.

For example, if you want to show more recent documents first, use recip(rord(creationDate),1,1000,1000)

 1.4 best practice: Solr 1.4 best practice is to avoid rord. For date boosting, use recip(ms(NOW,mydatefield),3.16e-11,1,1) or similar. See show more recent documents first, specifically recip, ms and Date Boosting sections.

Examples

 :TODO:  cleanup and expand examples

Search across multiple fields, specifying (via boosts) how important each field is relative each other

http://localhost:8983/solr/select/?q=video&defType=dismax&qf=features^20.0+text^0.3

You can boost results that have a field that matches a specific value...

http://localhost:8983/solr/select/?q=video&defType=dismax&qf=features^20.0+text^0.3&bq=cat:electronics^5.0

Using the "mm" param, 1 and 2 word queries require that all of the optional clauses match, but for queries with three or more clauses one missing clause is allowed...

http://localhost:8983/solr/select/?q=belkin+ipod&defType=dismax&mm=2
http://localhost:8983/solr/select/?q=belkin+ipod+gibberish&defType=dismax&mm=2
http://localhost:8983/solr/select/?q=belkin+ipod+apple&defType=dismax&mm=2