ElasticSearch 全文搜索之基础语法API

蔚落 2023-06-27 03:15 22阅读 0赞

> 全文实例基于Elasticsearch7.x

这篇博客介绍ElasticSearch全文搜索的基础语法, 后续的复合搜索(bool)和过滤(filter)都是基于这些基础语法的.

全文搜索在搜索时, 会对输入的搜索文本进行分词, 然后去倒排索引中进行匹配, 只要能匹配上任意一个关键词(词项), 就可以作为结果返回.

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdGlhbnhpYW5nX2thb2xh_size_16_color_FFFFFF_t_70]

### Rest API ###

**添加全文搜索实例数据**

POST /blogs/_bulk
    {"index": {}}
    {"post_date": "2020-01-01", "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen.", "author_id": 11401}
    {"index": {}}
    {"post_date": "2020-01-02", "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis.", "author_id": 11402}
    {"index": {}}
    {"post_date": "2020-01-03", "title": "My dog barks", "content": "I see a lot of barking dogs on the road.", "author_id": 11403}

**match\_all**

搜索索引的全部文档.

GET /blogs/_search
    {
      "query": {
        "match_all": {}
      }
    }

**match**

match匹配有三种语法.

(1) or

GET /blogs/_search
    {
      "query": {
        "match": {
          "content": "brown rabbits"
        }
      }
    }

或者

GET /blogs/_search
    {
      "query": {
        "match": {
          "content": {
            "query": "brown rabbits",
            "operator": "or"
          }
        }
      }
    }

分词后的关键词各自去匹配文档, 只要有一个关键词配置上就返回.

(2) and

GET /blogs/_search
    {
      "query": {
        "match": {
          "content": {
            "query": "brown rabbits",
            "operator": "and"
          }
        }
      }
    }

要求分词后的关键词在同一个文档中.(不需要是连在一起的短语)

(3) minimun\_should\_match

GET /blogs/_search
    {
      "query": {
        "match": {
          "content": {
            "query": "Quick brown rabbits",
            "minimum_should_match": 2
          }
        }
      }
    }

分词后的关键词至少有n个在同一个文档中.

**match\_phrase**

短语搜索默认情况下要求关键词必须相邻, 也可以通过slop控制关键词间隔的单词数.

(1) 默认

GET /blogs/_search
    {
      "query": {
        "match_phrase": {
          "title": "Quick brown"
        }
      }
    }

默认情况下slop等于0.

(2) 手动设置slop

GET /blogs/_search
    {
      "query": {
        "match_phrase": {
          "title": {
            "query": "Quick rabbits",
            "slop": 1
          }
        }
      }
    }

**dis\_max与tie\_breaker**

前面的实例都是在一个字段上进行搜索, 但在实际场景中很多都是在多个字段上进行搜索, 那我们如何在这种情况下找到最佳匹配的结果呢?

*  dis\_max取单个字段上匹配分数最高的分数作为整体评分.
 *  tie\_breaker在dis\_max基础上, 将其他字段的匹配分数与tie\_breaker系数相乘, 然后再加上dis\_max获取的分数.

(1) dis\_max

GET /blogs/_search
    {
      "query": {
        "dis_max": {
          "queries": [
            {
              "match": {
                "title": "brown fox"
              }
            },
            {
              "match": {
                "content": "brown fox"
              }
            }
          ]
        }
      }
    }

(2) dis\_max与tie\_breaker

GET /blogs/_search
    {
      "query": {
        "dis_max": {
          "tie_breaker": 0.7, 
          "queries": [
            {
              "match": {
                "title": "brown fox"
              }
            },
            {
              "match": {
                "content": "brown fox"
              }
            }
          ]
        }
      }
    }

**multi\_match**

在多个字段中匹配搜索文本除了可以使用dis\_max与tie\_breaker之外, 还可以使用multi\_match. 这里涉及到几种匹配策略:

*  best-fields  
    文档的某个字段匹配尽可能多的关键词, 那么这个文档会优先返回.
 *  most-fields  
    某个关键词匹配文档尽可能多的属性, 那么这个文档会优先返回.
 *  cross\_fields  
    跨越多个字段搜索一个关键词.

> 举例说明: doc1的field1匹配的三个关键词, doc2的field1, field2都匹配上了同一个关键词. 如果是best-fields策略, 则doc1的相关度分数要更高, 如果是most-fields策略, 则doc2的相关度分数要更高.

(1) best-fields

GET /blogs/_search
    {
      "query": {
        "multi_match": {
          "query": "Quick pets",
          "type": "best_fields", 
          "fields": ["title", "content"]
        }
      }
    }

(2) most\_fields

GET /blogs/_search
    {
      "query": {
        "multi_match": {
          "query": "Quick pets",
          "type": "most_fields", 
          "fields": ["title", "content"]
        }
      }
    }

(3) cross\_fields

GET /blogs/_search
    {
      "query": {
        "multi_match": {
          "query": "Quick pets",
          "type": "cross_fields", 
          "operator": "and", 
          "fields": ["title", "content"]
        }
      }
    }

加了and之后, 要求title和content字段上都必须要匹配上Quick或pets.

**term**

Term 是表达语意的最小单位. 基于term进行搜索, 搜索文本不会被分词, 将会作为一个整体去倒排索引中进行匹配.

text类型的字段因为在创建倒排索引时会分词, 所以对text类型的字段进行term搜索不会有结果, 有三种解决方法:

*  使用text的多字段类型keyword进行搜索.
 *  创建Mapping时将fielddata参数设置为true.
 *  创建Mapping时将字段类型设置为keyword.

> 详细讲解: https://blog.csdn.net/litianxiang\_kaola/article/details/103344432

(1) term

GET /blogs/_search
    {
      "query": {
        "term": {
          "post_date": {
            "value": "2020-01-01"
          }
        }
      }
    }
    
    GET /blogs/_search
    {
      "query": {
        "term": {
          "title.keyword": {
            "value": "My dog barks"
          }
        }
      }
    }

(2) terms

GET /blogs/_search
    {
      "query": {
        "terms": {
          "title.keyword": [
            "My dog barks",
            "Quick brown rabbits"
          ]
        }
      }
    }

**range**

限制搜索范围.

GET /blogs/_search
    {
      "query": {
        "range": {
          "post_date": {
            "gte": "2020-01-01",
            "lte": "2020-01-03"
          }
        }
      }
    }

**排序与分页**

排序与分页的语法与搜索同级.

GET /blogs/_search
    {
      "query": {
        "range": {
          "post_date": {
            "gte": "2020-01-01",
            "lte": "2020-01-03"
          }
        }
      },
      "sort": [
        {
          "author_id": {
            "order": "desc"
          }
        }
      ],
      "from": 0,
      "size": 2
    }

**\_source filtering**

设置返回的字段.

GET /blogs/_search
    {
      "_source": ["post_date","title"], 
      "query": {
        "range": {
          "post_date": {
            "gte": "2020-01-01",
            "lte": "2020-01-03"
          }
        }
      }
    }

### Java API ###

下面介绍Elasticsearch Java Client 的使用, 我们来将上文的实例转化为 Java Client.

(1) main方法

public static void main(String[] args) throws IOException {
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(
                        new HttpHost("localhost", 9200, "http")));
    
        bulkIndex(client);
        
        matchAll(client);
    
        match(client);
    
        matchPhrase(client);
    
        disMaxAndTieBreaker(client);
    
        multiMatch(client);
    
        term(client);
    
        range(client);
    
        pagination(client);
    
        sourceFiltering(client);
    
        client.close();
    }

新增文档和查询文档请求不要一起执行, 这样会查不到文档, 因为新增文档后需要1s时间进行倒排索引创建, 这也是ES被称为近实时的原因.

(2) 添加搜索数据

private static void bulkIndex(RestHighLevelClient client) throws IOException {
        BulkRequest bulkRequest = new BulkRequest();
    
        bulkRequest.add(new IndexRequest("blogs").id("1")
                .source(XContentType.JSON, "post_date", "2020-01-01", "title", "Quick brown rabbits", "content", "Brown rabbits are commonly seen.", "author_id", 11401));
        bulkRequest.add(new IndexRequest("blogs").id("2")
                .source(XContentType.JSON, "post_date", "2020-01-02", "title", "Keeping pets healthy", "content", "My quick brown fox eats rabbits on a regular basis.", "author_id", 11402));
        bulkRequest.add(new IndexRequest("blogs").id("3")
                .source(XContentType.JSON, "post_date", "2020-01-03", "title", "My dog barks", "content", "I see a lot of barking dogs on the road.", "author_id", 11403));
    
        client.bulk(bulkRequest, RequestOptions.DEFAULT);
    }

(3) match\_all

private static void matchAll(RestHighLevelClient client) throws IOException {
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.matchAllQuery());
    
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    }

(4) match

private static void match(RestHighLevelClient client) throws IOException {
        //1. or
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        MatchQueryBuilder matchQueryBuilder = new MatchQueryBuilder("content", "brown rabbits");
        matchQueryBuilder.operator(Operator.OR);
        searchSourceBuilder.query(matchQueryBuilder);
    
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    
        System.out.println("------------------------------");
    
        //2. and
        SearchRequest searchRequest2 = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder2 = new SearchSourceBuilder();
        MatchQueryBuilder matchQueryBuilder2 = new MatchQueryBuilder("content", "brown rabbits");
        matchQueryBuilder2.operator(Operator.AND);
        searchSourceBuilder2.query(matchQueryBuilder2);
    
        searchRequest2.source(searchSourceBuilder2);
        SearchResponse searchResponse2 = client.search(searchRequest2, RequestOptions.DEFAULT);
        SearchHit[] hits2 = searchResponse2.getHits().getHits();
        for (SearchHit hit : hits2) {
            System.out.println(hit.getSourceAsString());
        }
    
        System.out.println("------------------------------");
    
        //3. minimun_should_match
        SearchRequest searchRequest3 = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder3 = new SearchSourceBuilder();
        MatchQueryBuilder matchQueryBuilder3 = new MatchQueryBuilder("content", "Quick brown rabbits");
        matchQueryBuilder3.minimumShouldMatch("3");
        searchSourceBuilder3.query(matchQueryBuilder3);
    
        searchRequest3.source(searchSourceBuilder3);
        SearchResponse searchResponse3 = client.search(searchRequest3, RequestOptions.DEFAULT);
        SearchHit[] hits3 = searchResponse3.getHits().getHits();
        for (SearchHit hit : hits3) {
            System.out.println(hit.getSourceAsString());
        }
    }

(5) match\_phrase

private static void matchPhrase(RestHighLevelClient client) throws IOException {
        //1. 默认
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        MatchPhraseQueryBuilder matchPhraseQueryBuilder = new MatchPhraseQueryBuilder("title", "Quick brown");
        searchSourceBuilder.query(matchPhraseQueryBuilder);
    
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    
        System.out.println("------------------------------");
    
        //2. 指定slop
        SearchRequest searchRequest2 = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder2 = new SearchSourceBuilder();
        MatchPhraseQueryBuilder matchPhraseQueryBuilder2 = new MatchPhraseQueryBuilder("title", "Quick rabbits");
        matchPhraseQueryBuilder2.slop(1);
        searchSourceBuilder2.query(matchPhraseQueryBuilder2);
    
        searchRequest2.source(searchSourceBuilder2);
        SearchResponse searchResponse2 = client.search(searchRequest2, RequestOptions.DEFAULT);
        SearchHit[] hits2 = searchResponse2.getHits().getHits();
        for (SearchHit hit : hits2) {
            System.out.println(hit.getSourceAsString());
        }
    }

(6) dis\_max和tie\_breaker

private static void disMaxAndTieBreaker(RestHighLevelClient client) throws IOException {
        //1. dis_max
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        DisMaxQueryBuilder disMaxQueryBuilder = new DisMaxQueryBuilder();
        MatchQueryBuilder matchQueryBuilder1 = new MatchQueryBuilder("title", "brown fox");
        MatchQueryBuilder matchQueryBuilder2 = new MatchQueryBuilder("content", "brown fox");
        disMaxQueryBuilder.add(matchQueryBuilder1);
        disMaxQueryBuilder.add(matchQueryBuilder2);
        searchSourceBuilder.query(disMaxQueryBuilder);
    
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    
        System.out.println("------------------------------");
    
        //2. dis_max + tie_breaker
        SearchRequest searchRequest2 = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder2 = new SearchSourceBuilder();
        DisMaxQueryBuilder disMaxQueryBuilder2 = new DisMaxQueryBuilder();
        MatchQueryBuilder matchQueryBuilder3 = new MatchQueryBuilder("title", "brown fox");
        MatchQueryBuilder matchQueryBuilder4 = new MatchQueryBuilder("content", "brown fox");
        disMaxQueryBuilder2.add(matchQueryBuilder3);
        disMaxQueryBuilder2.add(matchQueryBuilder4);
        disMaxQueryBuilder2.tieBreaker(0.7f);
        searchSourceBuilder2.query(disMaxQueryBuilder2);
    
        searchRequest2.source(searchSourceBuilder2);
        SearchResponse searchResponse2 = client.search(searchRequest2, RequestOptions.DEFAULT);
        SearchHit[] hits2 = searchResponse2.getHits().getHits();
        for (SearchHit hit : hits2) {
            System.out.println(hit.getSourceAsString());
        }
    }

(7) multi\_match

private static void multiMatch(RestHighLevelClient client) throws IOException {
        //1. best-fields
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        MultiMatchQueryBuilder multiMatchQueryBuilder = new MultiMatchQueryBuilder("Quick pets", "title", "content");
        multiMatchQueryBuilder.type(MultiMatchQueryBuilder.Type.BEST_FIELDS);
        searchSourceBuilder.query(multiMatchQueryBuilder);
    
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    
        System.out.println("------------------------------");
    
        //2. most_fields
        SearchRequest searchRequest2 = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder2 = new SearchSourceBuilder();
        MultiMatchQueryBuilder multiMatchQueryBuilder2 = new MultiMatchQueryBuilder("Quick pets", "title", "content");
        multiMatchQueryBuilder2.type(MultiMatchQueryBuilder.Type.MOST_FIELDS);
        searchSourceBuilder2.query(multiMatchQueryBuilder2);
    
        searchRequest2.source(searchSourceBuilder2);
        SearchResponse searchResponse2 = client.search(searchRequest2, RequestOptions.DEFAULT);
        SearchHit[] hits2 = searchResponse2.getHits().getHits();
        for (SearchHit hit : hits2) {
            System.out.println(hit.getSourceAsString());
        }
    
        System.out.println("------------------------------");
    
        //3. cross_fields
        SearchRequest searchRequest3 = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder3 = new SearchSourceBuilder();
        MultiMatchQueryBuilder multiMatchQueryBuilder3 = new MultiMatchQueryBuilder("Quick pets", "title", "content");
        multiMatchQueryBuilder3.type(MultiMatchQueryBuilder.Type.CROSS_FIELDS);
        multiMatchQueryBuilder3.operator(Operator.AND);
        searchSourceBuilder3.query(multiMatchQueryBuilder3);
    
        searchRequest3.source(searchSourceBuilder3);
        SearchResponse searchResponse3 = client.search(searchRequest3, RequestOptions.DEFAULT);
        SearchHit[] hits3 = searchResponse3.getHits().getHits();
        for (SearchHit hit : hits3) {
            System.out.println(hit.getSourceAsString());
        }
    }

(8) term

private static void term(RestHighLevelClient client) throws IOException {
        //1. term
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        TermQueryBuilder termQueryBuilder = new TermQueryBuilder("post_date", "2020-01-01");
        searchSourceBuilder.query(termQueryBuilder);
    
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    
        System.out.println("------------------------------");
    
        //2. term-text
        SearchRequest searchRequest2 = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder2 = new SearchSourceBuilder();
        TermQueryBuilder termQueryBuilder2 = new TermQueryBuilder("title.keyword", "My dog barks");
        searchSourceBuilder2.query(termQueryBuilder2);
    
        searchRequest2.source(searchSourceBuilder2);
        SearchResponse searchResponse2 = client.search(searchRequest2, RequestOptions.DEFAULT);
        SearchHit[] hits2 = searchResponse2.getHits().getHits();
        for (SearchHit hit : hits2) {
            System.out.println(hit.getSourceAsString());
        }
    
        System.out.println("------------------------------");
    
        //3. terms
        SearchRequest searchRequest3 = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder3 = new SearchSourceBuilder();
        TermsQueryBuilder termsQueryBuilder3 = new TermsQueryBuilder("title.keyword", "My dog barks", "Quick brown rabbits");
        searchSourceBuilder3.query(termsQueryBuilder3);
    
        searchRequest3.source(searchSourceBuilder3);
        SearchResponse searchResponse3 = client.search(searchRequest3, RequestOptions.DEFAULT);
        SearchHit[] hits3 = searchResponse3.getHits().getHits();
        for (SearchHit hit : hits3) {
            System.out.println(hit.getSourceAsString());
        }
    }

(9) range

private static void range(RestHighLevelClient client) throws IOException {
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        RangeQueryBuilder rangeQueryBuilder = new RangeQueryBuilder("post_date");
        rangeQueryBuilder.gte("2020-01-01");
        rangeQueryBuilder.lte("2020-01-03");
        searchSourceBuilder.query(rangeQueryBuilder);
    
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    }

(10) pagination

private static void pagination(RestHighLevelClient client) throws IOException {
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        RangeQueryBuilder rangeQueryBuilder = new RangeQueryBuilder("post_date");
        rangeQueryBuilder.gte("2020-01-01");
        rangeQueryBuilder.lte("2020-01-03");
        searchSourceBuilder.query(rangeQueryBuilder);
        searchSourceBuilder.sort(new FieldSortBuilder("author_id").order(SortOrder.DESC));
        searchSourceBuilder.from(0);
        searchSourceBuilder.size(2);
    
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    }

(11) source filtering

private static void sourceFiltering(RestHighLevelClient client) throws IOException {
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        RangeQueryBuilder rangeQueryBuilder = new RangeQueryBuilder("post_date");
        rangeQueryBuilder.gte("2020-01-01");
        rangeQueryBuilder.lte("2020-01-03");
        searchSourceBuilder.query(rangeQueryBuilder);
        String[] includeFields = new String[]{"title", "post_date"};
        String[] excludeFields = new String[]{};
        searchSourceBuilder.fetchSource(includeFields, excludeFields);
    
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    }

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdGlhbnhpYW5nX2thb2xh_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200102145725993.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdGlhbnhpYW5nX2thb2xh,size_16,color_FFFFFF,t_70