ElasticSearch IK中文分词插件安装使用 linux

ゞ浴缸里的玫瑰 2023-02-24 05:00 234阅读 0赞

可能一年才能想起来写一次博客 居然没忘记密码。。。最近用到了es各种坑坑绊绊，打算系统的总结一下，先写一篇备忘。

es英文分词默认是通过空格之类的拆分成单词吧，举个栗子，在kibana dev\_tools/console运行下面查询

GET _analyze
    {"analyzer" : "standard",  "text" : "hello world es hahha"}

返回结果如下

{
      "tokens" : [
        {
          "token" : "hello",
          "start_offset" : 0,
          "end_offset" : 5,
          "type" : "<ALPHANUM>",
          "position" : 0
        },
        {
          "token" : "world",
          "start_offset" : 6,
          "end_offset" : 11,
          "type" : "<ALPHANUM>",
          "position" : 1
        },
        {
          "token" : "es",
          "start_offset" : 12,
          "end_offset" : 14,
          "type" : "<ALPHANUM>",
          "position" : 2
        },
        {
          "token" : "hahha",
          "start_offset" : 15,
          "end_offset" : 20,
          "type" : "<ALPHANUM>",
          "position" : 3
        }
      ]
    }

如果是有中文搜索，es 默认的analyzer是standard分词，对于中文会按单个文字拆分，比如这么一句话“这是一首简单的小情歌”，每个字都是一个索引，就没有了单词的概念，这时搜索就会返回一些不是想要的结果。

GET _analyze
    {"analyzer" : "standard",  "text" : "这是一首简单的小情歌"}

{
      "tokens" : [
        {
          "token" : "这",
          "start_offset" : 0,
          "end_offset" : 1,
          "type" : "<IDEOGRAPHIC>",
          "position" : 0
        },
        {
          "token" : "是",
          "start_offset" : 1,
          "end_offset" : 2,
          "type" : "<IDEOGRAPHIC>",
          "position" : 1
        },
        {
          "token" : "一",
          "start_offset" : 2,
          "end_offset" : 3,
          "type" : "<IDEOGRAPHIC>",
          "position" : 2
        },
        {
          "token" : "首",
          "start_offset" : 3,
          "end_offset" : 4,
          "type" : "<IDEOGRAPHIC>",
          "position" : 3
        },
        {
          "token" : "简",
          "start_offset" : 4,
          "end_offset" : 5,
          "type" : "<IDEOGRAPHIC>",
          "position" : 4
        },
        {
          "token" : "单",
          "start_offset" : 5,
          "end_offset" : 6,
          "type" : "<IDEOGRAPHIC>",
          "position" : 5
        },
        {
          "token" : "的",
          "start_offset" : 6,
          "end_offset" : 7,
          "type" : "<IDEOGRAPHIC>",
          "position" : 6
        },
        {
          "token" : "小",
          "start_offset" : 7,
          "end_offset" : 8,
          "type" : "<IDEOGRAPHIC>",
          "position" : 7
        },
        {
          "token" : "情",
          "start_offset" : 8,
          "end_offset" : 9,
          "type" : "<IDEOGRAPHIC>",
          "position" : 8
        },
        {
          "token" : "歌",
          "start_offset" : 9,
          "end_offset" : 10,
          "type" : "<IDEOGRAPHIC>",
          "position" : 9
        }
      ]
    }

我用的es 是7.4版本 ，安装步骤很简单，[https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v7.4.0][https_github.com_medcl_elasticsearch-analysis-ik_releases_tag_v7.4.0] 这里下载.zip，版本最好完全对应。到linux 相应es安装目录的plugins目录下新建一个叫ik的文件夹，再把下载好的zip扔到ik这个目录，再执行unzip xxxxx.zip 解压到ik文件夹就行了。

安装完后需要重启一下es，我是jps看了一下进程号，再kill -9 干掉es，回到bin目录 ./elasticsearch -d重启es。

重启后打开kibana试试效果。

GET \_analyze  
\{"analyzer" : "ik\_smart",  "text" : "这是一首简单的小情歌"\}

{
      "tokens" : [
        {
          "token" : "这是",
          "start_offset" : 0,
          "end_offset" : 2,
          "type" : "CN_WORD",
          "position" : 0
        },
        {
          "token" : "一首",
          "start_offset" : 2,
          "end_offset" : 4,
          "type" : "CN_WORD",
          "position" : 1
        },
        {
          "token" : "简单",
          "start_offset" : 4,
          "end_offset" : 6,
          "type" : "CN_WORD",
          "position" : 2
        },
        {
          "token" : "的",
          "start_offset" : 6,
          "end_offset" : 7,
          "type" : "CN_CHAR",
          "position" : 3
        },
        {
          "token" : "小",
          "start_offset" : 7,
          "end_offset" : 8,
          "type" : "CN_CHAR",
          "position" : 4
        },
        {
          "token" : "情歌",
          "start_offset" : 8,
          "end_offset" : 10,
          "type" : "CN_WORD",
          "position" : 5
        }
      ]
    }

GET \_analyze  
\{"analyzer" : "ik\_max\_word",  "text" : "这是一首简单的小情歌"\}

{
      "tokens" : [
        {
          "token" : "这是",
          "start_offset" : 0,
          "end_offset" : 2,
          "type" : "CN_WORD",
          "position" : 0
        },
        {
          "token" : "一首",
          "start_offset" : 2,
          "end_offset" : 4,
          "type" : "CN_WORD",
          "position" : 1
        },
        {
          "token" : "一",
          "start_offset" : 2,
          "end_offset" : 3,
          "type" : "TYPE_CNUM",
          "position" : 2
        },
        {
          "token" : "首",
          "start_offset" : 3,
          "end_offset" : 4,
          "type" : "COUNT",
          "position" : 3
        },
        {
          "token" : "简单",
          "start_offset" : 4,
          "end_offset" : 6,
          "type" : "CN_WORD",
          "position" : 4
        },
        {
          "token" : "的",
          "start_offset" : 6,
          "end_offset" : 7,
          "type" : "CN_CHAR",
          "position" : 5
        },
        {
          "token" : "小",
          "start_offset" : 7,
          "end_offset" : 8,
          "type" : "CN_CHAR",
          "position" : 6
        },
        {
          "token" : "情歌",
          "start_offset" : 8,
          "end_offset" : 10,
          "type" : "CN_WORD",
          "position" : 7
        }
      ]
    }

感觉还是ik\_smart比较智能一些？

[https_github.com_medcl_elasticsearch-analysis-ik_releases_tag_v7.4.0]: https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v7.4.0