【Elasticsearch-05】文档基本CRDU与批量操作

朴灿烈づ我的快乐病毒、 2021-10-18 13:24 275阅读 0赞

### 1. 文档CRDU规则 ###

*  规则：

PUT/POST/GET/DELETE 索引名称/_doc

*  文档相关的操作Type都用\_doc

<table> 
 <thead> 
  <tr> 
   <th>操作</th> 
   <th>举个栗子</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>index</td> 
   <td>PUT my_index/_doc/1 <br>{“user”:“Mike”, “comment”:“You know, for search”}</td> 
   <td>如果ID不存在，创建新的文档，否则，先删除现有的文档，再创建新的文档，版本会增加</td> 
  </tr> 
  <tr> 
   <td>create</td> 
   <td>PUT my_index/_create/1 <br>{“user”:“Mike”, “comment”:“You know, for search”} <br>POST my_index/_doc (不指定ID, 自动生成) <br>{“user”:“Mike”, “comment”:“You know, for search”}</td> 
   <td>- 有两种方式，<br>- 1. 指定ID <br>- 2. 不指定ID，系统自动生成一串随机字符串<br>- 如果ID已经存在，会失败</td> 
  </tr> 
  <tr> 
   <td>read</td> 
   <td>GET my_index/_update/1</td> 
   <td>无</td> 
  </tr> 
  <tr> 
   <td>update</td> 
   <td>POST my_index/_update <br>{“doc”: {“user”:“Mike”, “comment”:“You know, for search”}}</td> 
   <td>文档必须已经存在，更新只会对相应字段做增量修改</td> 
  </tr> 
  <tr> 
   <td>delete</td> 
   <td>DELETE my_index/_doc/1</td> 
   <td>无</td> 
  </tr> 
 </tbody> 
</table>

### Kibana 实际操作 ###

*  打开Dev Tool，先创建一个索引  
    ![图片: https://uploader.shimo.im/f/26IyWjgUJGEo0Uod.png][https_uploader.shimo.im_f_26IyWjgUJGEo0Uod.png]  
    ![图片: https://uploader.shimo.im/f/nBrbt8eHE3EtH9gS.png][https_uploader.shimo.im_f_nBrbt8eHE3EtH9gS.png]

#### Create 一个文档 ####

*  支持自动生成文档ID和指定文档ID两种方式：
    
     *  1.  指定文档ID创建：

PUT users/_create/1
    { 
      "firstName":"Jinjuan",
      "lastName": "Wu",
      "tags":["Java", "ElasticSearch"]
    }

*  1.1 操作成功  
    ![图片: https://uploader.shimo.im/f/iIlOAb67SskJAFUs.png][https_uploader.shimo.im_f_iIlOAb67SskJAFUs.png]
 *  1.  不指定文档ID创建

POST /users/_doc
    { 
      "firstName":"Jinjuan1",
      "lastName": "Wu",
      "tags":["Java", "ElasticSearch"]
    }

*  2.1 创建成功  
    ![图片: https://uploader.shimo.im/f/OsOVPC2b2ZU3rUI5.png][https_uploader.shimo.im_f_OsOVPC2b2ZU3rUI5.png]
 *  使用HTTP PUT users/\_create/1创建时，URI中显示指定\_create，此时如果该ID的文档已经存在，显示操作失败  
    ![图片: https://uploader.shimo.im/f/p1o3pmmLLHkwdH9M.png][https_uploader.shimo.im_f_p1o3pmmLLHkwdH9M.png]  
    ![图片: https://uploader.shimo.im/f/bIApZGCAAg0avdzM.png][https_uploader.shimo.im_f_bIApZGCAAg0avdzM.png]

#### Get一个文档 ####

![图片: https://uploader.shimo.im/f/wZc8g4ad8EkAzfaS.png][https_uploader.shimo.im_f_wZc8g4ad8EkAzfaS.png]  
![图片: https://uploader.shimo.im/f/QJW5LFOsgr8FJSAJ.png][https_uploader.shimo.im_f_QJW5LFOsgr8FJSAJ.png]

*  文档源信息：

<table> 
 <thead> 
  <tr> 
   <th>字段</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>_index</td> 
   <td>索引名称</td> 
  </tr> 
  <tr> 
   <td>_type</td> 
   <td>版本信息，同一个Id的文档，即使被删除，Version号也会不断增加</td> 
  </tr> 
  <tr> 
   <td>version</td> 
   <td>版本信息，同一个Id的文档，即使被删除，Version号也会不断增加</td> 
  </tr> 
  <tr> 
   <td>_source</td> 
   <td>默认包含了文档的所有原始信息</td> 
  </tr> 
 </tbody> 
</table>

#### Index文档 ####

*  Index和Update还有Create不一样的地方：如果文档不存在，就索引新的文档，否则现有文档会被删除，新的文档会被索引，版本信息 + 1

PUT users/_doc/1
    { 
      "tags":["少女", "女汉子"]
    }

![图片: https://uploader.shimo.im/f/xBNOMAYVNTEtCudy.png][https_uploader.shimo.im_f_xBNOMAYVNTEtCudy.png]

*  GET users/\_doc/1 查看一下，firstName和lastName都没了  
    ![图片: https://uploader.shimo.im/f/IXYi6dspv1YEAnr3.png][https_uploader.shimo.im_f_IXYi6dspv1YEAnr3.png]

#### Update文档 ####

*  Update方法不会删除原来的文档，而是实现真正的数据更新
 *  Post方法 / Payload，注意：需要包含在\*\*“doc”\*\*中

##### 更新没有的字段 #####

![图片: https://uploader.shimo.im/f/pyloIm4W7HwzDvsg.png][https_uploader.shimo.im_f_pyloIm4W7HwzDvsg.png]  
![图片: https://uploader.shimo.im/f/QO1mzxlZ54g5c2oe.png][https_uploader.shimo.im_f_QO1mzxlZ54g5c2oe.png]

##### Get一下 #####

![图片: https://uploader.shimo.im/f/5QTJJpMlnqAg537K.png][https_uploader.shimo.im_f_5QTJJpMlnqAg537K.png]

##### 更新已存在的字段 #####

POST users/_update/1
    { 
      "doc":{ 
        "tags" : [
          "是少女啊",
          "女汉子"
        ],
        "emails":["532791689@qq.com","wujinjuan5@163.com"]
      }
    }

![图片: https://uploader.shimo.im/f/3bPGwjYLwhIQncPu.png][https_uploader.shimo.im_f_3bPGwjYLwhIQncPu.png]

### Bulk API ###

背景：频繁地调用API查询会增加时间成本，就像多次连接数据库带来的成本一样，Bulk API能做到：

*  在一次API调用中，对不同的索引进行操作
 *  支持四种类型操作
    
     *  index
     *  create
     *  update
     *  delete
 *  可以在URI中指定index，也可以在请求的Payload中进行
 *  操作中单条操作失败，并不会影响其他操作
 *  返回结果包罗了每一条操作执行的结果

##### Bulk的格式 #####

{ action:{ metadata}}\n
    { requestbody}

<table> 
 <thead> 
  <tr> 
   <th>action（行为）</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>create</td> 
   <td>文档不存在时创建</td> 
  </tr> 
  <tr> 
   <td>create</td> 
   <td>更新文档</td> 
  </tr> 
  <tr> 
   <td>index</td> 
   <td>创建新文档或替换已有文档</td> 
  </tr> 
  <tr> 
   <td>delete</td> 
   <td>删除一个文档</td> 
  </tr> 
 </tbody> 
</table>

*  metadata：\_index, \_type, \_id
 *  requestbody: 请求体参数们

#### 示例 ####

POST _bulk
    { "index":{ "_index":"users", "_id":"1"}}
    { "tags":"532791689@qq.com"}

*  如果你不想每个操作都写\_index的话，你可以在POST后面指定索引名，比如这样：（前提是操作同一个索引）

POST /users/_bulk
    { "index":{ "_id":"1"}}
    { "tags":"532791689@qq.com"}

#### 批量添加 ####

POST /users/_bulk
    { "create":{ "_id":5}}
    { "firstName" : "San", "lastName" : "Zhang"}
    { "create":{ "_id":6}}
    { "firstName" : "Wu", "lastName" : "Zhao"}

![图片: https://uploader.shimo.im/f/kg1X2E7vwpg0SWND.png][https_uploader.shimo.im_f_kg1X2E7vwpg0SWND.png]

#### 批量删除 ####

POST /users/_bulk
    { "delete":{ "_id":5}}
    { "index":{ "_id":8}}
    { "name":"zhangsan"}
    { "delete":{ "_id":6}}

![图片: https://uploader.shimo.im/f/a38K6HzjfvcPvsvn.png][https_uploader.shimo.im_f_a38K6HzjfvcPvsvn.png]  
![图片: https://uploader.shimo.im/f/AsywjW5b5Sgab415.png][https_uploader.shimo.im_f_AsywjW5b5Sgab415.png]

#### bulk一次最大处理多少数据量 ####

*  bulk会把将要处理的数据载入内存中，所以数据量是有限制的，最佳的数据量不是一个确定的数值，它取决于你的硬件，你的文档大小以及复杂性，你的索引以及搜索的负载。
 *  一般建议是1000-5000个文档，大小建议是5-15MB，默认不能超过100M，可以在es的配置文件（即$ES\_HOME下的config下的elasticsearch.yml）中。

### 批量获取 - mget ###

*  批量操作，可以减少网络连接所产生的开销，提高性能

GET /users/_mget
    { 
      "ids":["1","2","3","4","5","6"]
    }

*  查询结果太长，截图不全就放到代码块了。可见，查不到的数据会显示found:false  
    ![图片: https://uploader.shimo.im/f/vNLOYbk3D88EOdHM.png][https_uploader.shimo.im_f_vNLOYbk3D88EOdHM.png]

{ 
      "docs" : [
        { 
          "_index" : "users",
          "_type" : "_doc",
          "_id" : "1",
          "_version" : 16,
          "_seq_no" : 19,
          "_primary_term" : 1,
          "found" : true,
          "_source" : { 
            "tags" : [
              "是少女啊",
              "女汉子"
            ],
            "emails" : [
              "532791689@qq.com",
              "wujinjuan5@163.com"
            ],
            "firstName" : "Jinjuan",
            "lastName" : "Wu"
          }
        },
        { 
          "_index" : "users",
          "_type" : "_doc",
          "_id" : "2",
          "_version" : 2,
          "_seq_no" : 16,
          "_primary_term" : 1,
          "found" : true,
          "_source" : { 
            "title" : "Html5",
            "price" : 45
          }
        },
        { 
          "_index" : "users",
          "_type" : "_doc",
          "_id" : "3",
          "_version" : 1,
          "_seq_no" : 3,
          "_primary_term" : 1,
          "found" : true,
          "_source" : { 
            "firstName" : "Da",
            "lastName" : "Wu",
            "tags" : [
              "Java",
              "ElasticSearch"
            ]
          }
        },
        { 
          "_index" : "users",
          "_type" : "_doc",
          "_id" : "4",
          "found" : false
        },
        { 
          "_index" : "users",
          "_type" : "_doc",
          "_id" : "5",
          "_version" : 1,
          "_seq_no" : 20,
          "_primary_term" : 1,
          "found" : true,
          "_source" : { 
            "firstName" : "San",
            "lastName" : "Zhang"
          }
        },
        { 
          "_index" : "users",
          "_type" : "_doc",
          "_id" : "6",
          "_version" : 1,
          "_seq_no" : 21,
          "_primary_term" : 1,
          "found" : true,
          "_source" : { 
            "firstName" : "Wu",
            "lastName" : "Zhao"
          }
        }
      ]
    }

### 常见错误返回 ###

<table> 
 <thead> 
  <tr> 
   <th>问题</th> 
   <th>原因</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>无法连接</td> 
   <td>网络故障或集群挂了</td> 
  </tr> 
  <tr> 
   <td>连接无法关闭</td> 
   <td>网络故障或节点出错</td> 
  </tr> 
  <tr> 
   <td>429</td> 
   <td>集群过于繁忙</td> 
  </tr> 
  <tr> 
   <td>4xx</td> 
   <td>请求体格式有错</td> 
  </tr> 
  <tr> 
   <td>500</td> 
   <td>集群内部错误</td> 
  </tr> 
 </tbody> 
</table>

[https_uploader.shimo.im_f_26IyWjgUJGEo0Uod.png]: /images/20211018/2c129e6b5b5941be94d66b274edbbab3.png
[https_uploader.shimo.im_f_nBrbt8eHE3EtH9gS.png]: /images/20211018/3dd0c1ba0fbd4c5695201135b396db66.png
[https_uploader.shimo.im_f_iIlOAb67SskJAFUs.png]: /images/20211018/9db1446cb2674a5cae32e797f49bfd75.png
[https_uploader.shimo.im_f_OsOVPC2b2ZU3rUI5.png]: /images/20211018/479e0be61b214436997ac1bf4eff05a6.png
[https_uploader.shimo.im_f_p1o3pmmLLHkwdH9M.png]: /images/20211018/565b04211701427c8d3627a4c985fd83.png
[https_uploader.shimo.im_f_bIApZGCAAg0avdzM.png]: /images/20211018/16218839379f49b888e54627e7e5fc32.png
[https_uploader.shimo.im_f_wZc8g4ad8EkAzfaS.png]: /images/20211018/f159804285224cbcabb97e245dd53849.png
[https_uploader.shimo.im_f_QJW5LFOsgr8FJSAJ.png]: /images/20211018/8150588e64fb46c78d8d7c3e6a088161.png
[https_uploader.shimo.im_f_xBNOMAYVNTEtCudy.png]: /images/20211018/ff923ca31f2f4744ac8a85939b3efbd5.png
[https_uploader.shimo.im_f_IXYi6dspv1YEAnr3.png]: /images/20211018/64e1759a280e4ee29ed02591f388b5a4.png
[https_uploader.shimo.im_f_pyloIm4W7HwzDvsg.png]: /images/20211018/8d907ef8c16c4b48a37dbf3173ccba1b.png
[https_uploader.shimo.im_f_QO1mzxlZ54g5c2oe.png]: /images/20211018/e2f8b2d2b1ef453dae7341b276e63f3c.png
[https_uploader.shimo.im_f_5QTJJpMlnqAg537K.png]: /images/20211018/a44e5c5f998347b3bb35fb36033b113d.png
[https_uploader.shimo.im_f_3bPGwjYLwhIQncPu.png]: /images/20211018/ad973fc2e3ab47728bb8d82b87a8f1b5.png
[https_uploader.shimo.im_f_kg1X2E7vwpg0SWND.png]: /images/20211018/3201143e89d0459ab0f2db3f00effe07.png
[https_uploader.shimo.im_f_a38K6HzjfvcPvsvn.png]: /images/20211018/53ff6c924b9c4867bb1ecff6116b857e.png
[https_uploader.shimo.im_f_AsywjW5b5Sgab415.png]: /images/20211018/e0979adfc34449cd9892726654a0a6f8.png
[https_uploader.shimo.im_f_vNLOYbk3D88EOdHM.png]: /images/20211018/72a79c2589804e419858415baf88a237.png