深度学习：bert embedding用法详解

﹏ヽ暗。殇╰゛Y 2022-04-08 11:40 1230阅读 0赞

## 环境配置 ##

### 下载bert已训练好的模型 ###

如**[`BERT-Base, Chinese`][BERT-Base_ Chinese]**: Chinese Simplified and Traditional, 12-layer, 768-hidden, 12-heads, 110M parameters

解压到目录/.../chinese\_L-12\_H-768\_A-12/，其中...为你自己的某个目录。

\[[https://github.com/google-research/bert\#pre-trained-models][https_github.com_google-research_bert_pre-trained-models]\]

### 安装`bert-as-service` ###

`bert-as-service` uses BERT as a sentence encoder and hosts it as a service via ZeroMQ, allowing you to map sentences into fixed-length representations in just two lines of code.

pip3 install bert-serving-server \# server

pip3 install bert-serving-client \# client, independent of \`bert-serving-server\`

Note: 要求Python >= 3.5 及 Tensorflow >= 1.10（pip3 install tf-nightly）。

[\-柚子皮-][-_-]

### 开启bert服务 ###

**如果只是想获取句子的embedding**

bert-serving-start -model\_dir /Users/youzipi/files/mine/data/chinese\_L-12\_H-768\_A-12/ -num\_worker=4 -max\_seq\_len=4 &

Note: 相当于将句子中的词的embedding取均值输出。

**如果想获取句子中单个词的embedding**

nohup bert-serving-start -pooling\_strategy NONE -model\_dir /home/piting.pt/data/chinese\_L-12\_H-768\_A-12/ -num\_worker=2 -max\_seq\_len=4 -port=5557 -port\_out=5558 >> ../../log.txt 2>& 1 &  
sleep 240

成功提示：... I:WORKER-3:\[\_\_i:gen:322\]: ready and listening!

如果出错：1 zmq.error.ZMQError: Address already in use. 可能是别人已经开启，直接用别人开启的服务也可以，如果和别人开启的服务器版本对不上也会出错，这时可以在上面的bert-serving-start 命令中间添加 -port=5557 -port\_out=5558；且代码中调用时也使用相同port接口：bc = BertClient(show\_server\_config=False, port=5557, port\_out=5558)。

2 shutting down... terminated!。可能是因为已经在运行，但是如果关了port的话也没关系。jobs查看后台运行。

**停止bert服务**

老版本不支持，只能手动crtl+c或者kill，更新版本可以使用命令停止。

pip install -U bert-serving-server bert-serving-client

bert-serving-terminate -port \*\*\*

开启的服务有这些：

93771.+  20   0  120820   1920    808 S   0.0  0.0   0:00.02 sshd

93772.+  20   0  116576   3404   1800 S   0.0  0.0   0:00.09 bash

98198.+  20   0 1631412 201932  77316 T   0.0  0.1   0:01.72 bert-serving-st

99737.+  20   0 1631252 126112   1896 T   0.0  0.0   0:00.00 ZMQbg/3

...

99751.+  20   0 25.379g 1.272g 215688 T   0.0  0.5   3:46.51 ZMQbg/3

### 获取bert embedding ###

**获取句子的embedding**

from bert_serving.client import BertClient
    
    bc = BertClient(show_server_config=False)
    bert_embed = bc.encode(['价格比比较不错的酒店。', 'then do it right', 'then do it better'])
    print(bert_embed)

**获取句子中单个词的embedding**

from bert_serving.client import BertClient
    
    bc = BertClient(show_server_config=False)
    bert_embed = bc.encode(['我'])[0][1]
    print(bert_embed)

Note：可以从请求后的输出看到进度。如I:SINK:\[\_\_i:\_ru:342\]:send back size: 1 job id: b'd3dc7ea7-8018-488f-9980-98cabde69021\#2083'表示已处理完2083条请求。

**一些需要说明的示例**

（使用pooling\_strategy = NONE及`max_seq_len = 25`）

# max_seq_len = 25
    # pooling_strategy = NONE
    
    bc = BertClient(show_server_config=False)vec = bc.encode(['hey you', 'whats up?'])
    
    vec  # [2, 25, 768]
    vec[0]  # [1, 25, 768], sentence embeddings for `hey you`
    vec[0][0]  # [1, 1, 768], word embedding for `[CLS]`
    vec[0][1]  # [1, 1, 768], word embedding for `hey`
    vec[0][2]  # [1, 1, 768], word embedding for `you`
    vec[0][3]  # [1, 1, 768], word embedding for `[SEP]`
    vec[0][4]  # [1, 1, 768], word embedding for padding symbol
    vec[0][25]  # error, out of index!

如果使用pooling\_strategy = NONE则可以看到cls, sep都编码了，我们只取需要的，所以max\_seq\_len大小至少为str长度+2，即‘我’+cls+sep至少为3。另外pad的embedding为0不需要考虑。

如果是在map-reduce中使用bert\_encode()，可以并行开多个bert client，类似：

num_parallel_calls = 4
    num_concurrent_clients = num_parallel_calls * 2
    bc_clients = [BertClient(show_server_config=False) for _ in range(num_concurrent_clients)]
    
    
    def bert_encode(word):
        bc = bc_clients.pop()
        vocb_bert = bc.encode([word])[0][1]
        bc_clients.append(bc)
        return vocb_bert

\[example can [be found example4.py][]\]

\[[bert-as-service][]\]

from: [https://blog.csdn.net/pipisorry][https_blog.csdn.net_pipisorry]

ref:

[BERT-Base_ Chinese]: https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip
[https_github.com_google-research_bert_pre-trained-models]: https://github.com/google-research/bert#pre-trained-models
[-_-]: https://me.csdn.net/pipisorry
[be found example4.py]: https://github.com/hanxiao/bert-as-service/blob/master/example/example4.py
[bert-as-service]: https://github.com/hanxiao/bert-as-service
[https_blog.csdn.net_pipisorry]: https://blog.csdn.net/pipisorry