bert的一系列资料

骑猪看日落 2022-02-25 12:46 186阅读 0赞

1.bert-as-service

[https://github.com/hanxiao/bert-as-service][https_github.com_hanxiao_bert-as-service]

2.参考文献

*  [两行代码玩转 Google BERT 句向量词向量][Google BERT]
 *  [hanxiao/bert-as-service][https_github.com_hanxiao_bert-as-service]
 *  [google-research/bert][google-research_bert]
 *  [利用Bert构建句向量并计算相似度][Bert]

3.预训练模型下载链接：[https://linux.ctolib.com/article/wiki/99669][https_linux.ctolib.com_article_wiki_99669]

The links to the models are here (right-click, 'Save link as...' on the name):

*  **[`BERT-Base, Uncased`][BERT-Base_ Uncased]**: 12-layer, 768-hidden, 12-heads, 110M parameters
 *  **[`BERT-Large, Uncased`][BERT-Large_ Uncased]**: 24-layer, 1024-hidden, 16-heads, 340M parameters
 *  **[`BERT-Base, Cased`][BERT-Base_ Cased]**: 12-layer, 768-hidden, 12-heads , 110M parameters
 *  **[`BERT-Large, Cased`][BERT-Large_ Cased]**: 24-layer, 1024-hidden, 16-heads, 340M parameters
 *  **[`BERT-Base, Multilingual Cased (New, recommended)`][BERT-Base_ Multilingual Cased _New_ recommended]**: 104 languages, 12-layer, 768-hidden, 12-heads, 110M parameters
 *  **[`BERT-Base, Multilingual Uncased (Orig, not recommended)`][BERT-Base_ Multilingual Uncased _Orig_ not recommended] (Not recommended, use `Multilingual Cased` instead)**: 102 languages, 12-layer, 768-hidden, 12-heads, 110M parameters
 *  **[`BERT-Base, Chinese`][BERT-Base_ Chinese]**: Chinese Simplified and Traditional, 12-layer, 768-hidden, 12-heads, 110M parameters

我们发布了论文中的BERT-Base和BERT-Large模型。

Uncased表示在WordPiece tokenization之前文本已经变成小写了，例如，John Smith becomes john smith。Uncased模型也去掉了所有重音标志。

Cased表示保留了真实的大小写和重音标记。通常，除非你已经知道大小写信息对你的任务来说很重要（例如，命名实体识别或词性标记），否则Uncased模型会更好。

Each .zip file contains three items:

*  A TensorFlow checkpoint (`bert_model.ckpt`) containing the pre-trained weights (which is actually 3 files).
 *  A vocab file (`vocab.txt`) to map WordPiece to word id.
 *  A config file (`bert_config.json`) which specifies the hyperparameters of the model.

[https_github.com_hanxiao_bert-as-service]: https://github.com/hanxiao/bert-as-service
[Google BERT]: https://mp.weixin.qq.com/s?__biz=MzIwMTc4ODE0Mw==&mid=2247493033&idx=1&sn=1ae1cd347126b10d6a857cd9bba7b601&chksm=96ea3a29a19db33f3c07723ed6e5ecbb8d2ff1b1617f1cf0d39cb3cc1e6e9c325cc29147d58d&scene=0&xtrack=1#rd
[google-research_bert]: https://github.com/google-research/bert
[Bert]: https://netycc.com/2018/12/05/%E5%88%A9%E7%94%A8bert%E6%9E%84%E5%BB%BA%E5%8F%A5%E5%90%91%E9%87%8F%E5%B9%B6%E8%AE%A1%E7%AE%97%E7%9B%B8%E4%BC%BC%E5%BA%A6/
[https_linux.ctolib.com_article_wiki_99669]: https://linux.ctolib.com/article/wiki/99669
[BERT-Base_ Uncased]: https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-12_H-768_A-12.zip
[BERT-Large_ Uncased]: https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-24_H-1024_A-16.zip
[BERT-Base_ Cased]: https://storage.googleapis.com/bert_models/2018_10_18/cased_L-12_H-768_A-12.zip
[BERT-Large_ Cased]: https://storage.googleapis.com/bert_models/2018_10_18/cased_L-24_H-1024_A-16.zip
[BERT-Base_ Multilingual Cased _New_ recommended]: https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip
[BERT-Base_ Multilingual Uncased _Orig_ not recommended]: https://storage.googleapis.com/bert_models/2018_11_03/multilingual_L-12_H-768_A-12.zip
[BERT-Base_ Chinese]: https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip