论文阅读《An Effective Transition-based Model for Discontinuous NER》

ゞ浴缸里的玫瑰 2022-12-19 14:26 113阅读 0赞

# 0.总结 #

*  使用transition-based model。这种model 使用传统数据结构（stack ）作为基础，自定义栈的操作类型，从而达到一个 `entity recognization` 的效果
 *  如果需要本文对应的ppt，可以从我的[GitHub][]中获取哇~
 *  文章来源：CSDN@LawsonAbs

--------------------

本博客分成三部分：

*  part1是论文笔记，主要是概括论文的主要内容；
 *  part2采用问答的方式来解答可能对论文产生疑问的地方；
 *  part3部分是对如何实现论文的细节探讨。

# PART1论文笔记 #

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center]![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 1]![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 2]这意思就是：可以通过添加细粒度的实体，从而将一个discontinuous NER 问题转换成一个 nested NER 问题。下面以文中的例子来说明这个问题：  
![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 1][]重新添加：`Body Location` `General Feeling` 两个细粒度的实体，加上原本的`Adverse drug event`实体名，那么就可以将之前单纯的`Discontinuous NER`变成了一个`Nested NER`问题。这也就说明二者本质上是可以相互转换的（作者应该就是想表示这个意思）

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 2]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 3]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 3]这里先假设向量 c i c\_i ci的维度是n。

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 4]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 5]  
下面详细说一下 s i a = s o f t m a x ( s i T W i a B ) B s\_i^a = softmax(s\_i^TW\_i^aB)B sia=softmax(siTWiaB)B 公式中各个变量的含义：

*   s i s\_i si is the vector representation of a span, so you can imagine  s i ∈ R d s\_i \\in R^d si∈Rd.
 *   B B B is a sequence of vectors, each of which represents a token in the buffer. so the shape is  B ∈ R d × l B \\in R^\{d \\times l\} B∈Rd×l 【那么这里的 l l l就是buffer 中tokens 的个数】
 *  the  W a W\_a Wa has the shape of  R d × d R^\{d \\times d\} Rd×d, the formula  S i T W a B S\_i^T W\_aB SiTWaB part will get the annotation weights, whose shape is  R 1 × l R^\{1 \\times l\} R1×l.
 *  And the final  s a s\_a sa has the shape of  R 1 × d R^\{1 \\times d\} R1×d.

作者站在attention的角度描述了一下上面这个公式，如下文：

> An alternative explanation if you are familiar with attention:  
> you can imagine  s i s\_i si is the query vector, B plays the role of both key vectors and value vectors.  
> and the  s a s\_a sa is the weighted sum of B (value vectors), whose weights are calculated based on the correlation between key ( s i s\_i si) and query vector (B).

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 6]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 4]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 5]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 6]

# PART2疑问 #

## 1. 该用什么数据集？什么部分？ ##

下载得到的 CADEC，解压后得到的文件夹如下所示：  
![在这里插入图片描述][20201108145034493.png]  
那么是用v1，还是v2?  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 7]

*  使用CADEC.v2文件夹中的数据即可。
 *  但是即使在上述的文件夹中的数据仍然数据量比较大，所以作者只取了ADE的部分  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 7]  
    但是不清楚是否是因为typo，原作者写成了 ADR。

## 2.预处理数据的作用是什么？效果是什么？ ##

待更新~

## 3.其它准备工作 ##

主要是语料库的下载。

*  需要下载elmo 模型和训练结果，下载地址为：https://allennlp.org/elmo。主要是下载weights 和options 两个文件  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 8]将下载得到的文件放到目录：/data/dai031/Corpora/ELMo/elmo\_2x4096\_512\_2048cnn\_2xhighway\_5.5B 下。
 *  需要下载glove算法预训练得到的embedding信息，也就是文件**glove.6B.100d.txt**：可以在 https://www.kaggle.com/danielwillgeorge/glove6b100dtxt 中下载得到。

# PART3代码实现 #

在加载文件 **glove.6B.100d.txt** 时不动代码卡住不动了：  
![在这里插入图片描述][20201108165303208.png_pic_center]这个问题该怎么解决呢？

> 发现自己的确是个zz（原作者也不按常理出牌啊），把日志直接写到文件中了，而没有在窗口中输出。

文件地址如下：  
![在这里插入图片描述][20201108205912965.png_pic_center]日志输出过程（训练有点儿慢）如下：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 8]

[GitHub]: https://github.com/LawsonAbs/discontinuousNER
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center]: https://img-blog.csdnimg.cn/20201108205244443.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70]: /images/20221120/2292f3f374dc42eeba8ce68e7f746c18.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20201108205418725.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 2]: /images/20221120/9339846b5abc4729a3adfbc0670d75b3.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 1]: https://img-blog.csdnimg.cn/20201120194007586.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 2]: https://img-blog.csdnimg.cn/20201108205448334.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 3]: https://img-blog.csdnimg.cn/20201108205530850.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20201108205552593.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/2020110820560499.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20201108205628986.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20201108205647443.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 4]: https://img-blog.csdnimg.cn/20201108205703396.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 5]: https://img-blog.csdnimg.cn/20201108205723287.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 6]: https://img-blog.csdnimg.cn/20201108205739482.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#pic_center
[20201108145034493.png]: https://img-blog.csdnimg.cn/20201108145034493.png#
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 7]: https://img-blog.csdnimg.cn/20201108144955836.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 7]: https://img-blog.csdnimg.cn/20201108152211802.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70 8]: https://img-blog.csdnimg.cn/20201108163458382.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#
[20201108165303208.png_pic_center]: https://img-blog.csdnimg.cn/20201108165303208.png#pic_center
[20201108205912965.png_pic_center]: https://img-blog.csdnimg.cn/20201108205912965.png#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5_size_16_color_FFFFFF_t_70_pic_center 8]: https://img-blog.csdnimg.cn/2020110820595842.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdTE2NjU5,size_16,color_FFFFFF,t_70#pic_center