【自然语言处理】tf.contrib.seq2seq.GreedyEmbeddingHelper源码解析

待我称王封你为后i 2021-11-02 00:16 171阅读 0赞

## 前言 ##

本文衔接[TrainingHelper][]，也可以衔接[BasicDecoder][]。先说明一下，GreedyEmbeddingHelper主要作用是接收开始符，然后生成指定长度大小的句子。

## 正文 ##

[GreedyEmbeddingHelper代码传送门][GreedyEmbeddingHelper]

class GreedyEmbeddingHelper(Helper):
      """A helper for use during inference.
      Uses the argmax of the output (treated as logits) and passes the
      result through an embedding layer to get the next input.
      """
    
      def __init__(self, embedding, start_tokens, end_token):
        """Initializer.
        Args:
          embedding: A callable that takes a vector tensor of `ids` (argmax ids),
            or the `params` argument for `embedding_lookup`. The returned tensor
            will be passed to the decoder input.
          start_tokens: `int32` vector shaped `[batch_size]`, the start tokens.
          end_token: `int32` scalar, the token that marks end of decoding.
        Raises:
          ValueError: if `start_tokens` is not a 1D tensor or `end_token` is not a
            scalar.
        """
        if callable(embedding):
          self._embedding_fn = embedding
        else:
          self._embedding_fn = (
              lambda ids: embedding_ops.embedding_lookup(embedding, ids))
    
        self._start_tokens = ops.convert_to_tensor(
            start_tokens, dtype=dtypes.int32, name="start_tokens")
        self._end_token = ops.convert_to_tensor(
            end_token, dtype=dtypes.int32, name="end_token")
        if self._start_tokens.get_shape().ndims != 1:
          raise ValueError("start_tokens must be a vector")
        self._batch_size = array_ops.size(start_tokens)
        if self._end_token.get_shape().ndims != 0:
          raise ValueError("end_token must be a scalar")
        self._start_inputs = self._embedding_fn(self._start_tokens)

在GreedyEmbeddingHelper初始阶段，接收一个embedding矩阵，以便后面的embedding\_lookup。可以注意到在TrainingHelper并不需要这个，是因为在训练阶段，我们给TrainingHelper的就是\[batch\_size, seq\_len, embed\_size\]的输入，已经是词向量了。而在推理阶段，我们只给了一个开始符，给了我们需要的句子长度，所以我们在输出一个词的时候还需要进行embedding\_lookup成词向量作为下一个时刻的输入。

def initialize(self, name=None):
        finished = array_ops.tile([False], [self._batch_size])
        return (finished, self._start_inputs)

第一个输入，在TrainingHelper的第一个输入是inputs\[0\]，而这里的第一个输入是开始符向量（注意开始符是一个\[batch\_size\]的向量，里面的元素不一定都一样。因为有时候我们可能在生成到一半的句子中才开始推理，这时候的第一个开始符生成一半句子的最后一个词）。当然，这里的finished肯定是都是False的。

def sample(self, time, outputs, state, name=None):
        """sample for GreedyEmbeddingHelper."""
        del time, state  # unused by sample_fn
        # Outputs are logits, use argmax to get the most probable id
        if not isinstance(outputs, ops.Tensor):
          raise TypeError("Expected outputs to be a single Tensor, got: %s" %
                          type(outputs))
        sample_ids = math_ops.argmax(outputs, axis=-1, output_type=dtypes.int32)
        return sample_ids

这里是采样的意思，判断一个词根据什么情况来在这里，Greedy是贪婪的意思，也就是这个采样遵循贪心算法，选取最大概率输出对应词作为采样的词。

def next_inputs(self, time, outputs, state, sample_ids, name=None):
        """next_inputs_fn for GreedyEmbeddingHelper."""
        del time, outputs  # unused by next_inputs_fn
        finished = math_ops.equal(sample_ids, self._end_token)
        all_finished = math_ops.reduce_all(finished)
        next_inputs = control_flow_ops.cond(
            all_finished,
            # If we're finished, the next_inputs value doesn't matter
            lambda: self._start_inputs,
            lambda: self._embedding_fn(sample_ids))
        return (finished, next_inputs, state)

但是GreedyEmbeddingHelper其实也关注next\_inputs，因为上一个采样的词需要当成当前的输入。

## 总结 ##

Helper类型很多，SampleEmbeddingHelper，CustomHelper，ScheduledEmbeddingTrainingHelper，ScheduledOutputTrainingHelper，InferenceHelper，其实大多大同小异，学会了训练阶段的Helper和推理阶段的Helper的典型，也就是上面两个，就可以触类旁通。  
全部的代码在[Helper.py][]这里，有需要延伸的可以继续看看。

[TrainingHelper]: https://blog.csdn.net/tudaodiaozhale/article/details/93382356
[BasicDecoder]: https://blog.csdn.net/tudaodiaozhale/article/details/90813573
[GreedyEmbeddingHelper]: https://github.com/tensorflow/tensorflow/blob/r1.14/tensorflow/contrib/seq2seq/python/ops/helper.py#L554-L628
[Helper.py]: https://github.com/tensorflow/tensorflow/blob/r1.14/tensorflow/contrib/seq2seq/python/ops/helper.py