faster-rcnn网络数据集的获取逻辑

红太狼 2023-06-16 02:00 12阅读 0赞

参考[tf-faster-rcnn][]工程的[layer.py][]函数，整体逻辑为：

1.  shuffle图片(roidb)顺序；
2.  初始化batch的起始位置，读取配置文件中的batch(faster rcnn检测的batch为1);
3.  根据起始位置和batch获得当前训练的图片；
4.  迭代值起始位置等于图片总数时，重新shuffle图片顺序；
5.  迭代值达到max\_iters时停止读取数据；

def __init__(self, roidb, num_classes, random=False):
        """Set the roidb to be used by this layer during training."""
        self._roidb = roidb
        self._num_classes = num_classes
        # Also set a random flag
        self._random = random
        self._shuffle_roidb_inds()
    
      def _shuffle_roidb_inds(self):
        """Randomly permute the training roidb."""
        # If the random flag is set, 
        # then the database is shuffled according to system time
        # Useful for the validation set
        if self._random:  #验证集时设置随机种子，保证shuffle的顺序一致
          st0 = np.random.get_state()
          millis = int(round(time.time() * 1000)) % 4294967295
          np.random.seed(millis)
        
        if cfg.TRAIN.ASPECT_GROUPING: #默认为False
          widths = np.array([r['width'] for r in self._roidb])
          heights = np.array([r['height'] for r in self._roidb])
          horz = (widths >= heights)
          vert = np.logical_not(horz)
          horz_inds = np.where(horz)[0]
          vert_inds = np.where(vert)[0]
          inds = np.hstack((
              np.random.permutation(horz_inds),
              np.random.permutation(vert_inds)))
          inds = np.reshape(inds, (-1, 2))
          row_perm = np.random.permutation(np.arange(inds.shape[0]))
          inds = np.reshape(inds[row_perm, :], (-1,))
          self._perm = inds
        else:
          self._perm = np.random.permutation(np.arange(len(self._roidb))) #shuffle roidb
        # Restore the random state
        if self._random:
          np.random.set_state(st0)
          
        self._cur = 0
    
      def _get_next_minibatch_inds(self):
        """Return the roidb indices for the next minibatch."""
        
        if self._cur + cfg.TRAIN.IMS_PER_BATCH >= len(self._roidb): #重新shuffle roidb
          self._shuffle_roidb_inds()
    
        db_inds = self._perm[self._cur:self._cur + cfg.TRAIN.IMS_PER_BATCH] #当前batch数据id
        self._cur += cfg.TRAIN.IMS_PER_BATCH
    
        return db_inds
    
      def _get_next_minibatch(self):
        """Return the blobs to be used for the next minibatch.
    
        If cfg.TRAIN.USE_PREFETCH is True, then blobs will be computed in a
        separate process and made available through self._blob_queue.
        """
        db_inds = self._get_next_minibatch_inds()
        minibatch_db = [self._roidb[i] for i in db_inds]  #当前训练数据
        return get_minibatch(minibatch_db, self._num_classes)
          
      def forward(self):
        """Get blobs and copy them into this layer's top blob vector."""
        blobs = self._get_next_minibatch()
        return blobs

问题汇总：

*  Q: IndexError: list index out of range 
 *  A: 该问题多是由于重置训练样本，导致data/cache/\{dataset\}.pkl文件中的数据量与实际数据量不符导致的。删除pkl文件，重新生成即可。
 *  记一次诡异的错误，信息如下。查看源码发现\[self.\_roidb\[i\] for i in db\_inds\]中db\_inds数据时根据self\_roidb长度采样的值(具体流程如上代码)，可运行中i值居然等于self.\_roidb的长度，导致越界。  期间由于数据本身的缘故，删除了一部分数据，然后新生成了pkl文件，可还是报错，此时的训练是基于新的pkl文件和之前训练的ckpt模型。  现重新删除模型和pkl文件训练时结果正常。

Traceback (most recent call last):
      File "./tools/trainval_net.py", line 141, in <module>
        max_iters=args.max_iters)
      File "/workspace/tf-faster-rcnn-master/tools/../lib/model/train_val.py", line 377, in train_net
        sw.train_model(sess, max_iters)
      File "/workspace/tf-faster-rcnn-master/tools/../lib/model/train_val.py", line 278, in train_model
        blobs = self.data_layer.forward()
      File "/workspace/tf-faster-rcnn-master/tools/../lib/roi_data_layer/layer.py", line 87, in forward
        blobs = self._get_next_minibatch()
      File "/workspace/tf-faster-rcnn-master/tools/../lib/roi_data_layer/layer.py", line 82, in _get_next_minibatch
        minibatch_db = [self._roidb[i] for i in db_inds]
      File "/workspace/tf-faster-rcnn-master/tools/../lib/roi_data_layer/layer.py", line 82, in <listcomp>
        minibatch_db = [self._roidb[i] for i in db_inds]
    IndexError: list index out of range

**pkl文件的查看**。工程的根目录下运行python，进入交互式环境，运行如下命令即可。

import tools._init_paths
    from model.train_val import get_training_roidb
    from model.config import cfg
    from datasets.factory import get_imdb
    import datasets.imdb
    
    def get_roidb(imdb_name):
    	imdb = get_imdb(imdb_name)
    	print('Loaded dataset `{:s}` for training'.format(imdb.name))
    	imdb.set_proposal_method(cfg.TRAIN.PROPOSAL_METHOD)
    	print('Set proposal method: {:s}'.format(cfg.TRAIN.PROPOSAL_METHOD))
    	roidb = get_training_roidb(imdb)
    	return roidb
    
    imdb_name=${DATASET}  #imdb_name为factory.py文件中__sets的keys值，如voc_2007_train
    roidb=get_roidb(imdb_name)  #得到list变量

ps：roidb中的信息如下，若设置水平翻转，翻转的数据位于所有原数据整体之后排列。

>>> len(roidbs)
    18798
    >>> roidbs[0]
    {'max_classes': array([1]), 'seg_areas': array([9065.], dtype=float32), 'boxes': array([[456, 348, 640, 396]], dtype=uint16), 'flipped': False, 'image': 'data/car_multi_part/JPEGImages/91_autohomecar__wKgH2loO01CABJgrAAIHnj70_no787.jpg', 'height': 768, 'max_overlaps': array([1.], dtype=float32), 'gt_overlaps': <1x2 sparse matrix of type '<class 'numpy.float32'>'
    	with 1 stored elements in Compressed Sparse Row format>, 'width': 1024, 'gt_classes': array([1], dtype=int32)}
    >>> roidbs[9399]      #翻转的图片位于所有的原数据后
    {'max_classes': array([1]), 'boxes': array([[383, 348, 567, 396]], dtype=uint16), 'flipped': True, 'image': 'data/car_multi_part/JPEGImages/91_autohomecar__wKgH2loO01CABJgrAAIHnj70_no787.jpg', 'height': 768, 'max_overlaps': array([1.], dtype=float32), 'gt_overlaps': <1x2 sparse matrix of type '<class 'numpy.float32'>'
    	with 1 stored elements in Compressed Sparse Row format>, 'width': 1024, 'gt_classes': array([1], dtype=int32)}

参考文献：

1.  [tf-faster-rcnn][]

[tf-faster-rcnn]: https://github.com/endernewton/tf-faster-rcnn
[layer.py]: https://github.com/endernewton/tf-faster-rcnn/blob/master/lib/roi_data_layer/layer.py