NCHW - NHWC - NC/32HW32

小咪咪 2023-01-07 07:13 151阅读 0赞

# **NCHW - NHWC - NC/32HW32** #

## 1. 4-D Tensor Descriptor ##

A 4-D Tensor descriptor is used to define the format for batches of 2D images with 4 letters: N, C, H, W for respectively the batch size, the number of feature maps, the height and the width. The letters are sorted in decreasing order of the strides.  
使用 4-D 张量描述符来定义具有 4 个字母的 2D 图像批处理的格式：N, C, H, W 分别表示批处理大小，特征图的数量，高度和宽度。字母按降序排列。

The commonly used 4-D tensor formats are:

NCHW
    NHWC
    CHWN

数据布局由二维图像的四个字母表示。  
N：批次大小，表示一个批次中的图像数量。  
C：通道，表示一张图像中的通道数。  
W：宽度，表示图像水平维度的像素数。  
H：高度，表示图像垂直维度的像素数。

## 2. 5-D Tensor Description ##

A 5-D Tensor descriptor is used to define the format of the batch of 3D images with 5 letters: N, C, D, H, W for respectively the batch size, the number of feature maps, the depth, the height, and the width. The letters are sorted in decreasing order of the strides.  
5-D Tensor 描述符用于定义 5 个字母的 3D 图像批处理的格式：N, C, D, H, W 分别表示批处理大小，特征图的数量，深度，高度和宽度。字母按降序排列。

The commonly used 5-D tensor formats are called:

NCDHW
    NDHWC
    CDHWN

## 3. Data Layout Formats ##

Consider a batch of images in 4D with the following dimensions:

*  N is the batch size; 1.
 *  C is the number of feature maps (i.e., number of channels); 64.
 *  H is the image height; 5.
 *  W is the image width; 4.

To keep the example simple, the image pixel elements are expressed as a sequence of integers, 0, 1, 2, 3, and so on.

**Figure 1. Example with N=1, C=64, H=5, W=4.**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY_size_16_color_FFFFFF_t_70_pic_center]

### 3.1 NCHW Memory Layout ###

The above 4D Tensor is laid out in the memory in the NCHW format as below:  
上面的 4D 张量以NCHW 格式放置在内存中，如下所示：

1.  Beginning with the first channel (c=0), the elements are arranged contiguously in row-major order.  
    从第一个通道 (c = 0) 开始，元素按行优先顺序连续排列。
2.  Continue with second and subsequent channels until the elements of all the channels are laid out.  
    继续第二和后续通道，直到所有通道的元素都已布置好。
3.  Proceed to the next batch (if N is > 1).  
    进行下一批 (如果 N > 1)。

**Figure 2. NCHW Memory Layout**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY_size_16_color_FFFFFF_t_70_pic_center 1]

### 3.2 NHWC Memory Layout ###

For the NHWC memory layout, the corresponding elements in all the C channels are laid out first, as below:

1.  Begin with the first element of channel 0, then proceed to the first element of channel 1, and so on, until the first elements of all the C channels are laid out.  
    从通道 0 的第一个元素开始，然后进行到通道 1 的第一个元素，依此类推，直到布置了所有 C 通道的第一个元素。
2.  Next, select the second element of channel 0, then proceed to the second element of channel 1, and so on, until the second element of all the channels are laid out.  
    接下来，选择通道 0 的第二个元素，然后继续到通道 1 的第二个元素，依此类推，直到所有通道的第二个元素都被布置好为止。
3.  Follow the row-major order of channel 0 and complete all the elements.  
    遵循通道 0 的主要行顺序，并完成所有元素。
4.  Proceed to the next batch (if N is > 1).  
    进行下一批 (如果 N > 1)。

**Figure 3. NHWC Memory Layout**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY_size_16_color_FFFFFF_t_70_pic_center 2]

## 3.3 NC/32HW32 Memory Layout ##

The NC/32HW32 is similar to NHWC, with a key difference. For the NC/32HW32 memory layout, the 64 channels are grouped into two groups of 32 channels each—first group consisting of channels c0 through c31, and the second group consisting of channels c32 through c63. Then each group is laid out using the NHWC format.  
NC/32HW32 与 NHWC 类似，但有一个关键区别。对于 NC/32HW32 内存布局，将 64 个通道分为两组，每组 32 个通道，第一组由通道 c0 至 c31 组成，第二组由通道 c32 至 c63 组成。然后使用 NHWC 格式对每个组进行布局。

**Figure 4. NC/32HW32 Memory Layout**  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY_size_16_color_FFFFFF_t_70_pic_center 3]

For the generalized NC/xHWx layout format, the following observations apply:  
对于通用的 NC/xHWx 布局格式，适用以下注意事项：

*  Only the channel dimension, C, is grouped into x channels each.  
    每个通道尺寸 C 均被分组为 x 个通道。
 *  When x = 1, each group has only one channel. Hence, the elements of one channel (i.e, one group) are arranged contiguously (in the row-major order), before proceeding to the next group (i.e., next channel). This is the same as NCHW format.  
    当 x = 1 时，每个组只有一个通道。因此，在进行下一个组 (即，下一个通道) 之前，一个通道 (即，一组)的元素被连续地排列 (以行的主要顺序)。这与 NCHW 格式相同。
 *  When x = C, then NC/xHWx is identical to NHWC, i.e., the entire channel depth C is considered as a single group. The case x = C can be thought of as vectorizing the entire C dimension as one big vector, laying out all the Cs, followed by the remaining dimensions, just like NHWC.  
    当 x = C 时，NC/xHWx 与 NHWC 相同，即，整个通道深度 C 被视为一个组。可以将 x = C 的情况视为将整个 C 维向量矢量化为一个大向量，将所有 C 布局在一起，然后排列其余维，就像 NHWC 一样。
 *  The tensor format CUDNN\_TENSOR\_NCHW\_VECT\_C can also be interpreted in the following way: The NCHW INT8x32 format is really N x (C/32) x H x W x 32 (32 Cs for every W), just as the NCHW INT8x4 format is N x (C/4) x H x W x 4 (4 Cs for every W). Hence the “VECT\_C” name - each W is a vector (4 or 32) of Cs.  
    张量格式 CUDNN\_TENSOR\_NCHW\_VECT\_C 也可以按以下方式解释：NCHW INT8x32 格式实际上是 N x (C/32) x H x W x 32 (32 Cs for every W)，就像 NCHW INT8x4 格式是 N x (C/4) x H x W x 4 (4 Cs for every W)。因此，the “VECT\_C” name - each W is a vector (4 or 32) of Cs.

## References ##

[https://docs.nvidia.com/deeplearning/cudnn/developer-guide/index.html][https_docs.nvidia.com_deeplearning_cudnn_developer-guide_index.html]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY_size_16_color_FFFFFF_t_70_pic_center]: /images/20221119/87e981cef0fe4045a151f7c955535c07.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY_size_16_color_FFFFFF_t_70_pic_center 1]: https://img-blog.csdnimg.cn/20210115234206204.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY_size_16_color_FFFFFF_t_70_pic_center 2]: https://img-blog.csdnimg.cn/20210115234217580.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY_size_16_color_FFFFFF_t_70_pic_center 3]: https://img-blog.csdnimg.cn/20210115234235210.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2NoZW5neXExMTY=,size_16,color_FFFFFF,t_70#pic_center
[https_docs.nvidia.com_deeplearning_cudnn_developer-guide_index.html]: https://docs.nvidia.com/deeplearning/cudnn/developer-guide/index.html