Python+OpenCV：立体图像深度图（Depth Map from Stereo Images）

青旅半醒 2022-12-24 10:54 134阅读 0赞

## Python+OpenCV：立体图像深度图（Depth Map from Stereo Images） ##

### 理论 ###

If we have two images of same scene, we can get depth information from that in an intuitive way.

Below is an image and some simple mathematical formulas which prove that intuition.

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdWJpbmc4NjA5_size_16_color_FFFFFF_t_70][]

The above diagram contains equivalent triangles. Writing their equivalent equations will yield us following result:

![20201127195246998.png][]

x and x′ are the distance between points in image plane corresponding to the scene point 3D and their camera center.

B is the distance between two cameras (which we know) and f is the focal length of camera (already known).

So in short, the above equation says that the depth of a point in a scene is inversely proportional to the difference in distance of corresponding image points and their camera centers.

So with this information, we can derive the depth of all pixels in an image.

So it finds corresponding matches between two images. We have already seen how epiline constraint make this operation faster and accurate. Once it finds matches, it finds the disparity. Let's see how we can do it with OpenCV.

### Depth Map from Stereo Images in OpenCV ###

####################################################################################################
    # 立体图像深度图(Depth Map from Stereo Images)
    def lmc_cv_depth_map():
        """
            函数功能: 立体图像深度图(Depth Map from Stereo Images). 
        """
    
        image1 = lmc_cv.imread('D:/99-Research/TestData/cv/stereo/uncalib/board_l.png',
                               lmc_cv.IMREAD_GRAYSCALE)
        image2 = lmc_cv.imread('D:/99-Research/TestData/cv/stereo/uncalib/board_r.png',
                               lmc_cv.IMREAD_GRAYSCALE)
    
        # create a disparity map
        # numDisparities: 最大视差值与最小视差值之差, 窗口大小必须是16的整数倍, int 型.
        # blockSize: 匹配的块大小. 它必须是 >=1的奇数. 通常情况下, 它应该在5-11的范围内. 这里设置为大于11也可以, 但必须为奇数.
        stereo = lmc_cv.StereoBM_create(numDisparities=16, blockSize=17)
        disparity = stereo.compute(image1, image2)
    
        stacking_images = []
        # stacking images side-by-side
        stacking_image = np.hstack((image1, image2, disparity))
        stacking_images.append(stacking_image)
    
        # 显示图像
        for i in range(len(stacking_images)):
            pyplot.figure('Depth Map from Stereo Images %d' % (i + 1), figsize=(16, 9))
            # pyplot.subplot(1, 1, 1)
            pyplot.imshow(stacking_images[i], 'gray')
            pyplot.title('Depth Map from Stereo Images')
            pyplot.xticks([])
            pyplot.yticks([])
            pyplot.savefig('%02d.png' % (i + 1))
        pyplot.show()

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdWJpbmc4NjA5_size_16_color_FFFFFF_t_70 1][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdWJpbmc4NjA5_size_16_color_FFFFFF_t_70 2][]

Below image contains the original image (left) and its disparity map (right). As you can see, the result is contaminated with high degree of noise.

By adjusting the values of numDisparities and blockSize, you can get a better result.

There are some parameters when you get familiar with StereoBM, and you may need to fine tune the parameters to get better and smooth results.

Parameters:

*  texture\_threshold: filters out areas that don't have enough texture for reliable matching.
 *  Speckle range and size: Block-based matchers often produce "speckles" near the boundaries of objects, where the matching window catches the foreground on one side and the background on the other. In this scene it appears that the matcher is also finding small spurious matches in the projected texture on the table. To get rid of these artifacts we post-process the disparity image with a speckle filter controlled by the speckle\_size and speckle\_range parameters. speckle\_size is the number of pixels below which a disparity blob is dismissed as "speckle." speckle\_range controls how close in value disparities must be to be considered part of the same blob.
 *  Number of disparities: How many pixels to slide the window over. The larger it is, the larger the range of visible depths, but more computation is required.
 *  min\_disparity: the offset from the x-position of the left pixel at which to begin searching.
 *  uniqueness\_ratio: Another post-filtering step. If the best matching disparity is not sufficiently better than every other disparity in the search range, the pixel is filtered out. You can try tweaking this if texture\_threshold and the speckle filtering are still letting through spurious matches.
 *  prefilter\_size and prefilter\_cap: The pre-filtering phase, which normalizes image brightness and enhances texture in preparation for block matching. Normally you should not need to adjust these.

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdWJpbmc4NjA5_size_16_color_FFFFFF_t_70]: /images/20221120/7b7a946a2db44b60a5a041d928d2f7ad.png
[20201127195246998.png]: https://img-blog.csdnimg.cn/20201127195246998.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdWJpbmc4NjA5_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20201127201505704.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdWJpbmc4NjA5,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdWJpbmc4NjA5_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/2020112720151678.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdWJpbmc4NjA5,size_16,color_FFFFFF,t_70