opencv-python画局部放大图

爱被打了一巴掌 2023-02-28 11:08 3阅读 0赞

### 文章目录 ###

*  为什么要画局部放大图？
 *  程序逻辑
 *  程序实例

这项功能的目的是为了方便使用opencv做图像标注工具。

# 为什么要画局部放大图？ #

在做图像数据标注时，很难一次就做到精准标注，经常需要微调才能达到比较好的标注效果。如果目标比较小，即使微调也难以做到精准，所以就需要另外一个窗口对标注区域进行局部放大以方便微调。

# 程序逻辑 #

本文中标注信息以矩形框作为示例，矩形框是图像标注中最常用到的一种标注信息形态。其他标注信息的设计逻辑雷同。

程序主要逻辑是：**鼠标在任意窗口中做的操作都要同步映射到另外一个窗口**。主要通过两个维度进行控制：1. 鼠标在主窗口还是在放大窗口；2. 鼠标的操作。其中主窗口指的是用来显示完整大图的窗口，放大窗口是用来显示局部放大区域的窗口。

*  鼠标在主窗口
    
     *  鼠标移动：以`当前点（current_pt）`为中心显示一块放大区域，显示的范围由一个参数指定`default_zoom_image_size`。为了确保以current\_pt为中心向四个方向均能取到default\_zoom\_image\_size一半的值，current\_pt的活动范围是大图减去一定的边缘区域，这个边缘区域就是default\_zoom\_image\_size的一半。在活动范围内，current\_pt等同于鼠标当前点，在边缘区域，current\_pt等于距离鼠标当前点最近的边缘点。
     *  鼠标画矩形框：左键按下并拖动可以画矩形框，矩形框不会超出大图边界
     *  删除矩形框：有两种情况会删除矩形框 —— 1. 左键画出来的矩形框面积为零；2. 右键点击
 *  鼠标在放大窗口
    
     *  鼠标移动：什么也不做。
     *  鼠标画矩形框：左键按下并拖动可以画矩形框，矩形框可以超出放大窗口的边界，但是不会超出大图边界
     *  删除矩形框：同主窗口的两种情况
     *  鼠标左键或右键点击：可以起到选择current\_pt的作用

总体来说，我们在主窗口做任何操作都可以比较随意，因为主窗口中除了矩形框外，图像本身不发生变化。而在放大窗口中做操作就需要相对保守一些，不然会非常乱，以至于无法操作。所以我们在主窗口中画矩形框时，放大窗口会随着矩形框不停改变；而我们在放大窗口中画矩形框时，放大窗口则保持不变（不然眼花缭乱）。

# 程序实例 #

把程序逻辑搞明白后下面代码就比较容易懂了。细节就不多说了，只说一个需要注意的大方向：我们需要为两个窗口各写一个鼠标回调函数。  
先看一下单纯画矩形框的代码可能会有助于理解下面程序中的一些细节：[opencv-python鼠标画矩形框：cv2.rectangle()][opencv-python_cv2.rectangle]

# -*- coding: utf-8 -*-
    import cv2
    
    
    class Rect(object):
        def __init__(self, pt1=(0, 0), pt2=(0, 0)):
            self.tl = pt1
            self.br = pt2
            self.regularize()
    
        def regularize(self):
            """ make sure tl = TopLeft point, br = BottomRight point """
            tl = (min(self.tl[0], self.br[0]), min(self.tl[1], self.br[1]))
            br = (max(self.tl[0], self.br[0]), max(self.tl[1], self.br[1]))
            self.tl = tl
            self.br = br
    
        def get_center(self):
            """ get center point of Rect """
            center_x = (self.tl[0] + self.br[0]) // 2
            center_y = (self.tl[1] + self.br[1]) // 2
            return center_x, center_y
    
        def get_width(self):
            """ get width of Rect """
            return abs(self.br[0] - self.tl[0])
    
        def get_height(self):
            """ get height of Rect """
            return abs(self.br[1] - self.tl[1])
    
        def height_over_width(self):
            """ ratio of height over width """
            return self.get_height() / self.get_width()
    
        def get_area(self):
            """ get area of Rect """
            return self.get_width() * self.get_height()
    
    
    class DrawZoom(object):
        def __init__(self, image, color,
                     current_pt=(0, 0),
                     default_zoom_image_size=(256, 256)):
            self.original_image = image
            self.color = color
            self.thickness = 2
            self.current_pt = current_pt
            self.default_zoom_image_size = default_zoom_image_size
    
            self.rect_in_big_image = Rect()
            self.rect_in_zoom_image = Rect()
            self.zoom_offset = (0, 0)
    
            self.is_drawing_big = False
            self.is_drawing_zoom = False
            self.exist_rect = False
    
            self.big_image = image.copy()
            self.zoom_image = None
            self.zoom_image_backup = None
            self.get_zoom_image()
    
        def get_image_height(self):
            """ get height of big image """
            return self.original_image.shape[0]
    
        def get_image_width(self):
            """ get width of big image """
            return self.original_image.shape[1]
    
        def get_margin_height(self):
            """ get height of margin. in the margin area of big image, coordinate of current_pt does NOT change """
            return self.default_zoom_image_size[0] // 2
    
        def get_margin_width(self):
            """ get width of margin """
            return self.default_zoom_image_size[1] // 2
    
        def get_zoom_image(self, height_ratio_expand=0.2, width_ratio_expand=0.2):
            """ get zoom image for two cases: the rect exists or not. height_ratio_expand and width_ratio_expand are used for expanding some area of rect """
            if not self.exist_rect:
                self.get_zoom_image_for_current_pt()
            elif self.rect_in_big_image.get_area() > 0:
                self.get_zoom_image_for_rect(height_ratio_expand,
                                             width_ratio_expand)
    
        def get_zoom_image_for_current_pt(self):
            """ get zoom image for current mouse point (when rect does not exist) """
            # (x, y) is center coordinate
            x = max(self.current_pt[0], self.get_margin_width())
            x = min(x, self.get_image_width() - self.get_margin_width())
            y = max(self.current_pt[1], self.get_margin_height())
            y = min(y, self.get_image_height() - self.get_margin_height())
    
            tl_x = x - self.get_margin_width()
            tl_y = y - self.get_margin_height()
            br_x = x + self.get_margin_width()
            br_y = y + self.get_margin_height()
            tl_x, tl_y, br_x, br_y = self.shrink_rect(tl_x, tl_y, br_x, br_y)
            self.zoom_image = self.big_image[tl_y:br_y, tl_x:br_x]
            self.zoom_image_backup = self.original_image[tl_y:br_y, tl_x:br_x]
            self.zoom_offset = (tl_x, tl_y)
    
        def get_zoom_image_for_rect(self, height_ratio_expand, width_ratio_expand):
            """ get zoom image when rect exists """
            if self.rect_in_big_image.get_area() == 0:
                return None
            height_over_width_for_win_zoom = \
                self.default_zoom_image_size[1] / self.default_zoom_image_size[0]
            center = self.rect_in_big_image.get_center()
            if self.rect_in_big_image.height_over_width() > \
                    height_over_width_for_win_zoom:
                half_height = int(0.5 * (1 + height_ratio_expand) *
                                  self.rect_in_big_image.get_height())
                half_width = int(half_height / height_over_width_for_win_zoom)
            else:
                half_width = int(0.5 * (1 + width_ratio_expand) *
                                 self.rect_in_big_image.get_width())
                half_height = int(half_width * height_over_width_for_win_zoom)
            tl_x = center[0] - half_width
            tl_y = center[1] - half_height
            br_x = center[0] + half_width
            br_y = center[1] + half_height
            tl_x, tl_y, br_x, br_y = self.shrink_rect(tl_x, tl_y, br_x, br_y)
            self.zoom_image = self.big_image[tl_y:br_y, tl_x:br_x]
            self.zoom_image_backup = self.original_image[tl_y:br_y, tl_x:br_x]
            self.zoom_offset = (tl_x, tl_y)
    
        @staticmethod
        def clip(value, low, high):
            """ clip value between low and high """
            output = max(value, low)
            output = min(output, high)
            return output
    
        def shrink_point(self, x, y):
            """ shrink point (x, y) to inside big image """
            x_shrink = self.clip(x, 0, self.get_image_width())
            y_shrink = self.clip(y, 0, self.get_image_height())
            return x_shrink, y_shrink
    
        def shrink_rect(self, pt1_x, pt1_y, pt2_x, pt2_y):
            """ shrink rect to inside big image """
            pt1_x = self.clip(pt1_x, 0, self.get_image_width())
            pt1_y = self.clip(pt1_y, 0, self.get_image_height())
            pt2_x = self.clip(pt2_x, 0, self.get_image_width())
            pt2_y = self.clip(pt2_y, 0, self.get_image_height())
            rect = Rect((pt1_x, pt1_y), (pt2_x, pt2_y))
            rect.regularize()
            tl_x, tl_y = rect.tl
            br_x, br_y = rect.br
            return tl_x, tl_y, br_x, br_y
    
        def reset_big_image(self):
            """ reset big_image (for show) using original image """
            self.big_image = self.original_image.copy()
    
        def reset_zoom_image(self):
            """ reset zoom_image (for show) using the zoom image backup """
            self.zoom_image = self.zoom_image_backup.copy()
    
        def draw_rect_in_big_image(self):
            """ draw rect in big image """
            cv2.rectangle(self.big_image,
                          self.rect_in_big_image.tl, self.rect_in_big_image.br,
                          color=self.color, thickness=self.thickness)
    
        def draw_rect_in_zoom_image(self):
            """ draw rect in zoom image """
            cv2.rectangle(self.zoom_image,
                          self.rect_in_zoom_image.tl, self.rect_in_zoom_image.br,
                          color=self.color, thickness=self.thickness)
    
        def update_drawing_big(self):
            """ update drawing big image, and map the corresponding area to zoom image """
            if self.exist_rect:
                self.draw_rect_in_big_image()
            self.get_zoom_image()
    
        def update_drawing_zoom(self):
            """ update drawing big and zoom image when drawing rect in zoom image """
            if self.exist_rect:
                self.draw_rect_in_big_image()
                self.draw_rect_in_zoom_image()
    
    
    def onmouse_big_image(event, x, y, flags, draw_zoom):
        if event == cv2.EVENT_LBUTTONDOWN:
            # pick first point of rect
            draw_zoom.is_drawing_big = True
            draw_zoom.rect_in_big_image.tl = (x, y)
            draw_zoom.exist_rect = True
        elif draw_zoom.is_drawing_big and event == cv2.EVENT_MOUSEMOVE:
            # pick second point of rect and draw current rect
            draw_zoom.rect_in_big_image.br = draw_zoom.shrink_point(x, y)
            draw_zoom.reset_big_image()
            draw_zoom.update_drawing_big()
        elif event == cv2.EVENT_LBUTTONUP:
            # finish drawing current rect
            draw_zoom.is_drawing_big = False
            draw_zoom.rect_in_big_image.br = draw_zoom.shrink_point(x, y)
            draw_zoom.rect_in_big_image.regularize()
            if draw_zoom.rect_in_big_image.get_area() == 0:
                draw_zoom.reset_big_image()
                draw_zoom.rect_in_big_image = Rect()
                draw_zoom.exist_rect = False
            draw_zoom.update_drawing_big()
        elif (not draw_zoom.is_drawing_big) and event == cv2.EVENT_RBUTTONDOWN:
            # right button down to erase current rect
            draw_zoom.rect_in_big_image = Rect()
            draw_zoom.exist_rect = False
            draw_zoom.reset_big_image()
            draw_zoom.update_drawing_big()
        else:
            # default case: mouse move without rect
            draw_zoom.current_pt = (x, y)
            draw_zoom.update_drawing_big()
    
    
    def onmouse_zoom_image(event, x, y, flags, draw_zoom):
        if event == cv2.EVENT_LBUTTONDOWN:
            # pick first point of rect
            draw_zoom.is_drawing_zoom = True
            draw_zoom.rect_in_zoom_image.tl = (x, y)
            draw_zoom.rect_in_big_image.tl = (x + draw_zoom.zoom_offset[0],
                                              y + draw_zoom.zoom_offset[1])
            draw_zoom.exist_rect = True
        elif draw_zoom.is_drawing_zoom and event == cv2.EVENT_MOUSEMOVE:
            # pick second point of rect and draw current rect
            draw_zoom.rect_in_zoom_image.br = (x, y)
            draw_zoom.rect_in_big_image.br = draw_zoom.shrink_point(
                x + draw_zoom.zoom_offset[0], y + draw_zoom.zoom_offset[1])
            draw_zoom.reset_zoom_image()
            draw_zoom.reset_big_image()
            draw_zoom.update_drawing_zoom()
        elif event == cv2.EVENT_LBUTTONUP:
            # finish drawing current rect
            draw_zoom.is_drawing_zoom = False
            draw_zoom.rect_in_big_image.br = draw_zoom.shrink_point(
                x + draw_zoom.zoom_offset[0], y + draw_zoom.zoom_offset[1])
            draw_zoom.rect_in_big_image.regularize()
            if draw_zoom.rect_in_big_image.get_area() == 0:
                draw_zoom.reset_big_image()
                draw_zoom.rect_in_big_image = Rect()
                draw_zoom.rect_in_zoom_image = Rect()
                draw_zoom.exist_rect = False
                draw_zoom.current_pt = draw_zoom.shrink_point(
                    x + draw_zoom.zoom_offset[0], y + draw_zoom.zoom_offset[1])
            draw_zoom.update_drawing_big()
        elif (not draw_zoom.is_drawing_big) and event == cv2.EVENT_RBUTTONDOWN:
            # right button down to erase current rect
            draw_zoom.rect_in_big_image = Rect()
            draw_zoom.rect_in_zoom_image = Rect()
            draw_zoom.exist_rect = False
            draw_zoom.reset_big_image()
            draw_zoom.current_pt = draw_zoom.shrink_point(
                x + draw_zoom.zoom_offset[0], y + draw_zoom.zoom_offset[1])
            draw_zoom.update_drawing_big()
        else:
            # mousemove in zoom image will not change the content of image
            pass
    
    
    if __name__ == '__main__':
        WIN_NAME_BIG = 'big_image'
        WIN_NAME_ZOOM = 'zoom_image'
        image = cv2.imread('1.jpg')
        draw_zoom = DrawZoom(image, (0, 255, 0))
        cv2.namedWindow(WIN_NAME_BIG, 0)
        cv2.namedWindow(WIN_NAME_ZOOM, 0)
        cv2.setMouseCallback(WIN_NAME_BIG, onmouse_big_image, draw_zoom)
        cv2.setMouseCallback(WIN_NAME_ZOOM, onmouse_zoom_image, draw_zoom)
        while True:
            cv2.imshow(WIN_NAME_BIG, draw_zoom.big_image)
            cv2.imshow(WIN_NAME_ZOOM, draw_zoom.zoom_image)
            key = cv2.waitKey(30)
            if key == 27:  # ESC
                break
        cv2.destroyAllWindows()

结果：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JieTE5ODc_size_16_color_FFFFFF_t_70]

[opencv-python_cv2.rectangle]: https://blog.csdn.net/bby1987/article/details/107302398
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JieTE5ODc_size_16_color_FFFFFF_t_70]: /images/20230209/9d8728bd1a994703a3cebd70b6c19975.png