python3 多进程 multiprocessing 报错 AttributeError: ‘ForkAwareLocal‘ object has no attribute ‘connection‘

喜欢ヅ旅行 2022-09-12 04:51 71阅读 0赞

**目录**

错误代码

报错信息

错误原因

解决方法

--------------------

### 错误代码 ###

python多进程管理manager时候，当不使用join对当前进程（主进程）进行阻塞时会报错，具体代码及错误如下：

from multiprocessing import Process, Manager
    import time
    import os
     
     
    def info(title):
        print(title)
        print('module name:', __name__)
        print('parent process:', os.getppid())
        print('process id:', os.getpid())
        print("\n\n")
     
     
     
    def f(d, l,n):
        info('\033[32;1m subprocess line\033[0m')
        d[1] = '1'
        d['2'] = 2
        d[0.25] = None
        l.append(n)
        print(l)
     
     
    if __name__ == '__main__':
        info('\033[32;1mmain process line\033[0m')
        with Manager() as manager:
            d = manager.dict()
     
            l = manager.list(range(5))
            p_list = []
            for i in range(10):
                p = Process(target=f, args=(d, l,i))
                p_list.append(p)
                # p.join()
            for p in p_list:
                p.start()

### 报错信息 ###

Process Process-11:
    Traceback (most recent call last):
      File "/usr/lib/python3.4/multiprocessing/managers.py", line 724, in _callmethod
        conn = self._tls.connection
    AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
        self.run()
      File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "managerTest.py", line 19, in f
        d[1] = '1'
      File "<string>", line 2, in __setitem__
      File "/usr/lib/python3.4/multiprocessing/managers.py", line 728, in _callmethod
        self._connect()
      File "/usr/lib/python3.4/multiprocessing/managers.py", line 715, in _connect
        conn = self._Client(self._token.address, authkey=self._authkey)
      File "/usr/lib/python3.4/multiprocessing/connection.py", line 495, in Client
        c = SocketClient(address)
      File "/usr/lib/python3.4/multiprocessing/connection.py", line 624, in SocketClient
        s.connect(address)
    FileNotFoundError: [Errno 2] No such file or directory

### 错误原因 ###

启动工作进程后 主进程立即就结束了，创建multiprocessing.Manager的进程完成执行后，Manager服务器将关闭，这意味着您的共享列表对象现在无用了。

发生这种情况是因为Manager对象将其shutdown函数注册为multiprocessing模块的" finalizer"，这意味着它将在进程退出之前运行。这是在BaseManager.\_\_init\_\_中注册它的代码：

# register a finalizer
    self._state.value = State.STARTED
    self.shutdown = util.Finalize(
        self, type(self)._finalize_manager,
        args=(self._process, self._address, self._authkey,
              self._state, self._Client),
        exitpriority=0
        )

这是实际执行关闭操作的代码：

@staticmethod
    def _finalize_manager(process, address, authkey, state, _Client):
        '''
        Shutdown the manager process; will be registered as a finalizer
        '''
        if process.is_alive():
            util.info('sending shutdown message to manager')
            try:
                conn = _Client(address, authkey=authkey)
                try:
                    dispatch(conn, None, 'shutdown')
                finally:
                    conn.close()
            except Exception:
                pass
    
            process.join(timeout=1.0)
            if process.is_alive():
                util.info('manager still alive')
                if hasattr(process, 'terminate'):
                    util.info('trying to `terminate()` manager process')
                    process.terminate()
                    process.join(timeout=0.1)
                    if process.is_alive():
                        util.info('manager still alive after terminate')
    
        state.value = State.SHUTDOWN
        try:
            del BaseProxy._address_to_local[address]
        except KeyError:
            pass

### 解决方法 ###

在主进程中 对子进程进行 join 等待