pyspark java.net.SocketException: Connection reset by peer-蒲公英云

pyspark java.net.SocketException: Connection reset by peer

在window、运行pyspark训练模型，报错
Caused by: java.net.SocketException: Connection reset by peer: socket write error

py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.runJob.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 1, localhost, executor driver): java.net.SocketException: Connection reset by peer: socket write error
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
    ...
Caused by: java.net.SocketException: Connection reset by peer: socket write error
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
    ...

解决：修改python环境下的pyspark包中的worker. py 中的process函数

（注：python环境下的pyspark包是从spark目录拷贝过来的）
eg: E:\Anaconda3\envs\py27\Lib\site-packages\pyspark\worker. py
定位到 process 函数：

def process():
            iterator = deserializer.load_stream(infile)
            serializer.dump_stream(func(split_index, iterator), outfile)
            # 添加以下内容
            for obj in interator:
                pass