Alpine pyspark ModuleNotFoundError: No module named 'zlib'

小咪咪 2022-04-25 10:40 119阅读 0赞

# 一、问题 #

（在Alpine 上装好spark， 运行 pyspark）

bash-4.4\# pyspark  
Python 3.6.5 (default, May 30 2019, 09:48:14)  
\[GCC 6.4.0\] on linux  
Type "help", "copyright", "credits" or "license" for more information.  
Traceback (most recent call last):  
  File "/spark/python/pyspark/shell.py", line 30, in <module>  
    import pyspark  
  File "/spark/python/pyspark/\_\_init\_\_.py", line 41, in <module>  
    from pyspark.context import SparkContext  
  File "/spark/python/pyspark/context.py", line 28, in <module>  
    from pyspark import accumulators  
  File "/spark/python/pyspark/accumulators.py", line 98, in <module>  
    from pyspark.serializers import read\_int, PickleSerializer  
  File "/spark/python/pyspark/serializers.py", line 58, in <module>  
    import zlib  
ModuleNotFoundError: No module named 'zlib'  
>>>

# 二、解决办法 #

先升级Alpine，再安装pip（安装pip和这错没关系）

bash-4.4\# apk add --update py-pip

apk update  
apk add py-pip

fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86\_64/APKINDEX.tar.gz

fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86\_64/APKINDEX.tar.gz  
(1/6) Installing libbz2 (1.0.6-r6)  
(2/6) Installing expat (2.2.5-r0)  
(3/6) Installing gdbm (1.13-r1)  
(4/6) Installing python2 (2.7.15-r1)  
(5/6) Installing py-setuptools (39.1.0-r0)  
(6/6) Installing py2-pip (10.0.1-r0)  
Executing busybox-1.28.4-r0.trigger

检查，已经正常运行：

bash-4.4\# pyspark  
Python 2.7.15 (default, Aug 16 2018, 14:17:09)  
\[GCC 6.4.0\] on linux2  
...........................................  
Welcome to  
      \_\_\_\_              \_\_  
     / \_\_/\_\_  \_\_\_ \_\_\_\_\_/ /\_\_  
    \_\\ \\/ \_ \\/ \_ \`/ \_\_/  '\_/  
   /\_\_ / .\_\_/\\\_,\_/\_/ /\_/\\\_\\   version 1.6.2  
      /\_/

Using Python version 2.7.15 (default, Aug 16 2018 14:17:09)  
SparkContext available as sc, HiveContext available as sqlContext.  
>>>