网络爬虫（蜘蛛）Scrapy,Python安装！

╰半橙微兮° 2022-08-02 09:48 306阅读 0赞

原文转自 [http://blog.csdn.net/niying/article/details/27103081][http_blog.csdn.net_niying_article_details_27103081]

费了一番力气，装了Scrapy，标记一下。另外在上面的文章里，补充一点。

**Scrapy,Python安装，使用！**

**1、下载安装Python2.7.6，因为Scrapy还不支持3.x版本。**[**Latest Python 2 Release - Python 2.7.6**][Latest Python 2 Release - Python 2.7.6]**，安装时选择当前用户。**

**2、到**[**http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/**][http_sourceforge.net_projects_pywin32_files_pywin32_Build_20219]**下载安装**[**pywin32-219.win-amd64-py2.7.exe**][http_sourceforge.net_projects_pywin32_files_pywin32_Build_20219]

**3、将C:\\python27\\Scripts;C:\\python27;添加到环境变量里。**

**4、**[**http://slproweb.com/products/Win32OpenSSL.html**][http_slproweb.com_products_Win32OpenSSL.html]** 到这里下载**[**Visual C++ 2008 Redistributables**][Visual C_ 2008 Redistributables]**和**[**Win32 OpenSSL v1.0.1g Light**][Win32 OpenSSL v1.0.1g Light]**并安装。安装时选择当前用户。将OpenSSL的bin目录添加到环境变量（C:\\OpenSSL-Win32）**

**5、到**[**https://pypi.python.org/pypi/setuptools\#windows-7-or-graphical-install**][https_pypi.python.org_pypi_setuptools_windows-7-or-graphical-install]**下载**[**ez\_setup.py**][ez_setup.py]**，用Python运行**[**ez\_setup.py**][ez_setup.py]**（双击也行），会自动安装好工具、**

**6、到**[**https://pypi.python.org/pypi/zope.interface/4.1.1\#downloads**][https_pypi.python.org_pypi_zope.interface_4.1.1_downloads]** 下载**[**zope.interface-4.1.1-py2.7-win32.egg**][zope.interface-4.1.1-py2.7-win32.egg]** 安装，复制到Python/Scripts下，运行 easy\_install **[**zope.interface-4.1.1-py2.7-win32.egg**][zope.interface-4.1.1-py2.7-win32.egg]**。（如果后缀名不是egg,修改为egg）**

**补：---、到[http://www.microsoft.com/en-us/download/confirmation.aspx?id=44266][http_www.microsoft.com_en-us_download_confirmation.aspx_id_44266]下载 [Microsoft Visual C++ Compiler for Python 2.7][http_www.microsoft.com_en-us_download_confirmation.aspx_id_44266].**

**7、到**[**https://pypi.python.org/pypi/pyOpenSSL/0.13**][http_www.microsoft.com_en-us_download_confirmation.aspx_id_44266]**下载**[**pyOpenSSL-0.13-py2.7-win32.egg**][pyOpenSSL-0.13-py2.7-win32.egg]** 安装同6。**

**8、到**[**https://pypi.python.org/pypi/lxml/3.3.5**][https_pypi.python.org_pypi_lxml_3.3.5]**下载****[lxml-3.3.5.win32-py2.7][].exe**** 安装**

**9、到 **[**https://pypi.python.org/pypi/Twisted/14.0.0**][https_pypi.python.org_pypi_Twisted_14.0.0]** 下载**[**Twisted-14.0.0.win32-py2.7.exe**][Twisted-14.0.0.win32-py2.7.exe]**安装。（安装时选择当前用户）**

**10、到**[**https://pypi.python.org/pypi/cryptography/0.4\#downloads**][https_pypi.python.org_pypi_cryptography_0.4_downloads]**下载**[**cryptography-0.4-cp27-none-win32.whl**][cryptography-0.4-cp27-none-win32.whl]** 安装同6**

**11、**[**https://pypi.python.org/pypi/cffi/0.8.2**][https_pypi.python.org_pypi_cffi_0.8.2]**下载**[**cffi-0.8.2-cp27-none-win32.whl**][cffi-0.8.2-cp27-none-win32.whl]**安装同六**

**补：---、到[https://pypi.python.org/pypi/pyasn1][http_www.microsoft.com_en-us_download_confirmation.aspx_id_44266]下载 [pyasn1-0.1.8-py2.7.egg][cffi-0.8.2-cp27-none-win32.whl].**

**12、到**[**https://pypi.python.org/pypi/service\_identity/0.2\#downloads**][https_pypi.python.org_pypi_service_identity_0.2_downloads]**下载**[**service\_identity-0.2-py2.py3-none-any.whl**][service_identity-0.2-py2.py3-none-any.whl]**  安装同6**

**13、命令行下运行easy\_install -U scrapy将会自动下载并安装完成。**

![SouthEast][]

***注：以上安装以win32为例，如果是win64则全部要下载64位的安装包。在win64安装WIn64OpenSSL时，还要把C:\\OpenSSL-Win32\\bin目录下的libeay32.dll和ssleay32.dll放到C:\\windows\\system32下，不然运行scrapy时出错。***

***问题汇总：***

**1、如果没安装10-12步骤，会出现pyOpenSSL警告和未安装service\_identity错误，这两个都人先安装编译器，VC2008和VC2010都可以。如果不是VC2008则要建立一个环境变量：VS90COMNTOOLS，变量值和VS100COMNTOOLS一样就可以。**

**![SouthEast 1][]**

**2、如果出现“Unable to find vcvarsall.bat”，则是没有安装VC2008出现的。**

====================================================

关于Python的正则表达式：

1. [Python正则表达式的七个使用范例][Python]

2. [Python正则表达式指南][Python 1]

3. [Python正则表达式操作指南][Python 2]

====================================================

关于Python的爬虫实例

1. [\[Python\]网络爬虫][Python 3]

====================================================

Python程序打包

1. [ Python for windows][Python for windows]

[http_blog.csdn.net_niying_article_details_27103081]: http://blog.csdn.net/niying/article/details/27103081
[Latest Python 2 Release - Python 2.7.6]: https://www.python.org/download/releases/2.7.6
[http_sourceforge.net_projects_pywin32_files_pywin32_Build_20219]: http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/
[http_slproweb.com_products_Win32OpenSSL.html]: http://slproweb.com/products/Win32OpenSSL.html
[Visual C_ 2008 Redistributables]: http://www.microsoft.com/downloads/details.aspx?familyid=9B2DA534-3E03-4391-8A4D-074B9F2BC1BF
[Win32 OpenSSL v1.0.1g Light]: http://slproweb.com/download/Win32OpenSSL_Light-1_0_1g.exe
[https_pypi.python.org_pypi_setuptools_windows-7-or-graphical-install]: https://pypi.python.org/pypi/setuptools#windows-7-or-graphical-install
[ez_setup.py]: https://bootstrap.pypa.io/ez_setup.py
[https_pypi.python.org_pypi_zope.interface_4.1.1_downloads]: https://pypi.python.org/pypi/zope.interface/4.1.1#downloads
[zope.interface-4.1.1-py2.7-win32.egg]: https://pypi.python.org/packages/2.7/z/zope.interface/zope.interface-4.1.1-py2.7-win32.egg#md5=97fc757b020bb25b829f9c566d87c8c3
[http_www.microsoft.com_en-us_download_confirmation.aspx_id_44266]: https://pypi.python.org/pypi/pyOpenSSL/0.13
[pyOpenSSL-0.13-py2.7-win32.egg]: https://pypi.python.org/packages/2.7/p/pyOpenSSL/pyOpenSSL-0.13-py2.7-win32.egg#md5=e0b7b0d2cccca83ae38e11c3a105dfe4
[https_pypi.python.org_pypi_lxml_3.3.5]: https://pypi.python.org/pypi/lxml/3.3.5
[lxml-3.3.5.win32-py2.7]: https://pypi.python.org/packages/3.2/l/lxml/lxml-3.3.5.win32-py3.2.exe#md5=65f2643dc8de8e7c8e2c4c37bb585fc7
[https_pypi.python.org_pypi_Twisted_14.0.0]: https://pypi.python.org/pypi/Twisted/14.0.0
[Twisted-14.0.0.win32-py2.7.exe]: https://pypi.python.org/packages/2.7/T/Twisted/Twisted-14.0.0.win32-py2.7.exe#md5=ca9f6266ee0fa277611a04ceed9b700d
[https_pypi.python.org_pypi_cryptography_0.4_downloads]: https://pypi.python.org/pypi/cryptography/0.4#downloads
[cryptography-0.4-cp27-none-win32.whl]: https://pypi.python.org/packages/cp27/c/cryptography/cryptography-0.4-cp27-none-win32.whl#md5=9159a626742681bd961cb74f9b7e8643
[https_pypi.python.org_pypi_cffi_0.8.2]: https://pypi.python.org/pypi/cffi/0.8.2
[cffi-0.8.2-cp27-none-win32.whl]: https://pypi.python.org/packages/cp27/c/cffi/cffi-0.8.2-cp27-none-win32.whl#md5=748720929cf4175330f6459e13b4f45a
[https_pypi.python.org_pypi_service_identity_0.2_downloads]: https://pypi.python.org/pypi/service_identity/0.2#downloads
[service_identity-0.2-py2.py3-none-any.whl]: https://pypi.python.org/packages/2.7/s/service_identity/service_identity-0.2-py2.py3-none-any.whl#md5=859428617de6ac522b781a3d0a1d6689
[SouthEast]: /images/20220731/e163f1cdf3aa413d9d0a3a0d7c0e7d47.png
[SouthEast 1]: /images/20220731/86fbcad847fb46c099f3c123ae9ef05d.png
[Python]: http://blog.jobbole.com/74844/
[Python 1]: http://www.cnblogs.com/huxi/archive/2010/07/04/1771073.html
[Python 2]: http://wiki.ubuntu.org.cn/Python%E6%AD%A3%E5%88%99%E8%A1%A8%E8%BE%BE%E5%BC%8F%E6%93%8D%E4%BD%9C%E6%8C%87%E5%8D%97
[Python 3]: http://blog.csdn.net/pleasecallmewhy/article/details/8922826
[Python for windows]: http://sourceforge.net/projects/pywin32/files/pywin32/Build%20217/