Linux下安装Scrapy框架

心已赠人 2022-05-15 05:39 418阅读 0赞

### Linux系统：Centos7 64位 ###

### Scrapy框架：python的一个爬虫框架 ###

### python版本：python3.6.3 ###

### 安装步骤： ###

\[root@localhost Python3.6.3\]\# pip3 install scrapy

![70][]

安装过程可能报的错，解决方案：

[解决Could not find a version that satisfies the requirement Twisted>=13.1.0 (from Scrapy)][Could not find a version that satisfies the requirement Twisted_13.1.0 _from Scrapy]

[解决You are using pip version 9.0.1, however version 18.0 is available. You should consider upgrading][You are using pip version 9.0.1_ however version 18.0 is available. You should consider upgrading]

[解决import twisted.persisted.styles \# NOQA ModuleNotFoundError: No module named 'twisted.persisted'][import twisted.persisted.styles _ NOQA ModuleNotFoundError_ No module named _twisted.persisted]

安装成功：

在python3中导入scrapy:表示安装成功。

[root@localhost Python3.6.3]# python3
    Python 3.6.3 (default, Aug 21 2018, 20:41:20) 
    [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import scrapy
    >>> exit()

建立scrapy的软连接：

\[root@localhost Python3.6.3\] \# ln -s /usr/local/ Python3.6.3/bin/scrapy /usr/bin/scrapy

验证Scrapy软链接

[root@localhost Python3.6.3]# scrapy --v
    Scrapy 1.5.1 - no active project
    
    Usage:
      scrapy <command> [options] [args]
    
    Available commands:
      bench         Run quick benchmark test
      fetch         Fetch a URL using the Scrapy downloader
      genspider     Generate new spider using pre-defined templates
      runspider     Run a self-contained spider (without creating a project)
      settings      Get settings values
      shell         Interactive scraping console
      startproject  Create new project
      version       Print Scrapy version
      view          Open URL in browser, as seen by Scrapy
    
      [ more ]      More commands available when run from project directory
    
    Use "scrapy <command> -h" to see more info about a command

### 我的座右铭：不会，我可以学；落后，我可以追赶；跌倒，我可以站起来；我一定行。 ###

[70]: /images/20220515/f7bc2f5e83d74d9b99c63cfc9d6c79e5.png
[Could not find a version that satisfies the requirement Twisted_13.1.0 _from Scrapy]: https://blog.csdn.net/weixin_39220472/article/details/82050405
[You are using pip version 9.0.1_ however version 18.0 is available. You should consider upgrading]: https://blog.csdn.net/weixin_39220472/article/details/82050339
[import twisted.persisted.styles _ NOQA ModuleNotFoundError_ No module named _twisted.persisted]: https://blog.csdn.net/weixin_39220472/article/details/82050491