Python3 爬虫使用代理 IP

叁歲伎倆 2022-03-31 02:42 291阅读 0赞

写了比较详细的注释,就不废话了:

# -*- coding: utf-8 -*-
    
    #引入requests库,没有安装 请 cmd> pip install requests
    import requests
    
    #要使用的代理 IP
    #我在西刺上找的,过期了的话自己找过
    #西刺: https://www.xicidaili.com/
    proxy = "119.101.112.15:9999"
    #设置代理
    proxies = {
        'http': 'http://' + proxy,      #处理http连接的
        'https': 'https://' + proxy,    #处理https连接的
    }
    #设置请求头
    User_Agent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1062.0 Safari/536.3"
    headers = {'User-Agent': User_Agent}
    try:
        #这段,设置自动关闭多余连接,不然容易报 MaxRetryError
        s = requests.session()  #获取 会话
        s.keep_alive = False  #保持连接,设为 false
        s.adapters.DEFAULT_RETRIES = 300  #最大连接 改为300
        #打印请求头,代理信息
        print("headers",headers)
        print("proxies",proxies)
        #发起请求
        response = s.get( #不用 s.get( 的话 可以直接用 requests.get(
                #你用浏览器直接访问这个地址,可以看到你的IP
                "http://httpbin.org/get",
                proxies=proxies,
                headers=headers,
                timeout=5
        )
        #打印响应内容,结果有你代理IP的字样就成功了
        print(response.text)
    except requests.exceptions.ConnectionError as e:
        print('Error', e.args)

**报错 : 使用代理IP,遇到错误 \[WinError 10061\] 由于目标计算机积极拒**

**或者**

**报错:** Error (MaxRetryError("HTTPConnectionPool(host='119.101.112.9', port=9999): Max retries exceeded with url: 此处省略, 'Connection to 119.101.112.9 timed out. (connect timeout=5)'))"),)

**请看: [https://blog.csdn.net/weixin\_42144379/article/details/85568234][https_blog.csdn.net_weixin_42144379_article_details_85568234]**

**上文解决方案已经使用,还是报此错,点击 [百度解决方案][Link 1].**

**其他错误: [还是百度][Link 2]**

[https_blog.csdn.net_weixin_42144379_article_details_85568234]: https://blog.csdn.net/weixin_42144379/article/details/85568234
[Link 1]: https://www.baidu.com/s?ie=UTF-8&wd=Error%20%28MaxRetryError%28%22HTTPConnectionPool%28host=%27119.101.112.9%27,%20port=9999%29%3A%20Max%20retries%20exceeded%20with%20url%3A%20%E6%AD%A4%E5%A4%84%E7%9C%81%E7%95%A5,%20%27Connection%20to%20119.101.112.9%20timed%20out.%20%28connect%20timeout=5%29%27%29%29%22%29,%29
[Link 2]: http://www.baidu.com