使用python的selenium自动化登录获取cookie

桃扇骨 2021-09-29 10:14 462阅读 0赞

要爬取广点通广告平台数据，这平台居然没有提供开发用的API数据接口，登录是QQ登录，很复杂，不好抓取登录接口逻辑

于是就用这个

想录gif的不好录 -

文档：

https://www.yiibai.com/selenium/selenium_webdriver.html

python扩展

https://pypi.org/project/selenium/

selenium的python官方手册：

https://selenium-python-zh.readthedocs.io/en/latest/index.html

代码：

#!/usr/local/bin/python
    # -*- coding: UTF-8 -*-
    
    
    from selenium import webdriver # 从selenium导入webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    from selenium.webdriver.chrome.options import Options
    import json
    import time
    
    chrome_options = Options()
    chrome_options.add_argument('--no-sandbox')
    chrome_options.add_argument('--disable-dev-shm-usage')
    # 不启动界面显示- linux下命令行模式必须启用
    # chrome_options.add_argument('--headless')
    
    driver = webdriver.Chrome(chrome_options=chrome_options)  # Optional argument, if not specified will search path.
    driver.get('http://adnet.qq.com/index') # 获取百度页面
    driver.switch_to.frame('ptlogin_iframe') # 进入iframe
    
    # 选择账号密码登录
    selElement = driver.find_element_by_id('switcher_plogin')
    selElement.click()
    
    # 输入账号密码
    userElement = driver.find_element_by_id('u') 
    pwdButton = driver.find_element_by_id('p') #密码输入框
    subButton = driver.find_element_by_id('login_button') #密码输入框
    userElement.send_keys("111") #输入框输入
    pwdButton.send_keys("xxx") #输入框输入
    subButton.click() 
    
    # 显示等待
    try:
        element = WebDriverWait(driver, 10).until(
            EC.presence_of_element_located((By.ID, "page"))
        )
    finally:
        cookies = driver.get_cookies()
        # time.sleep(5)
        # driver.refresh('http://adnet.qq.com/index')
    
        with open("cookies.txt", "w") as fp:
            json.dump(cookies, fp)
    
    
        # 关闭浏览器
        driver.close()

很简洁吧，真真实实能解决登录这一块的问题，但是linux上环境的搭建真滴不容易

** Linux环境搭建**

大致流程

1、python

2、pip selenium

3、chrome brower

4、/usr/bin/chromedriver

参考博客：

Linux配置Selenium+Chrome+Python实现自动化测试：
    http://zhaoyabei.github.io/2016/08/29/Linux%E9%85%8D%E7%BD%AESelenium+Chrome+Python%E5%AE%9E%E7%8E%B0%E8%87%AA%E5%8A%A8%E5%8C%96%E6%B5%8B%E8%AF%95/

安装chrom-linux去官网：

https://www.chrome64bit.com/index.php/google-chrome-64-bit-for-linux

下载deb文件到机器上

dpkg -i google-chrome-stable_current_amd64.deb

但是报很多的依赖问题

![ContractedBlock.gif][] ![ExpandedBlockStart.gif][]

Unpacking google-chrome-stable (72.0.3626.121-1) ...
    dpkg: dependency problems prevent configuration of google-chrome-stable:
     google-chrome-stable depends on fonts-liberation; however:
      Package fonts-liberation is not installed.
     google-chrome-stable depends on libappindicator3-1; however:
      Package libappindicator3-1 is not installed.
     google-chrome-stable depends on libasound2 (>= 1.0.16); however:
      Package libasound2 is not installed.
     google-chrome-stable depends on libatk-bridge2.0-0 (>= 2.5.3); however:
      Package libatk-bridge2.0-0 is not installed.
     google-chrome-stable depends on libatk1.0-0 (>= 2.2.0); however:
      Package libatk1.0-0 is not installed.
     google-chrome-stable depends on libatspi2.0-0 (>= 2.9.90); however:
      Package libatspi2.0-0 is not installed.
     google-chrome-stable depends on libcairo2 (>= 1.6.0); however:
      Package libcairo2 is not installed.
     google-chrome-stable depends on libgdk-pixbuf2.0-0 (>= 2.22.0); however:
      Package libgdk-pixbuf2.0-0 is not installed.
     google-chrome-stable depends on libgtk-3-0 (>= 3.9.10); however:
      Package l
    dpkg: error processing package google-chrome-stable (--install):
     dependency problems - leaving unconfigured
    Processing triggers for mime-support (3.59ubuntu1) ...
    Processing triggers for man-db (2.7.5-1) ...
    Errors were encountered while processing:
     google-chrome-stable

sudo apt-get -f install

解决依赖问题

查看是否已经安装：

which google-chrome

转载于:https://www.cnblogs.com/xuweiqiang/p/10439976.html

[ContractedBlock.gif]: https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif
[ExpandedBlockStart.gif]: /images/20210725/d20c494867b94694b226445ccd3b2abb.png