Python爬取比比网中标标书并保存成PDF格式 雨点打透心脏的1/2处 2022-12-19 03:58 112阅读 0赞 ## 前言 ## 本文的文字及图片来源于网络,仅供学习、交流使用,不具有任何商业用途,如有问题请及时联系我们以作处理。 PS:如有需要Python学习资料的小伙伴可以加点击下方链接自行获取 [python免费学习资料以及群交流解答点击即可加入][python] -------------------- **python开发环境** * python 3.6 * pycharm import requests import parsel import pdfkit import time 相关模块pip安装即可 ## 目标网页分析 ## ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center] 1、先从列表页中获取详情页的URL地址 是静态网站,可以直接请求网页获取数据 ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 1] for page in range(1, 31): url = 'https://www.bibenet.com/mfzbu{}.html'.format(page) headers = { 'Referer': 'https://www.bibenet.com/mianfei/', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36' } response = requests.get(url=url, headers=headers) selector = parsel.Selector(response.text) urls = selector.css('body > div.wrap > div.clearFiex > div.col9.fl > div.secondary_box > table tr .fl a::attr(href)').getall() for page_url in urls: print(page_url) 2、从详情页中获取标题以及内容 response_2 = requests.get(url=page_url, headers=headers) selector_2 = parsel.Selector(response_2.text) article = selector_2.css('.container').get() title = selector_2.css('.detailtitle::text').get() 3、保存html网页数据并转成PDF html_str = """ <!doctype html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Document</title> </head> <body> {article} </body> </html> """ def download(article, title): html = html_str.format(article=article) html_path = 'D:\\python\\demo\\招标网\\文书\\' + title + '.html' pdf_path = 'D:\\python\\demo\\招标网\\文书\\' + title + '.pdf' with open(html_path, mode='wb', encoding='utf-8') as f: f.write(html) print('{}已下载完成'.format(title)) # exe 文件存放的路径 config = pdfkit.configuration(wkhtmltopdf='C:\\Program Files\\wkhtmltopdf\\bin\\wkhtmltopdf.exe') # 把 html 通过 pdfkit 变成 pdf 文件 pdfkit.from_file(html_path, pdf_path, configuration=config) ## 运行实现效果 ## ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 2] ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 3] ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 4] [python]: https://jq.qq.com/?_wv=1027&k=ThcV4SNF [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center]: /images/20221120/cd009fcdc2e84756be39b6a9791a832e.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 1]: /images/20221120/fd73ebd350684c5c911f1028f441109d.png [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 2]: https://img-blog.csdnimg.cn/2020110514373812.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA==,size_16,color_FFFFFF,t_70#pic_center [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 3]: https://img-blog.csdnimg.cn/20201105143749920.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA==,size_16,color_FFFFFF,t_70#pic_center [watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 4]: https://img-blog.csdnimg.cn/20201105143827368.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA==,size_16,color_FFFFFF,t_70#pic_center
还没有评论,来说两句吧...