python-爬虫58租房数据在高德地图上显示

落日映苍穹つ 2022-05-14 09:16 142阅读 0赞

先上图

![70][]

我的环境python2.7  
运行后生产csv文件rent.cvs,内容为58同城品牌合租爬取的合租房源数据  
使用火狐浏览器访问（谷歌浏览器加载csv时显示跨域问题）

爬取网站：[http://bj.58.com/pinpaigongyu/pn/5/?minprice=2000\_4000][http_bj.58.com_pinpaigongyu_pn_5_minprice_2000_4000]

![70 1][]

抓取代码：

#!usr/bin/python
    # -*- coding: utf-8 -*-
    
    from bs4 import BeautifulSoup
    import requests
    import csv
    import codecs
    import sys
    reload(sys)
    sys.setdefaultencoding('utf-8')
    
    url = "http://bj.58.com/pinpaigongyu/pn/{page}/?minprice=2000_4000"
    
    '''已完成的页数序号，初时为0'''
    page = 0
    
    csv_file = open("rent.csv","w")
    csv_file.write(codecs.BOM_UTF8)
    csv_writer = csv.writer(csv_file, delimiter=',')
    
    while True:
        page += 1
        print("fetch: ", url.format(page=page))
        response = requests.get(url.format(page=page))
        html = BeautifulSoup(response.text, "html.parser")
        house_list = html.select(".list > li")
    
        # 循环在读不到新的房源时结束
        if not house_list:
            break
    
        for house in house_list:
    
            house_title = house.select("h2")[0].string
            house_url = "http://bj.58.com/%s"%(house.select("a")[0]["href"])
            house_info_list = house_title.split()
    
            # 如果第二列是公寓名则取第一列作为地址
    
            if "公寓" in house_info_list[1] or "青年社区" in house_info_list[1]:
                house_location = house_info_list[0]
            else:
                house_location = house_info_list[1]
    
            house_money = house.select(".money")[0].select("b")[0].string
            print 'house_title=', house_title, 'house_location=', house_location
            csv_writer.writerow([house_title, house_location, house_money, house_url])
    
    csv_file.close()

运行python脚本，抓取过程...

![70 2][]![70 3][]![70 4][]![70 5][]

![70 6][]![70 7][]![70 8][]

将抓取数据存入csv文件：

![70 9][]

页面js代码：

全部源码下载：

[https://download.csdn.net/download/ardo\_pass/10648354][https_download.csdn.net_download_ardo_pass_10648354]

[70]: /images/20220514/a2dd041b563d4b228d075aad156591ee.png
[http_bj.58.com_pinpaigongyu_pn_5_minprice_2000_4000]: http://bj.58.com/pinpaigongyu/pn/5/?minprice=2000_4000
[70 1]: /images/20220514/46866a456ec14578aecb30c056a9d7b5.png
[70 2]: https://img-blog.csdn.net/20180905223011358?watermark/2/text/aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2FyZG9fcGFzcw==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70
[70 3]: https://img-blog.csdn.net/20180905223028446?watermark/2/text/aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2FyZG9fcGFzcw==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70
[70 4]: https://img-blog.csdn.net/20180905223034728?watermark/2/text/aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2FyZG9fcGFzcw==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70
[70 5]: /images/20220514/2bdc4adcc2984beb9d14ab1ffc5a95b8.png
[70 6]: https://img-blog.csdn.net/20180905222755950?watermark/2/text/aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2FyZG9fcGFzcw==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70
[70 7]: https://img-blog.csdn.net/20180905222823794?watermark/2/text/aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2FyZG9fcGFzcw==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70
[70 8]: /images/20220514/20afea2ff6c34772bad9db50d355f695.png
[70 9]: /images/20220514/e8319f25699a4f7e920456b877691907.png
[https_download.csdn.net_download_ardo_pass_10648354]: https://download.csdn.net/download/ardo_pass/10648354