【Pyhton爬虫】中国大学排名爬虫

淡淡的烟草味﹌ 2023-02-25 02:10 64阅读 0赞

[http://www.zuihaodaxue.com/zuihaodaxuepaiming-zongbang-2020.html][http_www.zuihaodaxue.com_zuihaodaxuepaiming-zongbang-2020.html]

![format_png][]

![format_png 1][]

![format_png 2][]

![format_png 3][]

![format_png 4][]

![format_png 5][]

![format_png 6][]

代码：

import requests
    from bs4 import BeautifulSoup
    import bs4
    def getHTMLtext(url):#获取URL信息
        try:
            r=requests.get(url,timeout=30)
            r.raise_for_status()#异常判断
            r.encoding=r.apparent_encoding#转换编码
            return r.text
        except:
            return""
    
    def fillUnivList(ulist,html):#将HTML信息放在列表中
        soup=BeautifulSoup(html,"html.parser")
        for tr in soup.find('tbody').children:
            if isinstance(tr,bs4.element.Tag):#判断类型
                tds=tr('td')
                ulist.append([tds[0].string,tds[1].string,tds[2].string,tds[3].string,tds[4].string])#大学排名、大学名称、大学。。
    
    def printUnivList(ulist,num):#打印出来
        print("{:^10}\t{:^6}\t{:^10}\t{:^6}\t{:^10}".format("排名","学校名称","省市","学校类型","总分"))
        for i in range(num):
            u=ulist[i]
            print("{:^10}\t{:^6}\t{:^10}\t{:^6}\t{:^10}".format(u[0],u[1],u[2],u[3],u[4]))   
    
    
    def main():
        uinfo=[]
        url="http://www.zuihaodaxue.com/zuihaodaxuepaiming-zongbang-2020.html"
        html=getHTMLtext(url)
        fillUnivList(uinfo,html)
        printUnivList(uinfo,20)#列举20所学校
    main()

效果

![format_png 7][]

我们发现会有对不齐的现象

优化

![format_png 8][]

![format_png 9][]

代码：

import requests
    from bs4 import BeautifulSoup
    import bs4
    def getHTMLtext(url):#获取URL信息
        try:
            r=requests.get(url,timeout=30)
            r.raise_for_status()#异常判断
            r.encoding=r.apparent_encoding#转换编码
            return r.text
        except:
            return""
    
    def fillUnivList(ulist,html):#将HTML信息放在列表中
        soup=BeautifulSoup(html,"html.parser")
        for tr in soup.find('tbody').children:
            if isinstance(tr,bs4.element.Tag):#判断类型
                tds=tr('td')
                ulist.append([tds[0].string,tds[1].string,tds[2].string,tds[3].string,tds[4].string])#大学排名、大学名称、大学。。
    
    def printUnivList(ulist,num):#打印出来
        tplt="{0:^10}\t{1:{5}^10}\t{2:^6}\t{3:^6}\t{4:^6}"
        print(tplt.format("排名","学校名称","省市","学校类型","总分",chr(12288)))
        for i in range(num):
            u=ulist[i]
            print(tplt.format(u[0],u[1],u[2],u[3],u[4],chr(12288)))   
    
    
    def main():
        uinfo=[]
        url="http://www.zuihaodaxue.com/zuihaodaxuepaiming-zongbang-2020.html"
        html=getHTMLtext(url)
        fillUnivList(uinfo,html)
        printUnivList(uinfo,20)#列举20所学校
    main()

效果

![format_png 10][]

是不是美观了不少？

[http_www.zuihaodaxue.com_zuihaodaxuepaiming-zongbang-2020.html]: http://www.zuihaodaxue.com/zuihaodaxuepaiming-zongbang-2020.html
[format_png]: /images/20230209/e7aaa9901cf3435785f7d2b57d111916.png
[format_png 1]: /images/20230209/1ccb2b66943e4cb88ec6ab95fc824fb3.png
[format_png 2]: /images/20230209/c72c1aee9d194502acb1d29a147d1901.png
[format_png 3]: /images/20230209/867eadba25c04128aa9ff16b18d7f7e4.png
[format_png 4]: /images/20230209/9166522e396a45f4925b825ebe74509e.png
[format_png 5]: /images/20230209/fb86156563664d1e856485a70dac45b1.png
[format_png 6]: /images/20230209/25bdbc739bcf42e793a75c9caaa3e69d.png
[format_png 7]: /images/20230209/4e2a35f931ad45cdac815795253c8f4a.png
[format_png 8]: /images/20230209/1f1233f66b524f2a8e37da69c2703160.png
[format_png 9]: /images/20230209/0232b00bfefe4e159c5a1beaf395a2a7.png
[format_png 10]: /images/20230209/dcdde886a5ec49ad851c9a175d7ba0f5.png