记录猫眼电影的自定义字体反爬

叁歲伎倆 2022-10-02 14:55 49阅读 0赞

### 猫眼电影用的是动态字体库 ###

#### **猫眼电影榜单国内票房榜，地址：[https://maoyan.com/board/1][https_maoyan.com_board_1]** ####

![1581786-20190414122856335-540556575.png][]

**首先需要获得字体文件，在页面或css里搜@font-face或font-famil**

![1581786-20190414201046285-515256562.png][]

![1581786-20190414201315438-1659639487.png][]

**在font刷新页面几次发现字体会变化，字体库是动态的**

### 解决方案 ###

**建立字和动态字体库字形的联系**

### **原理** ###

**字体用类似表的结构记录字，比如cmap记录了unicode索引和字形，这里反爬****用到的表示glyf字形表，表里记录了具体的字形笔画数据，**

**且表里只记录了字形数据，不关联其他表。****有专门的loca表按顺序记录glyf表里字形的位置，在使用字体时通过loca表来找到具体字形。**

**所以可以利用字形数据来找到自定义字体unicode与字的联系。**

**字体资料整理记录在： [https://www.cnblogs.com/shenyiyangle/p/10700156.html][https_www.cnblogs.com_shenyiyangle_p_10700156.html] **

**找关联思路：**

![1581786-20190414221005487-1334700990.png][]

**1.在猫眼电影下载一个字体做为基准，建立基准字体unicode和字的关系。**

**2.刷新网页后下载新字体，记为网站字体2，通过比较网站字体1和网站字体2的字形找到unicode和新unicode联系。**

**3.再通过相同的unicode来建立字和变化字体库unicode的联系，最后将新unicode替换成字。**

#  代码 #

headers={"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36"}
    r=requests.get("https://maoyan.com/board/1",headers=headers)
    font1_url="http:"+re.findall("url\(\'(\/\/.*?woff)\'\)",r.text,re.M)[0]
    #创建font目录保存基准字体
    if not os.path.exists("font"):
        font1=requests.get(font1_url,headers=headers) os.mkdir("font") with open("./font/base.woff","wb")as f: f.write(font1.content)

### 下载一次基准字体并保存到font目录 ###

base_font = TTFont('./font/base.woff')
    base_dict=[]
    for i in range(len(baseFont.getGlyphOrder()[2:])):
        print(f"对应的数字{i+1}:")
        w=input()
        base_dict.append({"code":baseFont.getGlyphOrder()[2:][i],"num":w})

### 建立基准字体的unicode和真实字符的关系，查看unicode使用的软件是FontCreator ###

![1581786-20190415152019886-1437334172.png][]

### 代码只需要执行一次，直接按顺序输入数字 ###

new_font_url="http:"+re.findall("url\(\'(\/\/.*?woff)\'\)",r.text,re.M)[0]
    font=requests.get(new_font_url,headers=headers)
    with open("new_font.woff","wb")as f:
        f.write(font.content)
    new_font = TTFont('new_font.woff')
    new_font_code_list=new_font.getGlyphOrder()[2:]

### 页面改变后的字体下载，获取unicode列表 ###

replace_dic=[]
    for i in range(10):
        news = new_font['glyf'][new_font_code_list[i]]
        for j in range(10):
            bases = base_font['glyf'][base_dict[j]["code"]]
            if news == bases: unicode=new_font_code_list[i].lower().replace("uni","&#x")+";" num= base_dict[j]["num"] replace_dic.append({"code":unicode,"num":num})

### 建立新unicode和字符的关系 ###

org_data=r.text
    for i in range(len(replace_dic)):
        new_data=new_data.replace(replace_dic[i]["code"],replace_dic[i]["num"])

### 全局替换unicode成字符 ###

tree=etree.HTML(org_data)
    dds=tree.xpath('//dl[@class="board-wrapper"]/dd')
    info=[]
    for dd in dds:
        title=dd.xpath('.//p[@class="name"]/a/@title')[0]
        star=dd.xpath('.//p[@class="star"]/text()')[0].replace("主演：","") time=dd.xpath('.//p[@class="releasetime"]/text()')[0].replace("上映时间：","") realticket=dd.xpath('.//p[@class="realtime"]//text()')[1]+dd.xpath('.//p[@class="realtime"]//text()')[2].strip() totalticket=dd.xpath('.//p[@class="total-boxoffice"]//text()')[1]+dd.xpath('.//p[@class="total-boxoffice"]//text()')[2].strip() info.append({"标题":title,"主演":star,"上映时间":time,"实时票房":realticket,"总票房":totalticket})

### 抓一些信息,下面是结果 ###

![1581786-20190415153410673-187829278.png][]

### 保存成csv ###

import csv
    csv_file = open("1325.csv", 'w', newline='')
    keys = []
    writer = csv.writer(csv_file)
    keys = info[1].keys()
    writer.writerow(keys)
    for dic in info: for key in keys: if key not in dic: dic[key ] = '' writer.writerow(dic.values()) csv_file.close()

### 结果 ###

![1581786-20190415153327402-1886784782.png][]

### 用的库 ###

![1581786-20190415153703872-704590040.png][]

转载于:https://www.cnblogs.com/shenyiyangle/p/10711065.html

[https_maoyan.com_board_1]: https://maoyan.com/board/1
[1581786-20190414122856335-540556575.png]: /images/20220109/d405f75b19d941f38d60ef93bdd57c99.png
[1581786-20190414201046285-515256562.png]: /images/20220109/c6f1a0df445d4103a51950c3833c191e.png
[1581786-20190414201315438-1659639487.png]: /images/20220109/b8c9f8a3f86e423da5826f581b55e7b0.png
[https_www.cnblogs.com_shenyiyangle_p_10700156.html]: https://www.cnblogs.com/shenyiyangle/p/10700156.html
[1581786-20190414221005487-1334700990.png]: /images/20220109/8c4f5b16498549dfa460d4a2c1d0e389.png
[1581786-20190415152019886-1437334172.png]: /images/20220109/85d06fad9cf243d2bca0f7e9d44339f5.png
[1581786-20190415153410673-187829278.png]: /images/20220109/e7dab3f970924b6aad4c9b6e8fa62c98.png
[1581786-20190415153327402-1886784782.png]: /images/20220109/0243a0a2b1d64c51b2689a708b025842.png
[1581786-20190415153703872-704590040.png]: /images/20220109/5581e0430f974300ab112b877b9e5bad.png