Fiddler+手机模拟器+java采集抖音评论数据

深碍√TFBOYSˉ_ 2022-11-07 11:54 246阅读 0赞

Fiddler：请求抓包工具  
模拟器：任意安卓模拟器，建议使用7.0以下的引擎（7.0以后对安全数字证书做了修改无法抓包）  
java:采用springmvc 主要用于抓包以后提交数据

先下载fiddler和拦截https插件，因为现在大多数的请求都是用的https  
官网下载地址  
[https://www.telerik.com/download/fiddler][https_www.telerik.com_download_fiddler]  
https请求插件  
[http://www.telerik.com/docs/default-source/fiddler/addons/fiddlercertmaker.exe?sfvrsn=2][http_www.telerik.com_docs_default-source_fiddler_addons_fiddlercertmaker.exe_sfvrsn_2]

CSDN直接下载[https://download.csdn.net/download/terry711/16129928][https_download.csdn.net_download_terry711_16129928]

安装完fiddler和插件以后打开软软件  
点击工具栏中的Tools—>Options  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 1]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 2]  
![在这里插入图片描述][20210329144653745.png]  
工具右上角显示代理地址的ip

这里使用逍遥手机模拟器，使用5.1版本 7.0以后https无法监听  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 3]  
在wifi中修改代理  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 4]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 5]  
高级代理手动  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 6]  
输入ip加端口好 192.168.1.9:8888  
安装证书  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 7]  
输入任意名称，设置手机密码。完成证书安装  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 8]  
请求内容  
点击操作  
![在这里插入图片描述][20210329145648813.png]  
![在这里插入图片描述][20210329145701833.png]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 9]  
就能查看评论  
![在这里插入图片描述][20210329145813690.png]

static function OnBeforeResponse(oSession: Session) { 
            if (m_Hide304s && oSession.responseCode == 304) { 
                oSession["ui-hide"] = "true";
            }
            //过滤无关请求，只关注特定请求
            if (oSession.fullUrl.Contains("aweme.snssdk.com/aweme/v2/comment/list")) { 
                oSession.utilDecodeResponse();
                //消除保存的请求可能存在乱码的情况
                var jsonString = oSession.GetResponseBodyAsString();
                var responseJSON = Fiddler.WebFormats.JSON.JsonDecode(jsonString);
                FiddlerObject.log('2222222222222222'+responseJSON.JSONObject['comments'][0]['text']); 
                var len = responseJSON.JSONObject["comments"];
                FiddlerObject.log(len);
                var str = "";      
                    
                for(var j in len){ 
                    var aweme= j["aweme_id"];
                    var text = j["text"];
                    var user = j["user"]["unique_id"];
                    var time = j["create_time"];
                    var bh=j["user"]["short_id"];
                    //str = str+"留言<"+text+" -- 时间<"+time+" --- 编号<"+bh+" --- 抖音号<"+user+"\r\n";
                    str=str+""+aweme+"---"+text+"---"+time+"---"+bh+"---"+user+"___"
                
                }
                var _xhr = new ActiveXObject('Microsoft.XMLHTTP');
                var url = 'http://*.*.*.*:8080/all/set';
                _xhr.onreadystatechange = function() { }
                _xhr.open('POST', url, true);
                //_xhr.setRequestHeader("Content-Type", "application/json; charset=utf-8");
                _xhr.setRequestHeader( "Content-Type", "application/x-www-form-urlencoded;charset=utf-8");
                _xhr.setRequestHeader("X-Requested-With", "XMLHttpRequest");
                FiddlerObject.log(str);
                _xhr.send("bh="+str); 
    
                    }
                    }

加入代码写入java程序入库。

以上内容仅供学习讨论。

[https_www.telerik.com_download_fiddler]: https://www.telerik.com/download/fiddler
[http_www.telerik.com_docs_default-source_fiddler_addons_fiddlercertmaker.exe_sfvrsn_2]: http://www.telerik.com/docs/default-source/fiddler/addons/fiddlercertmaker.exe?sfvrsn=2
[https_download.csdn.net_download_terry711_16129928]: https://download.csdn.net/download/terry711/16129928
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70]: /images/20221023/814ccdc5cdca4d48b63af8cd4c27033e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 1]: /images/20221023/0bd58ead6280431682c6428ddbd13ac5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 2]: /images/20221023/dc0a2bfcb4834944b37fe85dbce5741c.png
[20210329144653745.png]: /images/20221023/57f92c80f2ed4587bde50c0947faa93f.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 3]: /images/20221023/70226fbea4e8460a8a580fe95996282a.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 4]: /images/20221023/0687009c5cd94502910de9068d572c0b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 5]: /images/20221023/d62e7c444228478cb45173ddb90343a8.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 6]: /images/20221023/339e0b62f89545fcba84e8302e0a0206.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 7]: /images/20221023/7f37a256789e46c8a2e76b7ecd3bc201.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 8]: /images/20221023/3bb7010a438448f0941a9f769d1c11c2.png
[20210329145648813.png]: /images/20221023/690554adbaf04ee2a420a4108aaec242.png
[20210329145701833.png]: /images/20221023/753aef3160504143984de29b8010b410.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RlcnJ5NzEx_size_16_color_FFFFFF_t_70 9]: /images/20221023/638f7afeb5dc4a4fb846364e493de405.png
[20210329145813690.png]: /images/20221023/d701f3b04a9f445d82c2349f5b09eb6f.png