【HTML Parser】解析HTML：基于第三方库Jsoup

短命女 2022-01-05 19:41 227阅读 0赞

JSoup官方地址：[http://jsoup.org][http_jsoup.org]

Apache HttpComponents官方地址：[http://hc.apache.org/index.html][http_hc.apache.org_index.html]

1、抓取HTML内容

这里我们使用HttpClient库，根据URL请求远端的HTML

public static String getHTMLFromURL(String url) {
        String html = null;
        HttpClient httpClient = new DefaultHttpClient();
        HttpGet httpGet = new HttpGet(url);
        try {
            HttpResponse httpResponse = httpClient.execute(httpGet);
            int resStatu = httpResponse.getStatusLine().getStatusCode();
            if (resStatu == HttpStatus.SC_OK) {
                HttpEntity entity = httpResponse.getEntity();
                if (entity != null) {
                    html = EntityUtils.toString(entity);
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            httpClient.getConnectionManager().shutdown();
        }
        return html;
    }

2、解析HTML

示例，打印百度的标题

> 解析，获得Document对象

Document doc = Jsoup.parse(html);

> 使用 CSS 或 类似 JQuery 的 Selector 选择元素

Elements elements = doc.select("title");

> 打印元素的文本内容

System.out.println(ele.text());

String html = WebCrawler.getHTMLFromURL("http://www.baidu.com");
    if (html != null) {
        Document doc = Jsoup.parse(html);
        Elements elements = doc.select("title");
        for (Element element : linksElements) {
            System.out.println(element.text());
        }
    }

运行结果：

![20113801-956bf185f80d41b08f35e32cbc56951d.png][]

转载于:https://www.cnblogs.com/dyingbleed/archive/2013/03/20/2970841.html

[http_jsoup.org]: http://jsoup.org
[http_hc.apache.org_index.html]: http://hc.apache.org/index.html
[20113801-956bf185f80d41b08f35e32cbc56951d.png]: /images/20211228/7f47f6209ce148fda8a805e6be758f0c.png