HttpClient4 遇到的问题1：httpclient4.x 使用连接池发送https请求使用总结

朱雀 2021-02-15 15:44 646阅读 0赞

由于项目中需要用httpclient进行内部服务请求处理，但之前没有是用httpclient中提供的连接池，而是自己封装的一个类似池的结构，一段时间后产生了性能问题，转而尝试通过是用httpclient内部的池进行处理，而且需要支持https的连接。

所使用的httpclient的版本为4.4.1，一共用到了三个jar：httpclient-4.4.1.jar、httpclient-cache-4.4.1.jar、httpcore-4.4.1.jar。

所使用的连接池管理器的类为PoolingHttpClientConnectionManager，但httpclient部分参数，包括池的大小，最大路由，缓存，超时这些参数还未做深入研究；

public void init()  
     {  
       try {  
       //需要通过以下代码声明对https连接支持  
       SSLContext sslcontext = SSLContexts.custom().loadTrustMaterial(null,  
                       new TrustSelfSignedStrategy())  
               .build();  
       HostnameVerifier hostnameVerifier = SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER;  
       SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(  
               sslcontext,hostnameVerifier);  
       Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()  
               .register("http", PlainConnectionSocketFactory.getSocketFactory())  
               .register("https", sslsf)  
               .build();  
       //初始化连接管理器  
       poolConnManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);  
       // Increase max total connection to 200  
       poolConnManager.setMaxTotal(maxTotalPool);  
       // Increase default max connection per route to 20  
       poolConnManager.setDefaultMaxPerRoute(maxConPerRoute);  
      } catch (KeyManagementException e) {  
       // TODO Auto-generated catch block  
       e.printStackTrace();  
      } catch (NoSuchAlgorithmException e) {  
       // TODO Auto-generated catch block  
       e.printStackTrace();  
      } catch (KeyStoreException e) {  
       // TODO Auto-generated catch block  
       e.printStackTrace();  
      }  
     }  
    //获取连接  
     public CloseableHttpClient getConnection()  
     {  
          CloseableHttpClient httpClient = HttpClients.custom()  
    .setConnectionManager(poolConnManager).build();   
          return httpClient;  
     }  
    //发送请求 url为请求url，jsonstr为请求参数  
    public String post(String url, String jsonStr)  
     {  
      String returnStr = null;  
      //参数检测  
      if(url==null||"".equals(url))  
      {  
       return returnStr;  
      }  
      try {  
       HttpPost httpPost = new HttpPost(url);  
       List <NameValuePair> nvps = new ArrayList <NameValuePair>();  
       //设置post参数对  
       nvps.add(new BasicNameValuePair("jsonstr", jsonStr));  
       //设置编码，如果包含中文，一定要进行设置，否则按照系统默认字符集进行转码会出现乱码  
       httpPost.setEntity(new UrlEncodedFormEntity(nvps, "UTF-8"));  
       CloseableHttpResponse response = client.execute(httpPost);  
       //获取响应状态码  
       int status = response.getStatusLine().getStatusCode();  
                if (status >= 200 && status < 300) {  
                    HttpEntity entity = response.getEntity();  
                    return entity != null ? EntityUtils.toString(entity,"utf-8") : null;  
                } else {  
                    throw new ClientProtocolException("Unexpected response status: " + status);  
                }  
      } catch (UnsupportedEncodingException e) {  
       // TODO Auto-generated catch block  
       e.printStackTrace();  
      } catch (ClientProtocolException e) {  
       // TODO Auto-generated catch block  
       e.printStackTrace();  
      } catch (IOException e) {  
       // TODO Auto-generated catch block  
       e.printStackTrace();  
      }  
      //相应返回值  
      return returnStr;  
     }

在使用上述代码一段时间后，出现了tomcat异常后死掉的bug，而且是随机出现，没有找到具体原因，最终在加了一堆日志后，分析得知是因为在返回状态码非200的时候，httpclient的连接不会自动断开，需要手工断开，否则在连接达到pool的最大值后，无可用连接，最终导致tomcat死掉。

最后修改的代码如下：

public void init()  
        {  
             try {  
                SSLContext sslcontext = SSLContexts.custom().loadTrustMaterial(null,  
                                new TrustSelfSignedStrategy())  
                        .build();  
                HostnameVerifier hostnameVerifier = SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER;  
                SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(  
                        sslcontext,hostnameVerifier);  
                Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()  
                        .register("http", PlainConnectionSocketFactory.getSocketFactory())  
                        .register("https", sslsf)  
                        .build();  
                poolConnManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);  
                // Increase max total connection to 200  
                poolConnManager.setMaxTotal(maxTotalPool);  
                // Increase default max connection per route to 20  
                poolConnManager.setDefaultMaxPerRoute(maxConPerRoute);  
                SocketConfig socketConfig = SocketConfig.custom().setSoTimeout(socketTimeout).build();  
                poolConnManager.setDefaultSocketConfig(socketConfig);  
            } catch (Exception e) {  
                log.error("InterfacePhpUtilManager init Exception"+e.toString());  
            }  
        }  
        public CloseableHttpClient getConnection()  
        {  
            RequestConfig requestConfig = RequestConfig.custom().setConnectionRequestTimeout(connectionRequestTimeout)  
                    .setConnectTimeout(connectTimeout).setSocketTimeout(socketTimeout).build();  
            CloseableHttpClient httpClient = HttpClients.custom()  
                        .setConnectionManager(poolConnManager).setDefaultRequestConfig(requestConfig).build();  
            if(poolConnManager!=null&&poolConnManager.getTotalStats()!=null)  
            {  
                log.info("now client pool "+poolConnManager.getTotalStats().toString());  
            }  
            return httpClient;  
        }

其中输出的连接一共有四个状态，是需要时时关注的，可以作为调整参数的依据，具体含义为：

leased ：the number of persistent connections tracked by the connection manager currently being used to execute requests.  
      
    available ：the number idle persistent connections.  
      
    pending : the number of connection requests being blocked awaiting a free connection.  
      
    max: the maximum number of allowed persistent connections.

需要特别关注pending和leased的值，如果leased的值特别大，接近max，则需要修改max，如果pending的值也比较大，也需要调整max，并考虑设置timeout，可以设置两个timeout，一个是获取连接的timeout，另外一个是获取socket数据的timeout。代码中已经有相关示例了。

具体获取响应时，要改为如下代码，即在返回的状态码不是200时，要主动关闭连接

public String postPhp(String url, String jsonStr)  
        {  
            String returnStr = null;  
            //参数检测  
            if(url==null||"".equals(url))  
            {  
                return returnStr;  
            }  
            HttpPost httpPost = new HttpPost(url);  
            try {  
                  
                long currentTime=System.currentTimeMillis();  
                List <NameValuePair> nvps = new ArrayList <NameValuePair>();  
                nvps.add(new BasicNameValuePair("jsonstr", jsonStr));  
                httpPost.setEntity(new UrlEncodedFormEntity(nvps, "UTF-8"));  
                sysoutLog(currentTime+" 开始发送 请求：url"+url);  
                CloseableHttpResponse response = client.execute(httpPost);  
                int status = response.getStatusLine().getStatusCode();  
                if (status >= 200 && status < 300) {  
                    HttpEntity entity = response.getEntity();  
                    String resopnse="";  
                    if(entity != null)  
                    {  
                        resopnse=EntityUtils.toString(entity,"utf-8");  
                    }  
                    sysoutLog(currentTime+" 接收响应：url"+url+" status="+status);  
                    return entity != null ? resopnse : null;  
                } else {  
                    HttpEntity entity = response.getEntity();  
                    httpPost.abort();  
                    sysoutLog(currentTime+" 接收响应：url"+url+" status="+status+" resopnse="+EntityUtils.toString(entity,"utf-8"));  
                    throw new ClientProtocolException("Unexpected response status: " + status);  
                }  
            } catch (Exception e) {  
                httpPost.abort();  
                log.error(" Exception"+e.toString()+" url="+url+" jsonstr="+jsonStr);  
            }   
            return returnStr;  
        }