Apache Http Server与Tomcat实现负载均衡和集群

àì夳堔傛蜴生んèń 2022-06-11 00:41 222阅读 0赞

# 一、分布式实现原理 #

![image001.gif][]

如上图所示，主要通过Apache-Server作为中转服务器，实现多个tomcat服务器之间的分布式处理，用户直接请求Apache-Server，然后Apache-Server会将请求分发到具体的tomcat-server，之后tomcat-server响应客户请求并返回结果到Apache-Server，最后Apache-Server返回结果给用户

# 二、负载均衡 #

**文件说明：**

mod\_jk.conf

主要定义mod\_jk模块的位置以及mod\_jk模块的连接日志设置，还有定义worker.properties文件的位置。

worker.properties   
定义worker的参数，主要是连接tomcat主机的地址和端口信息。如果Tomcat与apache不在同一台机器上，或者需要做多台机器上tomcat的负载均衡只需要更改workers.properties文件中的相应定义即可。

％APACHE\_HOME％为你的安装目录

**环境说明：**

主要使用了一个Apache Server和两个Tomcat5.5，在同一台电脑上进行测试。

## 1. 准备软件 ##

Jdk1.6

下载地址：[http://java.sun.com][http_java.sun.com]

tomcat -6.0.29

下载地址：[http://jakarta.apache.org][http_jakarta.apache.org]

apache\_2.2.4-win32-x86-no\_ssl.msi

下载地址：[http://httpd.apache.org/download.cgi][http_httpd.apache.org_download.cgi]

mod\_jk-1.2.31-httpd-2.0.52.so **(****主要作用是建立Apache Server与Tomcat之间的连接)**

下载地址：[http://www.apache.org/dist/tomcat/tomcat-connectors/jk/binaries/][http_www.apache.org_dist_tomcat_tomcat-connectors_jk_binaries]

说明：apache-server安装完成后，可以在浏览器中输入[http://localhost/][http_localhost]来测试，如果出现**” It works!”**则表示安装成功。

##    2. 安装mod\_jk连接模块 ##

安装好Jdk、tomcat、apache后,加入mod\_jk连接模块，就是把mod\_jk-1.2.31-httpd-2.2.3.so文件拷贝到％APACHE\_HOME％\\modules下，把jk模块的配置放到单独的文件中来，在％APACHE\_HOME％\\conf目录新建mod\_jk.conf、workers.properties文件。

在httpd.conf最后加上**：**

\# JK module settings

Include conf/mod\_jk.conf

说明：以上表示将mod\_jk.conf配置文件包含进来

##    3. 修改mod\_jk.conf文件 ##

为了保持httpd.conf文件的简洁，把jk模块的配置放到单独的文件中来。

在mod\_jk.conf文件中添加以下内容：

\# Load mod\_jk2 module

LoadModule jk\_module modules/mod\_jk-1.2.31-httpd-2.2.3.so

\# Where to find workers.properties(引用workers配置文件)

JkWorkersFile conf/workers.properties

\# Where to put jk logs(log文件路径)

JkLogFile logs/mod\_jk2.log

\# Set the jk log level \[debug/error/info\](log级别)

JkLogLevel info

\# Select the log format(log格式)

JkLogStampFormat "\[%a %b %d %H:%M:%S %Y\] "

\# JkOptions indicate to send SSL KEY SIZE,

JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

\# JkRequestLogFormat set the request format

JkRequestLogFormat "%w %V %T"

\# Send JSPs for context / to worker named loadBalancer(URL转发配置，匹配的URL才转发到tomcat进行处理)

JkMount /\*.jsp controller

\# JkMount /\*.\* loadBalancer

## 4. 修改workers.properties文件 ##

在workers.properties文件中添加以下内容：

\#server 列表

worker.list = controller,tomcat1,tomcat2

\# tomcat1(ajp13 端口号，在tomcat下server.xml配置,默认8009)

worker.tomcat1.port=8009

\#tomcat的主机地址，如不为本机，请填写ip地址

worker.tomcat1.host=localhost

worker.tomcat1.type=ajp13

\#server的加权比重，值越高，分得的请求越多

worker.tomcat1.lbfactor = 1

\# tomcat2

worker.tomcat2.port=9009

worker.tomcat2.host=localhost

worker.tomcat2.type=ajp13

worker.tomcat2.lbfactor = 1

\# controller(负载均衡控制器)

worker.controller.type=lb

\# 指定分担请求的tomcat

worker.controller.balanced\_workers=tomcat1,tomcat2

\#worker.controller.sticky\_session=true

说明：此文件配置了2个tomcat服务器进行负载均衡处理，**sticky\_session**表述是否将对SESSION ID的请求路由回到相同的Tomcat worker。如果属性值不为0，它将被设置为JK\_TRUE，session将是粘性的，即SESSION ID的请求路由回到相同的Tomcat worker；当Tomcat正使用能够跨越多个Tomcat实例持久化session数据的Session Manager时，它将被设置为JK\_FALSE

## 5. 修改tomcat配置文件server.xml ##

**       **更改其中一个的设置打开tomcat2/conf/server.xml文件，修改里面所有的端口设置，将8改为9，如下：![image002.png][]        
![image004.png][]   
![image006.png][]

##    6. 编写一个测试页面teat1.jsp ##

建立一个test的web应用，里面新建一个test1.jsp,内容为

![ExpandedBlockStart.gif][]<%@ page language="java" contentType="text/html; charset=UTF-8"  
![InBlock.gif][]  
![ExpandedBlockEnd.gif][]    pageEncoding="UTF-8"%>  
![None.gif][]  
![None.gif][]<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">  
![None.gif][]  
![None.gif][]<html>  
![None.gif][]  
![None.gif][]<head>  
![None.gif][]  
![None.gif][]<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">  
![None.gif][]  
![None.gif][]<title>test1</title>  
![None.gif][]  
![None.gif][]</head>  
![None.gif][]  
![None.gif][]<body>  
![None.gif][]  
![ExpandedBlockStart.gif][]<%   
![InBlock.gif][]  
![InBlock.gif][]   System.out.println("===========================");  
![ExpandedBlockEnd.gif][]  
![None.gif][]%>  
![None.gif][]  
![None.gif][]</body>  
![None.gif][]  
![None.gif][]</html>

##    7. 启动服务器并进行测试 ##

依次启动apache-server、tomcat1、tomcat2，通过 [http://localhost/test/test1.jsp][http_localhost_test_test1.jsp] 访问，查看tomcat1的窗口，可以看到打印了一行"=========="，再刷新一次，tomcat2也打印了一条，再刷新，可以看到请求会被tomcat1,tomcat2轮流处理,实现了负载均衡

![image008.png][]

#    三、集群(session复制) #

只配置负载均衡还不行，还要session复制，也就是说其中任何一个tomcat的添加的session，是要同步复制到其它tomcat， 集群内的tomcat都有相同的session

## 1. tomcat配置 ##

1.1 修改tomcat1, tomcat2的server.xml文件添加集群内容，tomcat5.5无需添加，只需要去掉注释符，tomcat6.0需要添加，内容如下：

**Tomcat5.5配置**

![None.gif][]<Cluster className="org.apache.catalina.cluster.tcp.SimpleTcpCluster"  
![None.gif][]  
![None.gif][]                 managerClassName="org.apache.catalina.cluster.session.DeltaManager"  
![None.gif][]  
![None.gif][]                 expireSessionsOnShutdown="false"  
![None.gif][]  
![None.gif][]                 useDirtyFlag="true"  
![None.gif][]  
![None.gif][]                 notifyListenersOnReplication="true">  
![None.gif][]  
![None.gif][]   
![None.gif][]  
![None.gif][]            <Membership   
![None.gif][]  
![None.gif][]                className="org.apache.catalina.cluster.mcast.McastService"  
![None.gif][]  
![None.gif][]                mcastAddr="228.0.0.4"  
![None.gif][]  
![None.gif][]                mcastPort="45564"  
![None.gif][]  
![None.gif][]                mcastFrequency="500"  
![None.gif][]  
![None.gif][]                mcastDropTime="3000"/>  
![None.gif][]  
![None.gif][]   
![None.gif][]  
![None.gif][]            <Receiver   
![None.gif][]  
![None.gif][]                className="org.apache.catalina.cluster.tcp.ReplicationListener"  
![None.gif][]  
![None.gif][]                tcpListenAddress="auto"  
![None.gif][]  
![None.gif][]                tcpListenPort="4001"  
![None.gif][]  
![None.gif][]                tcpSelectorTimeout="100"  
![None.gif][]  
![None.gif][]                tcpThreadCount="6"/>  
![None.gif][]  
![None.gif][]   
![None.gif][]  
![None.gif][]            <Sender  
![None.gif][]  
![None.gif][]                className="org.apache.catalina.cluster.tcp.ReplicationTransmitter"  
![None.gif][]  
![None.gif][]                replicationMode="pooled"  
![None.gif][]  
![None.gif][]                ackTimeout="15000"  
![None.gif][]  
![None.gif][]                waitForAck="true"/>  
![None.gif][]  
![None.gif][]   
![None.gif][]  
![None.gif][]            <Valve className="org.apache.catalina.cluster.tcp.ReplicationValve"  
![None.gif][]  
![None.gif][]                   filter=".\*\\.gif;.\*\\.js;.\*\\.jpg;.\*\\.png;.\*\\.htm;.\*\\.html;.\*\\.css;.\*\\.txt;"/>  
![None.gif][]  
![None.gif][]                     
![None.gif][]  
![None.gif][]            <Deployer className="org.apache.catalina.cluster.deploy.FarmWarDeployer"  
![None.gif][]  
![None.gif][]                      tempDir="/tmp/war-temp/"  
![None.gif][]  
![None.gif][]                      deployDir="/tmp/war-deploy/"  
![None.gif][]  
![None.gif][]                      watchDir="/tmp/war-listen/"  
![None.gif][]  
![None.gif][]                      watchEnabled="false"/>  
![None.gif][]  
![None.gif][]                        
![None.gif][]  
![None.gif][]                  <ClusterListener className="org.apache.catalina.cluster.session.ClusterSessionListener"/>  
![None.gif][]  
![None.gif][]        </Cluster>

**Tomcat6.0配置**

![None.gif][]<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"  
![None.gif][]          channelSendOptions="6">  
![None.gif][]     
![None.gif][]    <Manager className="org.apache.catalina.ha.session.DeltaManager"   
![None.gif][]      expireSessionsOnShutdown="false"   
![None.gif][]      notifyListenersOnReplication="true"/>   
![None.gif][]     
![None.gif][]   <Channel className="org.apache.catalina.tribes.group.GroupChannel">   
![None.gif][]       <Membership className="org.apache.catalina.tribes.membership.McastService"   
![None.gif][]             address="228.0.0.4"   
![None.gif][]            port="45564"   
![None.gif][]             frequency="500"   
![None.gif][]             dropTime="3000"/>   
![None.gif][]       <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"   
![None.gif][]         address="auto"   
![None.gif][]         port="5000"   
![None.gif][]         selectorTimeout="100"   
![None.gif][]         maxThreads="6"/>   
![None.gif][]    
![None.gif][]       <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">   
![None.gif][]          <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>   
![None.gif][]       </Sender>   
![None.gif][]       <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>   
![None.gif][]       <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>   
![None.gif][]       <Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/>   
![None.gif][]   </Channel>   
![None.gif][]   <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"   
![None.gif][]      filter=".\*\\.gif;.\*\\.js;.\*\\.jpg;.\*\\.png;.\*\\.htm;.\*\\.html;.\*\\.css;.\*\\.txt;"/>   
![None.gif][]   <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"   
![None.gif][]          tempDir="/tmp/war-temp/"   
![None.gif][]          deployDir="/tmp/war-deploy/"   
![None.gif][]          watchDir="/tmp/war-listen/"   
![None.gif][]          watchEnabled="false"/>   
![None.gif][]  
![None.gif][]       <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>   
![None.gif][]</Cluster>

分别添加以上内容后，在tomcat2中，修改tcpListenPort="4001" 为4002

1.2. Engine增加jvmRoute属性设置，jvmRoute的值来自于workers.properties文件所设置的服务器名称。

\#server 列表

worker.list = controller,tomcat1,tomcat2

## 2. 添加test.jsp页面 ##

2.1. test.jsp添加以下内容：

![ExpandedBlockStart.gif][]<%@ page contentType="text/html; charset=UTF-8"%>  
![None.gif][]  
![ExpandedBlockStart.gif][]<%@ page import="java.util.\*"%>  
![None.gif][]  
![None.gif][]<html>  
![None.gif][]  
![None.gif][]<head>  
![None.gif][]  
![None.gif][]<title>Cluster App Test</title>  
![None.gif][]  
![None.gif][]</head>  
![None.gif][]  
![None.gif][]<body>  
![None.gif][]  
![None.gif][]Server Info:  
![None.gif][]  
![ExpandedBlockStart.gif][]<%  
![InBlock.gif][]  
![InBlock.gif][]  out.println(request.getLocalAddr() + " : " + request.getLocalPort() + "<br>");  
![ExpandedBlockEnd.gif][]  
![None.gif][]%>  
![None.gif][]  
![ExpandedBlockStart.gif][]<%  
![InBlock.gif][]  
![InBlock.gif][]  out.println("<br> ID " + session.getId() + "<br>");  
![InBlock.gif][]  
![InBlock.gif][]  // 如果有新的 Session 属性设置  
![InBlock.gif][]  
![InBlock.gif][]  String dataName = request.getParameter("dataName");  
![InBlock.gif][]  
![InBlock.gif][]  if (dataName != null && dataName.length() > 0) \{  
![InBlock.gif][]  
![InBlock.gif][]     String dataValue = request.getParameter("dataValue");  
![InBlock.gif][]  
![InBlock.gif][]     session.setAttribute(dataName, dataValue);  
![InBlock.gif][]  
![InBlock.gif][]  \}  
![InBlock.gif][]  
![InBlock.gif][]  out.print("<b>Session 列表</b>");  
![InBlock.gif][]  
![InBlock.gif][]  Enumeration e = session.getAttributeNames();  
![InBlock.gif][]  
![InBlock.gif][]  while (e.hasMoreElements()) \{  
![InBlock.gif][]  
![InBlock.gif][]     String name = (String) e.nextElement();  
![InBlock.gif][]  
![InBlock.gif][]     String value = session.getAttribute(name).toString();  
![InBlock.gif][]  
![InBlock.gif][]     out.println(name + " = " + value + "<br>");  
![InBlock.gif][]  
![InBlock.gif][]     System.out.println(name + " = " + value);  
![InBlock.gif][]  
![InBlock.gif][]  \}  
![ExpandedBlockEnd.gif][]  
![None.gif][]%>  
![None.gif][]  
![None.gif][]<form action="test.jsp" method="POST">  
![None.gif][]  
![None.gif][]名称:<input type=text size=20 name="dataName"> <br>  
![None.gif][]  
![None.gif][]值:<input type=text size=20 name="dataValue"> <br>  
![None.gif][]  
![None.gif][]<input type=submit></form>  
![None.gif][]  
![None.gif][]</body>  
![None.gif][]  
![None.gif][]</html>

2.2. 修改web.xml文件，加入**<distributable/>**节点，如下所示：

![None.gif][]<?xml version="1.0" encoding="UTF-8"?>  
![None.gif][]  
![None.gif][]<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app\_2\_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app\_2\_5.xsd" id="WebApp\_ID" version="2.5">  
![None.gif][]  
![None.gif][]  <display-name>test</display-name>  
![None.gif][]  
![None.gif][]  <distributable/>  
![None.gif][]  
![None.gif][]  <welcome-file-list>  
![None.gif][]  
![None.gif][]    <welcome-file>index.html</welcome-file>  
![None.gif][]  
![None.gif][]    <welcome-file>index.htm</welcome-file>  
![None.gif][]  
![None.gif][]    <welcome-file>index.jsp</welcome-file>  
![None.gif][]  
![None.gif][]    <welcome-file>default.html</welcome-file>  
![None.gif][]  
![None.gif][]    <welcome-file>default.htm</welcome-file>  
![None.gif][]  
![None.gif][]    <welcome-file>default.jsp</welcome-file>  
![None.gif][]  
![None.gif][]  </welcome-file-list>  
![None.gif][]  
![None.gif][]</web-app>

## 3. Session复制测试 ##

测试步骤如下：

1)        启动apache-server、tomcat1、tomcat2

2)        访问[http://localhost/test/test.jsp][http_localhost_test_test.jsp]，输入名称：test0001、值：123并点击“提交查询内容”按钮，显示效果如下：

如上图所示，tomcat1创建了一个新的session，session中有属性test0001,值为123  
 ![image010.png][]

3)        关闭tomcat1服务器，tomcat1端口为8080，如下图：  
![image012.png][]

4)        在页面中再次点击“提交查询内容”按钮，效果如下：  
![image014.png][]  
    前端页面并没有发生改变，接下来查看后台情况：  
 ![image016.png][]     
  
    如图所示，可以发现session已成功复制到tomcat2中，以此证明tomcat集群已配置成功。

5)        另外来看看不关闭tomcat1服务器再次提交的情况  
![image018.png][]  
       如图所示，请求并没有转发到tomcat2服务器，而是再次转回tomcat1服务器，这种情况是由于配置了jvmRoute所致，以个人理解，配置了此属性后，apache-server会根据session情况来进行路由，同一个session会转发给同一个服务器。

6)        打开一个新的IE窗口，并访问[http://localhost/test/test.jsp][http_localhost_test_test.jsp]

![image020.png][]  
              ![image022.png][]   
             新窗口的请求转发到了tomcat2服务器，session的id为DD9E6C8181653B9BCCF534FC8760B264.tomcat2，根据测试结果可以说明，在不发生服务器关闭的情况下，每个session会绑定到同一个服务器中，而不会在服务器间发生复制。

[image001.gif]: /images/20220611/a6ddb24afab042b6b2f5bb569f523b40.png
[http_java.sun.com]: http://java.sun.com/
[http_jakarta.apache.org]: http://jakarta.apache.org/
[http_httpd.apache.org_download.cgi]: http://httpd.apache.org/download.cgi
[http_www.apache.org_dist_tomcat_tomcat-connectors_jk_binaries]: http://www.apache.org/dist/tomcat/tomcat-connectors/jk/binaries/
[http_localhost]: http://localhost/
[image002.png]: /images/20220611/1534117030da4dfaae89d1f65a7db12c.png
[image004.png]: /images/20220611/5e61054532d5463499f206b27e419a81.png
[image006.png]: /images/20220611/b54946904ff948d7a654d7181180d4ef.png
[ExpandedBlockStart.gif]: /images/20220611/fa1058778ed14384a3bfe1adef2d9d70.png
[InBlock.gif]: /images/20220611/596eac975f9348e4b35a64a67afee24c.png
[ExpandedBlockEnd.gif]: /images/20220611/42ea525ece7c494ea19605a6ca7628bf.png
[None.gif]: /images/20220611/41de4b1153964c53a593599331c02682.png
[http_localhost_test_test1.jsp]: http://localhost/test/test1.jsp
[image008.png]: /images/20220611/abfbc1dec64146efb467a8c39b8c1993.png
[http_localhost_test_test.jsp]: http://localhost/test/test.jsp
[image010.png]: /images/20220611/0b23ee19b2994f4988633c9126d55a76.png
[image012.png]: /images/20220611/13154c4dcd7045cd89f1a3c6170fd8a9.png
[image014.png]: /images/20220611/49f1e255be1f472bbcea1841e403d6f9.png
[image016.png]: /images/20220611/af75ba6f4450452fb954d49b0861d752.png
[image018.png]: /images/20220611/24c7ecbb268d42799c64ab7b12ce9350.png
[image020.png]: /images/20220611/bfafebb0f4f84bc395688ea2f68dc399.png
[image022.png]: /images/20220611/ad05530be4bc46eda597df6db2940ac2.png