整合apache+tomcat+keepalived实现高可用tomcat集群


Apache是一个强大的Web服务器在处理静态页面、处理大量网络客户请求、支持服务的种类以及可配置方面都有优势,高速并且强壮。但是没有JSP/Servlet的解析能力。整合Apache和Tomcat可以看作是用Tomcat做Apache的jsp/servlet解析插件,将两者优势结合起来。不过Tomcat作为一个Web服务器,本身具备了基本的Web服务功能,在SUN的力推下,将来或许越来越强壮到不需要借助Apache优势的地步。

Tomcat是一个免费的开放源代码的Web 应用服务器,属于轻量级应用服务器,在中小型系统和并发访问用户不是很多的场合下被普遍使用,是开发和调试JSP 程序的首选。Tomcat 很受广大程序员的喜欢,因为它运行时占用的系统资源小,扩展性好,支持负载平衡与邮件服务等开发应用系统常用的功能;而且它还在不断的改进和完善中,任何一个感兴趣的程序员都可以更改它或在其中加入新的功能。

Apache与tomcat的异同:

apache支持静态页,tomcat支持动态的,比如servlet等。

一般使用apache+tomcat的话,apache只是作为一个转发,对jsp的处理是由tomcat来处理的。

apache可以支持php\cgi\perl,但是要使用java的话,你需要tomcat在apache后台支撑,将java请求由apache转发给tomcat处理。

apache是web服务器,Tomcat是应用(java)服务器,它只是一个servlet(jsp也翻译成servlet)容器,可以认为是apache的扩展,但是可以独立于apache运行。

Apache是专门用了提供HTTP服务的,以及相关配置的(例如虚拟主机、URL转发等等)。

Tomcat是Apache组织在符合J2EE的JSP、Servlet标准下开发的一个JSP服务器;

Apache与tomcat整合的好处

如果客户端请求的是静态页面,则只需要Apache服务器响应请求。

如果客户端请求动态页面,则是Tomcat服务器响应请求。

因为jsp是服务器端解释代码的,这样整合就可以减少Tomcat的服务开销。

 

一.环境与功能说明

前端:apache配置负载均衡、反向代理(mod_jk模块)

后端:tomcat服务器

前端高可用:通过keepalived实现

 

系统版本:CentOS 6.4_x86_64(最小化)

apache版本:httpd-2.4.9

arp版本:apr-1.4.6

apr-util版本:apr-util-1.4.1

tomcat版本:apache-tomcat-7.0.53

Jdk版本:jdk-7u40-linux-x64

tomcat-commectors版本:tomcat-connectors-1.2.40

keepalived版本:keepalived-1.2.7-3.el6.x86_64(使用yum安装)

 

二.安装与配置apache服务器(172.16.10.72、172.16.10.75执行同样的安装与配置,这里只演示一次)

1.安装编译工具和依赖包

yum groupinstall "Development Tools" "Development Libraries" –y

2.下载软件包

yum -y install wget
wget http://mirror.esocc.com/apache//httpd/httpd-2.4.9.tar.gz
wget http://mirrors.axint.net/apache//apr/apr-1.4.6.tar.gz
wget http://mirrors.axint.net/apache//apr/apr-util-1.4.1.tar.gz

3.安装apache

3.1.安装apr和apr-tuil

httpd-2.4.9需要较新版本的apr和apr-util,因此需要事先对其进行升级。升级方式有两种,一种是通过源代码编译安装,一种是直接升级rpm包。以下是采用前者进行安装:

# tar xf apr-1.4.6.tar.gz # tar xf apr-util-1.4.1.tar.gz # cd apr-1.4.6 # ./configure --prefix=/usr/local/apr # make # make install # cd .. # cd apr-util-1.4.1 # ./configure --prefix=/usr/local/apr-util --with-apr=/usr/local/apr # make # make install # cd ..

3.2.安装http-2.4.9

# ./configure --prefix=/usr/local/apache --sysconfdir=/etc/httpd24 --enable-so --enable-ssl --enable-cgi --enable-rewrite --with-zlib --with-pcre --with-apr=/usr/local/apr --with-apr-util=/usr/local/apr-util/ --enable-modules=most --enable-mpms-shared=all # make && make install

4.配置SysV服务脚本/etc/init.d/httpd

# cp build/rpm/httpd.init /etc/init.d/httpd #修改如下内容: 
#
vi /etc/init.d/httpd httpd=${HTTPD-/usr/local/apache/bin/httpd} pidfile=${PIDFILE-/usr/local/apache/logs/${prog}.pid} lockfile=${LOCKFILE-/var/lock/subsys/${prog}} RETVAL=0 # check for 1.3 configuration check13 () { CONFFILE=/etc/httpd24/httpd.conf


#修改环境变量并把Httpd加入到系统服务

  # echo "PATH=/usr/local/apache/bin:$PATH" >> /etc/profile.d/http.sh
  # . /etc/profile.d/http.sh
  # ln -s /usr/local/apache/include/ /usr/include/httpd
  # chkconfig --add httpd

 5.安装tomcat-connectors

# wget http://apache.fayea.com/apache-mirror/tomcat/tomcat-connectors/jk/tomcat-connectors-1.2.40-src.tar.gz
# tar xf tomcat-connectors-1.2.40-src.tar.gz # cd tomcat-connectors-1.2.40-src/native/ # ./configure --with-apxs=/usr/local/apache/bin/apxs # make # make install

安装完成后会生成如下文件

# ls /usr/local/apache/modules/|grep 'mod_jk' mod_jk.so

启动httpd服务

# service httpd start

执行./configure --with-apxs=/usr/local/apache/bin/apxs时可能会遇到以下报错:

need to check for Perl first, apxs depends on it...
checking for perl... /usr/bin/perl
could not find /usr/local/apache/bin/apxs
configure: error: You must specify a valid --with-apxs path

解法办法:

vi /usr/local/apache/bin/apxs
把第一行改成:#!/usr/bin/perl -w

另外检查系统中是否已经安装过perl,如果没有安装,用yum安装即可。

三.安装与配置tomcat服务器(172.16.10.61、172.16.10.62执行同样的安装,这里只演示一次)

1.安装JDK

下载地址:http://download.oracle.com/

# rpm -ivh jdk-7u40-linux-x64.rpm # vim /etc/profile.d/java.sh export JAVA_HOME=/usr/java/latest export PATH=$JAVA_HOME/bin:$PATH # . /etc/profile.d/java.sh # java -version java version "1.7.0_40" Java(TM) SE Runtime Environment (build 1.7.0_40-b43) Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)

2.安装Tomcat

下载地址:http://apache.fayea.com/apache-mirror/tomcat/tomcat-7/v7.0.53/bin/apache-tomcat-7.0.53.tar.gz

# tar xf apache-tomcat-7.0.53.tar.gz -C /usr/local/ # cd /usr/local/ # ln -s apache-tomcat-7.0.53 tomcat # vim /etc/profile.d/tomcat.sh export CATALINA_HOME=/usr/local/tomcat export PATH=$CATALINA_HOME/bin:$PATH # . /etc/profile.d/tomcat.sh # /usr/local/tomcat/bin/catalina.sh version Using CATALINA_BASE: /usr/local/tomcat Using CATALINA_HOME: /usr/local/tomcat Using CATALINA_TMPDIR: /usr/local/tomcat/temp Using JRE_HOME: /usr/java/latest Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar Server version: Apache Tomcat/7.0.53 Server built: Mar 25 2014 06:20:16 Server number: 7.0.53.0 OS Name: Linux OS Version: 2.6.32-279.el6.x86_64 Architecture: amd64 JVM Version: 1.7.0_40-b43 JVM Vendor: Oracle Corporation

3.启动Tomcat服务并访问测试,默认访问端口为8080:

# /usr/local/tomcat/bin/catalina.sh start Using CATALINA_BASE: /usr/local/tomcat Using CATALINA_HOME: /usr/local/tomcat Using CATALINA_TMPDIR: /usr/local/tomcat/temp Using JRE_HOME: /usr/java/latest Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar Tomcat started. # ss -antpl | grep 8080
0      100                           :::8080                         :::*      users:(("java",1299,42))

4.开启Tomcat管理及状态页面

# vim /usr/local/tomcat/conf/tomcat-users.xml(在</tomcat-users>标签上一行添加如下两行) <role rolename="manager-gui"/>
  <user username="tomcat" password="tomcat" roles="manager-gui"/> #后台管理页面登录名及密码均为tomcat # /usr/local/tomcat/bin/catalina.sh stop # /usr/local/tomcat/bin/catalina.sh start

5.访问Tomcat后台管理

四.配置Apache使用mod_jk模块实现代理及负载均衡

1.编辑Apache主配置文件,实现代理功能(172.16.10.72)

echo "Include /etc/httpd24/extra/mod_jk.conf" >> /etc/httpd24/httpd.conf  #将此指令添加到文件末尾即可

2.创建mod_jk.conf及workers.properties文件

# cat >>/etc/httpd24/extra/mod_jk.conf <<EOF LoadModule jk_module modules/mod_jk.so JkWorkersFile /etc/httpd24/extra/workers.properties JkLogFile logs/mod_jk.log JkLogLevel debug JkMount /* TomcatA JkMount /status/ stat1 EOF # cat >>/etc/httpd24/extra/workers.properties <<EOF worker.list=TomcatA,stat1 worker.TomcatA.port=8009 worker.TomcatA.host=172.16.10.61 worker.TomcatA.type=ajp13 worker.TomcatA.lbfactor=1 worker.stat1.type = status EOF

3.编辑httpd.conf修改ServerName并重启httpd服务

#vi /etc/httpd24/httpd.conf
ServerName 172.16.10.72:80
# service httpd restart

4.修改tomcat服务器(172.16.10.61),添加测试页面

# cd /usr/local/tomcat/conf/ # cp server.xml server.xml.bak # vi server.xml <Engine name="Catalina" defaultHost="172.16.10.72">
<Host name="172.16.10.72"  appBase="/web/eivll0m" unpackWARs="true" autoDeploy="true">
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="eivll0m_access_log." suffix=".txt" pattern="%h %l %u %t &quot;%r&quot; %s %b" />
        <Context path="" docBase="/web/eivll0m" />
      </Host>

创建网站存放目录并创建测试页

# mkdir -p /web/eivll0m # vi /web/eivll0m/index.jsp <%@ page language="java" %>
<html>
  <head><title>TomcatA</title></head>
  <body>
    <h1><font color="red">TomcatA </font></h1>
    <table align="centre" border="1">
      <tr>
        <td>Session ID</td>
    <% session.setAttribute("abc","abc"); %>
        <td><%= session.getId() %></td>
      </tr>
      <tr>
        <td>Created on</td>
        <td><%= session.getCreationTime() %></td>
     </tr>
    </table>
  </body>
</html>

重启tomcat服务并查看是否启动成功

# catalina.sh stop # catalina.sh configtest # catalina.sh start # ss -antpl | grep java 0      1               ::ffff:127.0.0.1:8005                         :::*      users:(("java",1299,50)) 0      100                           :::8009                         :::*      users:(("java",1299,43)) 0      100                           :::8080                         :::*      users:(("java",1299,42))

5.访问Apache主机(172.16.10.72),验证是否代理成功

 6.修改Apache配置文件(mod_jk.conf、workers.properties),实现负载均衡

# cat >>/etc/httpd24/extra/mod_jk.conf <<EOF LoadModule jk_module modules/mod_jk.so JkWorkersFile /etc/httpd24/extra/workers.properties JkLogFile logs/mod_jk.log JkLogLevel debug JkMount /* lbcluster JkMount /status/ stat1 EOF 
# cat >>/etc/httpd/extra/workers.properties <<EOF worker.list=lbcluster,stat1 worker.TomcatA.port=8009 worker.TomcatA.host=172.16.10.61 worker.TomcatA.type=ajp13 worker.TomcatA.lbfactor=1 worker.TomcatB.port = 8009 worker.TomcatB.host=172.16.10.62 worker.TomcatB.type = ajp13 worker.TomcatB.lbfactor = 1 worker.lbcluster.type = lb worker.lbcluster.sticky_session = 0 #取值为{1|0}1将用户session与后端服务器绑定,0为不绑定,如果支持session复制或session共享可以设置为0 worker.lbcluster.balance_workers = TomcatA, TomcatB worker.stat1.type = status EOF

7.修改tomcat服务器(172.16.10.62),添加测试页面

# cd /usr/local/tomcat/conf/ # cp server.xml server.xml.bak # vi server.xml <Engine name="Catalina" defaultHost="172.16.10.72">
<Host name="172.16.10.72"  appBase="/web/eivll0m" unpackWARs="true" autoDeploy="true">
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="eivll0m_access_log." suffix=".txt" pattern="%h %l %u %t &quot;%r&quot; %s %b" />
        <Context path="" docBase="/web/eivll0m" />
      </Host>

创建网站存放目录并创建测试页

# mkdir -p /web/eivll0m # vi /web/eivll0m/index.jsp <%@ page language="java" %>
<html>
  <head><title>TomcatB</title></head>
  <body>
    <h1><font color="blue">TomcatB </font></h1>
    <table align="centre" border="1">
      <tr>
        <td>Session ID</td>
    <% session.setAttribute("abc","abc"); %>
        <td><%= session.getId() %></td>
      </tr>
      <tr>
        <td>Created on</td>
        <td><%= session.getCreationTime() %></td>
     </tr>
    </table>
  </body>
</html>

重启tomcat服务并查看是否启动成功

# catalina.sh stop # catalina.sh configtest # catalina.sh start # ss -antpl | grep java 0      1               ::ffff:127.0.0.1:8005                         :::*      users:(("java",1303,50)) 0      100                           :::8009                         :::*      users:(("java",1303,43)) 0      100                           :::8080                         :::*      users:(("java",1303,42))

8.访问Apache服务器(172.16.10.72),测试是否实现负载均衡(如下表示测试成功)

五.安装并配置Keepalived实现高可用

1.在apache服务器(172.16.10.72、172.16.10.75)安装keepalived

yum -y install keepalived

2.配置Keepalived(172.16.10.72)

# vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } notification_email_from admin@eivll0m.com smtp_server 172.16.10.0 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_httpd { script "killall -0 httpd" interval 1 weight -2 } vrrp_instance httpd_1 { state MASTER interface eth0 virtual_router_id 58 priority 100 #定义优先级 advert_int 1 authentication { auth_type PASS auth_pass 1058 } virtual_ipaddress { 172.16.10.77 #vip } track_script { chk_httpd } }

# service keepalived start #启动keepalived服务
# chkconfig keepalived on

3.配置Keepalived(172.16.10.75)

# vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } notification_email_from admin@eivll0m.com smtp_server 172.17.10.0 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_httpd { script "killall -0 httpd" interval 1 weight -2 } vrrp_instance httpd_1 { state BACKUP interface eth0 virtual_router_id 58 priority 99 #定义优先级 advert_int 1 authentication { auth_type PASS auth_pass 1058 } virtual_ipaddress { 172.16.10.77 #vip } track_script { chk_httpd } }
# service keepalived start  
# chkconfig keepalived on

4.编辑httpd.conf(172.16.10.72),并拷贝至172.16.10.75

# vi /etc/httpd24/httpd.conf ServerName www.eivll0m.com:80 # service httpd restart # cd /etc/httpd24/ # scp httpd.conf 172.16.10.75:/etc/httpd24/ # cd extra/ # scp mod_jk.conf workers.properties 172.16.10.75:/etc/httpd24/extra/
# service httpd restart #两台apache服务器分别执行

 5.编辑tomcat服务器(172.17.10.61和172.16.10.62)的server.xml

# vi server.xml <Engine name="Catalina" defaultHost="www.eivll0m.com">
<Host name="www.eivll0m.com"  appBase="/web/eivll0m" unpackWARs="true" autoDeploy="true">
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="eivll0m_access_log." suffix=".txt" pattern="%h %l %u %t &quot;%r&quot; %s %b" />
        <Context path="" docBase="/web/eivll0m" />
      </Host>
# catalina.sh stop
# catalina.sh start

5.通过www.eivll0m.com(172.16.10.77)进行访问测试

注意:测试时关闭所有服务器iptables,如果没有做DNS解析,可在测试机上修改hosts文件来实现。

 

6.模拟一台apache故障,验证keepalived是否可以成功转移服务

# ip addr show eth0   #172.16.10.72上执行 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:e9:b8:52 brd ff:ff:ff:ff:ff:ff inet 172.16.10.72/24 brd 172.16.10.255 scope global eth0 inet 172.16.10.77/32 scope global eth0 inet6 fe80::20c:29ff:fee9:b852/64 scope link tentative dadfailed valid_lft forever preferred_lft forever # ip addr show eth0 #172.16.10.75上执行 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:c8:1a:08 brd ff:ff:ff:ff:ff:ff inet 172.16.10.75/24 brd 172.16.10.255 scope global eth0 inet 192.168.200.16/32 scope global eth0 inet 192.168.200.17/32 scope global eth0 inet 192.168.200.18/32 scope global eth0 inet6 fe80::20c:29ff:fec8:1a08/64 scope link tentative dadfailed valid_lft forever preferred_lft forever # service httpd stop #停止172.16.10.72上的httpd服务 # ip addr show eth0 #172.16.10.72上执行 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:e9:b8:52 brd ff:ff:ff:ff:ff:ff inet 172.16.10.72/24 brd 172.16.10.255 scope global eth0 inet6 fe80::20c:29ff:fee9:b852/64 scope link tentative dadfailed valid_lft forever preferred_lft forever # ip addr show eth0 #172.16.10.75上执行 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:c8:1a:08 brd ff:ff:ff:ff:ff:ff inet 172.16.10.75/24 brd 172.16.10.255 scope global eth0 inet 192.168.200.16/32 scope global eth0 inet 192.168.200.17/32 scope global eth0 inet 192.168.200.18/32 scope global eth0 inet 172.16.10.77/32 scope global eth0 #vip已经成功转移过来 inet6 fe80::20c:29ff:fec8:1a08/64 scope link tentative dadfailed valid_lft forever preferred_lft forever

通过上面的步骤我们可以看到,在172.16.10.72上停止apache服务,keepalived会检测到,另一台keepalived服务器会接管vip,web服务不会中断,从而实现了高可用性。

7.配置tomcat实现session共享

7.1.配置172.16.10.61这台tomcat服务器

 

# vi /usr/local/tomcat/conf/server.xml  #在<Engine>标签内(对所有主机生效)添加或修改如下内容,另外也可以添加在<Host>标签内,对当前主机生效。并去掉此段前后注释
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="8">
          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>
          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership className="org.apache.catalina.tribes.membership.McastService"
                        address="224.0.0.1"   #组播地址
                        port="45564"
                        frequency="500"
                        dropTime="3000"/>
            <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="172.16.10.61"
                      port="4000"
                      autoBind="100"
                      selectorTimeout="5000"
                      maxThreads="6"/>
            <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
              <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
            </Sender>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
          </Channel>
          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 filter=""/>
          <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
          <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                    tempDir="/tmp/war-temp/"
                    deployDir="/tmp/war-deploy/"
                    watchDir="/tmp/war-listen/"
                    watchEnabled="false"/>
          <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
          <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>

 

# mkdir /web/eivll0m/WEB-INF
# cp /usr/local/tomcat/conf/web.xml /web/eivll0m/WEB-INF/
# vim /web/eivll0m/WEB-INF/web.xml
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
                      http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
  version="3.0">
<distributable/>                      #添加此行内容
# catalina.sh stop 
# catalina.sh start
# ss -tanlp | grep java
0      1               ::ffff:127.0.0.1:8005                         :::*      users:(("java",1935,55))
0      100                           :::8009                         :::*      users:(("java",1935,43))
0      100                           :::8080                         :::*      users:(("java",1935,42))
0      50           ::ffff:172.16.10.61:4000                         :::*      users:(("java",1935,44))

7.2.配置172.16.10.62这台tomcat服务器

# scp /usr/local/tomcat/conf/server.xml 172.16.10.62:/usr/local/tomcat/conf/
# scp -r /web/eivll0m/WEB-INF 172.16.10.62:/web/eivll0m/
# vi /usr/local/tomcat/conf/server.xml
                      address="172.16.10.62"   #更改IP
                      port="4000"
                      autoBind="100"
                      selectorTimeout="5000"
                      maxThreads="6"/>
# catalina.sh stop 
# catalina.sh start
# ss -antpl|grep java    
0      1               ::ffff:127.0.0.1:8005                         :::*      users:(("java",2041,61))
0      100                           :::8009                         :::*      users:(("java",2041,43))
0      100                           :::8080                         :::*      users:(("java",2041,42))
0      50           ::ffff:172.16.10.62:4000                         :::*      users:(("java",2041,44))

注:因为tomcat的session同步功能需要用到组播,windows默认情况下是开通组播服务的,但是linux默认情况下并没有开通,可以通过指令打开route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0,如果需要服务器启动时即开通组播需在/etc/sysconfig/static-routes文件内加入eht0 net 224.0.0.0 netmask 240.0.0.0。

 

7.3.验证session共享是否实现

 

通过以上截图可以看出session共享已经实现。


免责声明!

本站转载的文章为个人学习借鉴使用,本站对版权不负任何法律责任。如果侵犯了您的隐私权益,请联系本站邮箱yoyou2525@163.com删除。



 
粤ICP备18138465号  © 2018-2025 CODEPRJ.COM