zookeeper技术介绍 雨点打透心脏的1/2处 2022-05-28 11:50 145阅读 0赞 **Zookeeper技术介绍** **ZooKeeper是一个分布式的,开放源码的分布式应用程序协调服务**,是Google的Chubby一个开源的实现,是Hadoop和Hbase的重要组件。它是一个为分布式应用提供一致性服务的软件,提供的功能包括:配置维护、域名服务、分布式同步、组服务等。 ZooKeeper的目标就是封装好复杂易出错的关键服务,将简单易用的接口和性能高效、功能稳定的系统提供给用户。 ZooKeeper包含一个简单的原语集提供Java和C的接口。 ZooKeeper代码版本中,提供了分布式独享锁、选举、队列的接口,代码在zookeeper-3.4.3\\src\\recipes。其中分布锁和队列有Java和C两个版本,选举只有Java版本。 主要作用是用来解决分布式应用中经常遇到的一些数据管理问题,如集群管理、统一命名管理、分布式配置管理、分布式消息队列、分布式锁、分布式通知协调 **官方网站:https://zookeeper.apache.org/** zookeeper体系结构 zk的集群中有一个Leader,有一个或多个follower,它们之间是相互通信的,然后还存在一个客户端用来访问zk服务器。 服务器端具有fast fail特性,一旦主故障后,会选举从成为新的主,主从模式是目前最常见的模式。 zookeeper名字空间由节点NODE构成,类似于文件系统,其中各个节点相当于目录和文件,通过路径作为唯一标识,与文件系统不同的是,每个节点具有与之对应的数据内容,同时也可以具有子节点,它用来存储协调数据,如状态、配置、位置信息、每个节点存储的数据量很小,KB级别. **watches介绍** zk对节点的增、删、改、查都可以触发监听 watch事件是一次性触发器,当它监视的数据发生变化时,通知设置了该客户端 Zookeeper安装布署 **四台服务器** > zk-001 10.0.0.8 > > zk-002 10.0.0.9 > > zk-003 10.0.0.10 > > zk-client 10.0.0.100 **下载安装JDK环境** tar zxf jdk-8u60-linux-x64.tar.gz mv jdk1.8.0\_60 /usr/local/jdk export JAVA\_HOME=/usr/local/jdk export CLASSPATH=.:$JAVA\_HOME/lib/dt.jar:$JAVA\_HOME/lib/tools.jar export PATH=$JAVA\_HOME/bin:$PATH source /etc/profile java -version \[root@zk-001 ~\]\# java -version java version "1.8.0\_60" Java(TM) SE Runtime Environment (build 1.8.0\_60-b27) Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode) **下载安装zookeeper** wget http://www.apache.org/dist/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz tar zxf zookeeper-3.4.6.tar.gz mv zookeeper-3.4.6 /usr/local/zookeeper \[root@zk-001 ~\]\# cd /usr/local/zookeeper/ **配置环境变量** export ZOOKEEPER\_HOME=/usr/local/zookeeper export PATH=$ZOOKEEPER\_HOME/bin:$PATH source /etc/profile Zookeeper的配置 **三种配置模式** > 1、伪分布模式 > > 2、完全分布式 > > 3、独立模式 **独立模式配置** \[root@zk-001 zookeeper\]\# cd conf/ \[root@zk-001 conf\]\# ll total 12 \-rw-rw-r-- 1 1000 1000 535 Feb 20 2014 configuration.xsl \-rw-rw-r-- 1 1000 1000 2161 Feb 20 2014 log4j.properties \-rw-rw-r-- 1 1000 1000 922 Feb 20 2014 zoo\_sample.cfg \[root@zk-001 conf\]\# cp zoo\_sample.cfg zoo.cfg \[root@zk-001 conf\]\#vim zoo.cfg \# The number of milliseconds of each tick tickTime=2000 **\#心跳检查周期(单位毫秒)** \# the directory where the snapshot is stored. \# do not use /tmp for storage, /tmp here is just \# example sakes. dataDir=/tmp/zookeeper **\#存放数据目录** \# the port at which the clients will connect clientPort=2181 **\#供客户端连接的端口** **集群模式配置** tickTime=2000 initLimit=10 syncLimit=5 //server.n=host:port1:port2 **数字N必须是myid中的值,Myid文件位于dataDIR目录下,只能是一个n值** **//port1:leader端口,作为leader时,供follower连接的端口** **//port2:选举端口,选举leader时,供follower连接的端口** server.1=s1:2888:3888 server.2=s2:2888:3888 server.3=s3:2888:3888 **zk处理集群故障的算法是2n+1,最好在奇数机器数布署,如果同台服务器配置多实例就要注意端口冲突问题,可以写成如下** server.1=s1:2888:3888 server.2=s2:2889:3889 server.3=s3:2887:3887 配置过程如下 **配置hosts文件** vim /etc/hosts 10.0.0.8 zk-001 10.0.0.9 zk-002 10.0.0.10 zk-003 **配置各服务器配置文件** \[root@zk-001 conf\]\# mkdir /tmp/zookeeper \[root@zk-001 conf\]\# cd /tmp/zookeeper/ \[root@zk-001 zookeeper\]\# echo 8 >myid \[root@zk-001 conf\]\# egrep -v "^\#|^$" zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 dataDir=/tmp/zookeeper clientPort=2181 server.8=zk-001:2888:3888 server.9=zk-002:2888:3888 server.10=zk-003:2888:3888 \[root@zk-002 ~\]\# mkdir /tmp/zookeeper \[root@zk-002 ~\]\# cd /tmp/zookeeper/ \[root@zk-002 zookeeper\]\# echo 9 >myid \[root@zk-002 zookeeper\]\# ll total 4 \-rw-r--r-- 1 root root 2 May 13 15:51 myid \[root@zk-002 zookeeper\]\# cd /usr/local/zookeeper/conf/ \[root@zk-002 conf\]\# cp zoo\_sample.cfg zoo.cfg \[root@zk-002 conf\]\# vim zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 dataDir=/tmp/zookeeper clientPort=2181 server.8=zk-001:2888:3888 server.9=zk-002:2888:3888 server.10=zk-003:2888:3888 \[root@zk-003 ~\]\# mkdir /tmp/zookeeper \[root@zk-003 ~\]\# cd /tmp/zookeeper/ \[root@zk-003 zookeeper\]\# echo 10 >myid \[root@zk-003 zookeeper\]\# cd /usr/local/zookeeper/conf/ \[root@zk-003 conf\]\# cp zoo\_sample.cfg zoo.cfg \[root@zk-003 conf\]\# vim zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 dataDir=/tmp/zookeeper clientPort=2181 server.8=zk-001:2888:3888 server.9=zk-002:2888:3888 server.10=zk-003:2888:3888 **启动服务** \[root@zk-001 bin\]\# zkServer.sh start JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg Starting zookeeper ... STARTED \[root@zk-002 bin\]\# zkServer.sh start JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg Starting zookeeper ... STARTED \[root@zk-003 bin\]\# zkServer.sh start JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg Starting zookeeper ... STARTED **查看状态** \[root@zk-001 conf\]\# zkServer.sh status JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg Mode: follower \[root@zk-002 conf\]\# zkServer.sh status JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg Mode: follower \[root@zk-003 conf\]\# zkServer.sh status JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg Mode: leader 可以看出已经选举leader出来了,表明配置是正确的 **测试leader故障后切换情况** \[root@zk-003 conf\]\# jps 2694 Jps 2430 QuorumPeerMain \[root@zk-003 conf\]\# kill 2430 \[root@zk-003 conf\]\# jps 2724 Jps \[root@zk-002 conf\]\# zkServer.sh status JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg Mode: follower \[root@zk-001 conf\]\# zkServer.sh status JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg Mode: leader 可以看出zk-001这台服务器被选举成新的leader了 Zookeeper客户端四字符指令 > **conf 配置信息** > > **cons 连接信息** > > **dump 未处理会话节点** > > **envi 环境信息** > > **reqs 未处理请求 ** > > **stat 统计信息** > > **wchs 服务器watch的详细信息** > > **wchp 列出指定路径下服务器信息** \[root@zk-003 ~\]\# echo conf | nc 10.0.0.8 2181 clientPort=2181 dataDir=/tmp/zookeeper/version-2 dataLogDir=/tmp/zookeeper/version-2 tickTime=2000 maxClientCnxns=60 minSessionTimeout=4000 maxSessionTimeout=40000 serverId=8 initLimit=10 syncLimit=5 electionAlg=3 electionPort=3888 quorumPort=2888 peerType=0 \[root@zk-003 ~\]\# echo envi | nc 10.0.0.8 2181 Environment: zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT host.name=zk-001 java.version=1.8.0\_121 java.vendor=Oracle Corporation java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.121-0.b13.el6\_8.x86\_64/jre java.class.path=/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../build/lib/\*.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper/bin/../zookeeper-3.4.6.jar:/usr/local/zookeeper/bin/../src/java/lib/\*.jar:/usr/local/zookeeper/bin/../conf: java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib java.io.tmpdir=/tmp java.compiler=<NA> os.name=Linux os.arch=amd64 os.version=2.6.32-504.el6.x86\_64 user.name=root user.home=/root user.dir=/usr/local/zookeeper/conf \[root@zk-003 ~\]\# echo stat | nc 10.0.0.8 2181 Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT Clients: /10.0.0.10:41192\[0\](queued=0,recved=1,sent=0) Latency min/avg/max: 0/0/0 Received: 4 Sent: 3 Connections: 1 Outstanding: 0 Zxid: 0x200000000 Mode: leader Node count: 4 Zookeeper客户端命令 **通过zkCli.sh 连接服务器** zkCli.sh -server zk-001:2181 -server 10.0.0.9:2181 \[root@zk-001 ~\]\# zkCli.sh -server zk-002:2181 Connecting to zk-002:2181 WATCHER:: WatchedEvent state:SyncConnected type:None path:null \[zk: zk-002:2181(CONNECTED) 0\] h **\#\#帮助命令** ZooKeeper -server host:port cmd args stat path \[watch\] set path data \[version\] ls path \[watch\] delquota \[-n|-b\] path ls2 path \[watch\] setAcl path acl setquota -n|-b val path history redo cmdno printwatches on|off delete path \[version\] sync path listquota path rmr path get path \[watch\] create \[-s\] \[-e\] path data acl addauth scheme auth quit getAcl path close connect host:port \[zk: zk-002:2181(CONNECTED) 1\] ls / **\#列出目录下内容** \[zookeeper\] \[zk: zk-002:2181(CONNECTED) 2\] ls /zookeeper \[quota\] \[zk: zk-002:2181(CONNECTED) 3\] ls /zookeeper/quota \[\] \[zk: zk-002:2181(CONNECTED) 4\] create /root testfile **\#创建目录及数据** Created /root \[zk: zk-002:2181(CONNECTED)5\]get /root **\#获取数据** testfile cZxid = 0x200000002 ctime = Sat May 13 20:09:52 CST 2017 mZxid = 0x200000002 mtime = Sat May 13 20:09:52 CST 2017 pZxid = 0x200000002 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 8 numChildren = 0 \[zk: zk-002:2181(CONNECTED) 6\] set /root testfile-001 cZxid = 0x200000002 ctime = Sat May 13 20:09:52 CST 2017 mZxid = 0x200000003 mtime = Sat May 13 21:04:17 CST 2017 pZxid = 0x200000002 cversion = 0 dataVersion = 1 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 12 numChildren = 0 **需要注意的是ZK是不能一次创建多级节点的** \[zk: zk-002:2181(CONNECTED) 7\] create /root/s1/s1-1 \[zk: zk-002:2181(CONNECTED) 8\] ls /root **\#提示是空** \[\] \[zk: zk-002:2181(CONNECTED) 9\] create /root/s1 s1-data Created /root/s1 \[zk: zk-002:2181(CONNECTED) 10\] create /root/s2 s2-data Created /root/s2 \[zk: zk-002:2181(CONNECTED) 11\] create /root/s3 s3-data Created /root/s3 \[zk: zk-002:2181(CONNECTED) 13\] ls /root \[s3, s1, s2\] **查看状态** \[zk: zk-002:2181(CONNECTED) 14\] stat /root cZxid = 0x200000002 ctime = Sat May 13 20:09:52 CST 2017 mZxid = 0x200000003 mtime = Sat May 13 21:04:17 CST 2017 pZxid = 0x200000006 cversion = 3 dataVersion = 1 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 12 numChildren = 3 **删除目录** \[zk: zk-002:2181(CONNECTED) 15\] delete /root/s3 \[zk: zk-002:2181(CONNECTED) 16\] ls /root \[s1, s2\] **断开连接** \[zk: zk-002:2181(CONNECTED) 17\] close 2017-05-13 21:13:00,603 \[myid:\] - INFO\[main:ZooKeeper@684\] - Session: 0x95c0122efa50000 closed 2017-05-13 21:13:00,604 \[myid:\] - INFO\[main-EventThread:ClientCnxn$EventThread@512\] - EventThread shut down **重新连接** 由于ZK服务器间的数据是一致的,因此这次我连接其它服务器 \[zk: zk-002:2181(CLOSED) 18\] connect zk-001:2181 \[zk: zk-001:2181(CONNECTED) 19\] get /root testfile-001 cZxid = 0x200000002 ctime = Sat May 13 20:09:52 CST 2017 mZxid = 0x200000003 mtime = Sat May 13 21:04:17 CST 2017 pZxid = 0x200000007 cversion = 4 dataVersion = 1 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 12 numChildren = 2 \[zk: zk-001:2181(CONNECTED) 20\] get /root/s1 s1-data cZxid = 0x200000004 ctime = Sat May 13 21:07:42 CST 2017 mZxid = 0x200000004 mtime = Sat May 13 21:07:42 CST 2017 pZxid = 0x200000004 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 7 numChildren = 0
还没有评论,来说两句吧...