当前位置：首页 > news >正文

zookeeper高级特性

news 来源：原创 2024/4/28 23:45:49

Multitenancy多租户

一般我们都是通过如下方式连接zk

./zkCli.sh -server 172.26.1.70:2181,172.26.1.71:2181,172.26.1.60:2181

我们也可以在连接参数后面追加一个路径。这样的用法可以用来隔离应用

./zkCli.sh -server 172.26.1.70:2181,172.26.1.71:2181,172.26.1.60:2181/twx

Quota配额

先创建一个路径create /app ""
给路径限制不能超过2个子节点setquota -n 2 /app
查看配额listquota /app

[zk: 172.26.1.70:2181,172.26.1.71:2181,172.26.1.60:2181(CONNECTED) 14] setquota -n 2 /app
[zk: 172.26.1.70:2181,172.26.1.71:2181,172.26.1.60:2181(CONNECTED) 15] listquota /app
absolute path is /zookeeper/quota/app/zookeeper_limits
Output quota for /app count=2,bytes=-1=;byteHardLimit=-1;countHardLimit=-1
Output stat for /app count=1,bytes=0

创建子节点create /app/c1 ""

[zk: 172.26.1.70:2181,172.26.1.71:2181,172.26.1.60:2181(CONNECTED) 16] create /app/c1 ""
Created /app/c1
[zk: 172.26.1.70:2181,172.26.1.71:2181,172.26.1.60:2181(CONNECTED) 17] listquota /app
absolute path is /zookeeper/quota/app/zookeeper_limits
Output quota for /app count=2,bytes=-1=;byteHardLimit=-1;countHardLimit=-1
Output stat for /app count=2,bytes=0

Output statcount=2 (算上/app本身)

再创建子节点create /app/c2 ""仍然会成功，但是zk日志里会有warn提示。（经过实验，没发现这行报警。可能是我的日志级别配置有问题）
delquota /app删除配额

数据备份

zk所有数据都存储在dataDir所指向的那个目录。

该目录下有以下文件：

acceptedEpoch 接收的epoch
currentEpoch 当前节点的epoch
log.**** 事务日志
Snapshot***: 快照日志（对应树结构数据）

事务日志

“First, they are quite large (over 6 MB each), considering the tests were small.
Second, they have a large number as the filename suffix.”(文件都很大，虽然只有简单的几条数据)

“ZooKeeper preallocates files in rather large chunks to avoid the metadata management overhead of growing the file with each write.（预分配空间，优化写入速度）
If you do a hex dump of one of these files, you will see that it is full of bytes of the null character (\0)”

“The log files contain transactions tagged with zxids, but to ease recovery and allow for quick lookup, each log file’s suffix is the first zxid of the log file in hexadecimal.(事务日志的第一个事务id)
One nice thing about representing the zxid in hex is that you can easily distinguish the epoch part of the zxid from the counter.(可以很便捷的区分epoch)
”

查看事务日志的命令：./zkTxnLogToolkit.sh ../data1/version-2/log.300000001

22-4-25 上午10时38分16秒 session 0x3003568335e0000 cxid 0x0 zxid 0x300000001 createSession 30000
22-4-25 上午10时38分22秒 session 0x3003568335e0001 cxid 0x0 zxid 0x300000002 createSession 30000
22-4-25 上午10时38分45秒 session 0x1003562ca3b0001 cxid 0x0 zxid 0x300000003 closeSession v{}
22-4-25 上午10时38分47秒 session 0x3003568335e0000 cxid 0x0 zxid 0x300000004 closeSession v{}
22-4-25 上午10时44分00秒 session 0x3003568335e0001 cxid 0x1 zxid 0x300000005 create /master,master1.example.com:2223,[31,s{'world,'anyone}
],true,1
22-4-25 上午10时45分17秒 session 0x10035678a310000 cxid 0x0 zxid 0x300000006 createSession 30000
22-4-25 上午10时45分29秒 session 0x10035678a310000 cxid 0x1 zxid 0x300000007 error -110
22-4-25 上午10时47分52秒 session 0x10035678a310001 cxid 0x0 zxid 0x300000008 createSession 30000
22-4-25 上午10时47分53秒 session 0x3003568335e0001 cxid 0x0 zxid 0x300000009 closeSession v{'/master}

事务日志中很多空间都是预分配为空的，见下图：

事务日志只会记录 create、delet、set等修改数据的操作。read操作不会记录

快照记录

查看命令：./zkSnapShotToolkit.sh ../data2/version-2/snapshot.200000006

ZNode Details (count=5):
----
/
  cZxid = 0x00000000000000
  ctime = Thu Jan 01 08:00:00 CST 1970
  mZxid = 0x00000000000000
  mtime = Thu Jan 01 08:00:00 CST 1970
  pZxid = 0x00000000000000
  cversion = 0
  dataVersion = 0
  aclVersion = 0
  ephemeralOwner = 0x00000000000000
  dataLength = 0
----
/zookeeper
  cZxid = 0x00000000000000
  ctime = Thu Jan 01 08:00:00 CST 1970
  mZxid = 0x00000000000000
  mtime = Thu Jan 01 08:00:00 CST 1970
  pZxid = 0x00000000000000
  cversion = 0
  dataVersion = 0
  aclVersion = 0
  ephemeralOwner = 0x00000000000000
  dataLength = 0
----
/zookeeper/config
  cZxid = 0x00000000000000
  ctime = Thu Jan 01 08:00:00 CST 1970
  mZxid = 0x00000000000000
  mtime = Mon Apr 25 10:27:51 CST 2022
  pZxid = 0x00000000000000
  cversion = 0
  dataVersion = -1
  aclVersion = -1
  ephemeralOwner = 0x00000000000000
  dataLength = 132
----

快照日志反应的是当前znode-tree中的数据！

“Snapshot files are not preallocated, so the size more accurately reflects the amount of data they contain.

快照日志没有预分配，所以它的大小几乎和实际数据大小是一致的

The suffix used reflects the current zxid when the snapshot started.”

后缀由来：快照创建那个时刻的zxid

“As we discussed earlier, the snapshot file is actually a fuzzy snapshot; it is not a valid snapshot on its own until the transaction log is replayed over it.

快照文件实际上是一个模糊快照。也就是说，它并不能完全反映出集群实际的数据集（有可能有些数据还未从事务日志中同步过来）

Specifically, to restore a system, you must start replaying the transaction log starting at the zxid of the snapshot suffix or earlier.”

所以，在恢复系统的时候，需要从快照文件后缀的那个zxid开始依次执行事务日志。比如上面的快照文件后缀是200000006，那么在读取事务日志时就要从zxid=200000006开始。

综上所述，如果需要备份或恢复数据，把data目录下的所有文件拷贝至新服务器即可。

“You can do this by copying log files and snapshot files to another machine (like your laptop, for example), putting them in the empty data directory of a standalone server.

“There are a couple of things to keep in mind if you choose to do this.

First, ZooKeeper is a replicated service, so there is redundancy built into the system. If you do take a backup, you need to back up only the data of one of the servers.”如果是集群，只需备份其中一份即可。

欢迎访问个人博客：https://tangwx.site/archives/zk%E9%AB%98%E7%BA%A7%E7%89%B9%E6%80%A7md