- phpredisadmin工具
- rdbtools管理工具
- saltstack管理redis
- 通過codis完成redis管理
一：phpredisadmin工具：類似於mysqladmin管理mysql一樣
安裝環境：
[root@node5 ~]# yum install httpd php php-redis -y
下載管理包：
[root@node5 html]# git clone https://github.com/erikdubbelboer/phpRedisAdmin.git
更改名稱並解決依賴：
[root@node5 html]# mv phpRedisAdmin admin [root@node5 html]# cd admin [root@node5 admin]# git clone https://github.com/nrk/predis.git vendor #解決依賴包
啟動http,啟動前確認80端口沒有被占用，如果被占用可以修改監聽的默認端口為80以為的其他端口：
[root@node5 html]# /etc/init.d/httpd restart Stopping httpd: [FAILED] Starting httpd: [ OK ]
[root@node5 html]# chkconfig httpd on
訪問http://ServerIP/admin：
報錯，無法打開，查看apache錯誤日志如下：
[Mon Mar 21 21:24:47 2016] [error] [client 192.168.10.1] PHP Fatal error: require(): Failed opening required '/var/www/html/admin/includes/../vendor/autoload.php' (include_path='.:/usr/share/pear:/usr/share/php') in /var/www/html/admin/includes/common.inc.php on line 2
解決過程：
[root@node5 admin]# cd includes/ [root@node5 includes]# ls config.sample.inc.php config.inc.php [root@node5 includes]# vim config.inc.php $config = array( 'servers' => array( array( 'name' => 'local server', // Optional name. 'host' => '192.168.10.205', #配置為自己的主機IP 'port' => 6379, 'filter' => '*', // Optional Redis authentication. //'auth' => 'redispasswordhere' // Warning: The password is sent in plain-text to the Redis server. ),
保存退出后重新啟動httpd：
[root@node5 includes]# /etc/init.d/httpd restart
Stopping httpd: [ OK ]
Starting httpd: [ OK ]
再次測試還是不能訪問，報錯如下：
[Mon Mar 21 21:55:27 2016] [error] [client 192.168.10.1] PHP Fatal error: Call to undefined function mb_internal_encoding() in /var/www/html/admin/includes/common.inc.php on line 59
解決辦法：
[root@node5 admin]# vim /var/www/html/admin/includes/common.inc.php mb_internal_encoding('utf-8'); 將此行改為注釋，如下： #mb_internal_encoding('utf-8');
訪問：
配置管理多個reids Server：
[root@node5 includes]# pwd /var/www/html/admin/includes [root@node5 includes]# vim config.inc.php $config = array( 'servers' => array( array( 'name' => '182.168.10.205', // Optional name. 'host' => '192.168.10.205', 'port' => 6379, 'filter' => '*', // Optional Redis authentication. //'auth' => 'redispasswordhere' // Warning: The password is sent in plain-text to the Redis server. ), array( #一個主機一個配置，名稱自定義，IP和端口是Redis的IP和端口 'name' => '7001', // Optional name.name支持中文 'host' => '192.168.10.205', 'port' => 7001, ), array( 'name' => '7002', // Optional name. 'host' => '192.168.10.205', 'port' => 7002, ), array( 'name' => '7003', // Optional name. 'host' => '192.168.10.205', 'port' => 7003, ),
更改完成以后無需重啟apache，如果瀏覽器已經打開管理地址，刷新一下即可：
二：rdbtools管理工具：
使用pip命令安裝,如果沒有pip命令則先安裝pip：
[root@node5 ~]# yum install python-pip
使用pip安裝rdbtools： pip install rdbtools
[root@node5 ~]# pip install rdbtools /usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning You are using pip version 7.1.0, however version 8.1.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Collecting rdbtools /usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning Downloading rdbtools-0.1.6.tar.gz Installing collected packages: rdbtools Running setup.py install for rdbtools Successfully installed rdbtools-0.1.6
導出rdb中的key信息：
[root@node5 7001]# rdb -c memory dump_7001.rdb > memory.csv #然后將memory.csv文件下載到本地，即可用excel打開分析了，格式如下圖：
三：通過saltstack管理redis：
模塊名稱：redismod
官方文檔地址：https://docs.saltstack.com/en/latest/ref/modules/all/index.html
四：通過codis完成redis管理：
1.codis簡介：
Codis 是一個分布式 Redis 解決方案, 對於上層的應用來說, 連接到 Codis Proxy 和連接原生的 Redis Server 沒有明顯的區別 (不支持的命令列表), 上層應用可以像使用單機的 Redis 一樣使用, Codis 底層會處理請求的轉發, 不停機的數據遷移等工作, 所有后邊的一切事情, 對於前面的客戶端來說是透明的, 可以簡單的認為后邊連接的是一個內存無限大的 Redis 服務.其架構如下：
Codis 由四部分組成:
- Codis Proxy (codis-proxy) #負責對redis server 的代理，可以在前端使用haproxy做負載均衡，負責調度redis-client的請求。
- Codis Dashboard (codis-config) #codis的管理工具，可以添加刪除redis節點，發起數據遷移，還自帶一個http server，提供一個dashboard可以在瀏覽器進行觀察和配置。
- Codis Redis (codis-server) #基於redis 2.8做了一個分支，只能運行codis-server
- ZooKeeper/Etcd #Codis 依賴 ZooKeeper 來存放數據路由表和 codis-proxy 節點的元信息, codis-config 發起的命令都會通過 ZooKeeper 同步到各個存活的 codis-proxy.
codis-proxy 是客戶端連接的 Redis 代理服務, codis-proxy 本身實現了 Redis 協議, 表現得和一個原生的 Redis 沒什么區別 (就像 Twemproxy), 對於一個業務來說, 可以部署多個 codis-proxy, codis-proxy 本身是無狀態的.
codis-config 是 Codis 的管理工具, 支持包括, 添加/刪除 Redis 節點, 添加/刪除 Proxy 節點, 發起數據遷移等操作. codis-config 本身還自帶了一個 http server, 會啟動一個 dashboard, 用戶可以直接在瀏覽器上觀察 Codis 集群的運行狀態.
2.准備基礎環境：
安裝go語言並設置環境變量：
[root@node5 opt]# yum install golang -y #codis是基於go語言編寫的，因此要安裝go語言環境 [root@node5 opt]# mkdir /opt/gopath #保存go環境的路徑 [root@node5 opt]# vim /etc/profile export GOPATH=/opt/gopath
[root@node5 opt]# source /etc/profile
安裝配置zookeeper，需要有java環境:
[root@node5 ~]# wget http://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz [root@node5 ~]# mv zookeeper-3.4.6 /usr/local/zookeeper #移動到目錄 [root@node5 ~]# cd /usr/local/zookeeper/conf/ #進入到目錄 [root@node5 conf]# cp zoo_sample.cfg /opt/zoo.cfg #復制配置文件 [root@node5 conf]# cd /opt/ [root@node5 opt]# mkdir zk1 zk2 zk3 #准備zookeeper 服務ID，每個服務器的ID是不同的 [root@node5 opt]# echo 1 > zk1/myid [root@node5 opt]# echo 2 > zk2/myid [root@node5 opt]# echo 3 > zk3/myid [root@node5 opt]# vim zoo.cfg #編輯配置文件 dataDir=/opt/zk1 #保存數據的命令 clientPort=2181 #客戶端連接的端口 server.1 = 192.168.10.205:2887:3887 server.2 = 192.168.10.205:2888:3888 server.3 = 192.168.10.205:2889:3889 [root@node5 opt]# cp zoo.cfg zk1/zk1.cfg [root@node5 opt]# cp zoo.cfg zk2/zk2.cfg [root@node5 opt]# cp zoo.cfg zk3/zk3.cfg [root@node5 opt]# vim zk2/zk2.cfg #每個服務對應不同的目錄 dataDir=/opt/zk2
clientPort=2182 [root@node5 opt]# vim zk3/zk3.cfg dataDir=/opt/zk3
clientPort=2183
詳細配置:
[root@node5 opt]# grep '^[a-z]' zk3/zk3.cfg tickTime=2000 #服務器和客戶端的心跳維持間隔，間隔多久發送心跳 initLimit=10 #選舉的時候的時間間隔是10次，10次 * 2000微秒 syncLimit=5 # Leader 與 Follower 之間發送消息，請求和應答時間長度，最長不能超過多少個 tickTime 的時間長度 dataDir=/opt/zk3 #數據保存目錄 clientPort=2181 #客戶端連接的端口 server.1 = 192.168.10.205:2887:3887 #集群端口和ID配置 server.2 = 192.168.10.205:2888:3888 server.3 = 192.168.10.205:2889:3889 詳細解釋： tickTime：這個時間是作為 Zookeeper 服務器之間或客戶端與服務器之間維持心跳的時間間隔，也就是每個 tickTime 時間就會發送一個心跳。 dataDir：顧名思義就是 Zookeeper 保存數據的目錄，默認情況下，Zookeeper 將寫數據的日志文件也保存在這個目錄里。 clientPort：這個端口就是客戶端連接 Zookeeper 服務器的端口，Zookeeper 會監聽這個端口，接受客戶端的訪問請求。 initLimit：這個配置項是用來配置 Zookeeper 接受客戶端（這里所說的客戶端不是用戶連接 Zookeeper 服務器的客戶端，而是 Zookeeper 服務器集群中連接到 Leader 的 Follower 服務器）初始化連接時最長能忍受多少個心跳時間間隔數。當已經超過 5個心跳的時間（也就是 tickTime）長度后 Zookeeper 服務器還沒有收到客戶端的返回信息，那么表明這個客戶端連接失敗。總的時間長度就是 5*2000=10 秒 syncLimit：這個配置項標識 Leader 與 Follower 之間發送消息，請求和應答時間長度，最長不能超過多少個 tickTime 的時間長度，總的時間長度就是 2*2000=4 秒 server.A=B：C：D：其中 A 是一個數字，表示這個是第幾號服務器；B 是這個服務器的 ip 地址；C 表示的是這個服務器與集群中的 Leader 服務器交換信息的端口；D 表示的是萬一集群中的 Leader 服務器掛了，需要一個端口來重新進行選舉，選出一個新的 Leader，而這個端口就是用來執行選舉時服務器相互通信的端口。如果是偽集群的配置方式，由於 B 都是一樣，所以不同的 Zookeeper 實例通信端口號不能一樣，所以要給它們分配不同的端口號。
啟動每個zookeeper 服務：
[root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh start /opt/zk1/zk1.cfg JMX enabled by default Using config: /opt/zk1/zk1.cfg Starting zookeeper ... STARTED [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh start /opt/zk2/zk2.cfg JMX enabled by default Using config: /opt/zk2/zk2.cfg Starting zookeeper ... STARTED [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh start /opt/zk3/zk3.cfg JMX enabled by default Using config: /opt/zk3/zk3.cfg Starting zookeeper ... STARTED
確認端口啟動成功：
[root@node5 opt]# ss -tnl | grep 2181 LISTEN 0 50 :::2181 :::* [root@node5 opt]# ss -tnl | grep 2182 LISTEN 0 50 :::2182 :::* [root@node5 opt]# ss -tnl | grep 2183 LISTEN 0 50 :::2183 :::* [root@node5 opt]#
查看每個節點的狀態：
[root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh status /opt/zk1/zk1.cfg JMX enabled by default Using config: /opt/zk1/zk1.cfg Mode: follower [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh status /opt/zk2/zk2.cfg JMX enabled by default Using config: /opt/zk2/zk2.cfg Mode: leader #這是主的，另外兩個是從的，表示已經啟動完成 [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh status /opt/zk3/zk3.cfg JMX enabled by default Using config: /opt/zk3/zk3.cfg Mode: follower
連接到zookeeper節點：
[root@node5 opt]# /usr/local/zookeeper/bin/zkCli.sh -server 192.168.10.205:2181
[root@node5 opt]# /usr/local/zookeeper/bin/zkCli.sh -server 192.168.10.205:2181Connecting to 192.168.10.205:2181 2016-03-22 01:27:48,325 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2016-03-22 01:27:48,331 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node5.a.com 2016-03-22 01:27:48,332 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.7.0_45 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64/jre 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../build/lib/*.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper/bin/../zookeeper-3.4.6.jar:/usr/local/zookeeper/bin/../src/java/lib/*.jar:/usr/local/zookeeper/bin/../conf: 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:os.version=2.6.32-431.el6.x86_64 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2016-03-22 01:27:48,336 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/opt 2016-03-22 01:27:48,337 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=192.168.10.205:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@6ade299f Welcome to ZooKeeper! 2016-03-22 01:27:48,393 [myid:] - INFO [main-SendThread(192.168.10.205:2181):ClientCnxn$SendThread@975] - Opening socket connection to server 192.168.10.205/192.168.10.205:2181. Will not attempt to authenticate using SASL (unknown error) 2016-03-22 01:27:48,424 [myid:] - INFO [main-SendThread(192.168.10.205:2181):ClientCnxn$SendThread@852] - Socket connection established to 192.168.10.205/192.168.10.205:2181, initiating session JLine support is enabled 2016-03-22 01:27:48,621 [myid:] - INFO [main-SendThread(192.168.10.205:2181):ClientCnxn$SendThread@1235] - Session establishment complete on server 192.168.10.205/192.168.10.205:2181, sessionid = 0x1539a349fc40000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null
進入到目錄執行編譯：
[root@node5 codis]# pwd /opt/gopath/src/github.com/CodisLabs/codis [root@node5 codis]# ls cmd docker genver.sh MIT-LICENSE.txt test config.ini Dockerfile Godeps pkg vitess_license doc extern Makefile README.md wandoujia_licese.txt [root@node5 codis]# make
[root@node5 codis]# make GO15VENDOREXPERIMENT=0 GOPATH=`godep path` godep restore godep: [WARNING]: godep should only be used inside a valid go package directory and godep: [WARNING]: may not function correctly. You are probably outside of your $GOPATH. godep: [WARNING]: Current Directory: /opt/gopath/src/github.com/CodisLabs/codis godep: [WARNING]: $GOPATH: /opt/gopath/src/github.com/CodisLabs/codis/Godeps/_workspace GOPATH=`godep path`:$GOPATH go build -o bin/codis-proxy ./cmd/proxy GOPATH=`godep path`:$GOPATH go build -o bin/codis-config ./cmd/cconfig make -j4 -C extern/redis-2.8.21/ make[1]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21' cd src && make all make[2]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/src' rm -rf redis-server redis-sentinel redis-cli redis-benchmark redis-check-dump redis-check-aof *.o *.gcda *.gcno *.gcov redis.info lcov-html (cd ../deps && make distclean) make[3]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' (cd hiredis && make clean) > /dev/null || true (cd linenoise && make clean) > /dev/null || true (cd lua && make clean) > /dev/null || true (cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true (rm -f .make-*) make[3]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' (rm -f .make-*) echo STD=-std=c99 -pedantic >> .make-settings echo WARN=-Wall -W >> .make-settings echo OPT=-O2 >> .make-settings echo MALLOC=jemalloc >> .make-settings echo CFLAGS= >> .make-settings echo LDFLAGS= >> .make-settings echo REDIS_CFLAGS= >> .make-settings echo REDIS_LDFLAGS= >> .make-settings echo PREV_FINAL_CFLAGS=-std=c99 -pedantic -Wall -W -O2 -g -ggdb -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src -DUSE_JEMALLOC -I../deps/jemalloc/include >> .make-settings echo PREV_FINAL_LDFLAGS= -g -ggdb -rdynamic >> .make-settings (cd ../deps && make hiredis linenoise lua jemalloc) make[3]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' (cd hiredis && make clean) > /dev/null || true (cd linenoise && make clean) > /dev/null || true (cd lua && make clean) > /dev/null || true (cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true (rm -f .make-*) (echo "" > .make-ldflags) (echo "" > .make-cflags) MAKE hiredis cd hiredis && make static MAKE linenoise cd linenoise && make MAKE lua make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/linenoise' cc -Wall -Os -g -c linenoise.c cd lua/src && make all CFLAGS="-O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL " MYLDFLAGS="" AR="ar rcu" MAKE jemalloc cd jemalloc && ./configure --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops " LDFLAGS="" make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/lua/src' cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lapi.o lapi.c make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/hiredis' cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb net.c checking for xsltproc... /usr/bin/xsltproc checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb hiredis.c checking whether we are cross compiling... no checking for suffix of object files... make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/linenoise' cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb sds.c o checking whether we are using the GNU C compiler... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lcode.o lcode.c yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb async.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldebug.o ldebug.c yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldo.o ldo.c yes ldo.c: In function ‘f_parser’: ldo.c:496: warning: unused variable ‘c’ cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldump.o ldump.c checking for inttypes.h... yes checking for stdint.h... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lfunc.o lfunc.c yes checking for unistd.h... ar rcs libhiredis.a net.o hiredis.o sds.o async.o cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lgc.o lgc.c yes checking whether byte ordering is bigendian... make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/hiredis' cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o llex.o llex.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lmem.o lmem.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lobject.o lobject.c no checking size of void *... 8 checking size of int... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lopcodes.o lopcodes.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lparser.o lparser.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstate.o lstate.c 4 checking size of long... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstring.o lstring.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltable.o ltable.c 8 checking size of intmax_t... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltm.o ltm.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lundump.o lundump.c 8 checking build system type... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lvm.o lvm.c x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking whether pause instruction is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lzio.o lzio.c yes checking whether SSE2 intrinsics is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o strbuf.o strbuf.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o fpconv.o fpconv.c yes checking for ar... ar checking whether __attribute__ syntax is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lauxlib.o lauxlib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lbaselib.o lbaselib.c yes checking whether compiler supports -fvisibility=hidden... yes checking whether compiler supports -Werror... yes checking whether tls_model attribute is compilable... yes checking for a BSD-compatible install... /usr/bin/install -c checking for ranlib... ranlib checking for ld... /usr/bin/ld checking for autoconf... /usr/bin/autoconf checking for memalign... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldblib.o ldblib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o liolib.o liolib.c yes checking for valloc... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lmathlib.o lmathlib.c yes checking configured backtracing method... N/A checking for sbrk... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o loslib.o loslib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltablib.o ltablib.c yes checking whether utrace(2) is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstrlib.o lstrlib.c no checking whether valgrind is compilable... no checking STATIC_PAGE_SHIFT... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o loadlib.o loadlib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o linit.o linit.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_cjson.o lua_cjson.c 12 checking pthread.h usability... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_struct.o lua_struct.c yes checking pthread.h presence... yes checking for pthread.h... yes checking for pthread_create in -lpthread... yes checking for _malloc_thread_cleanup... no checking for _pthread_mutex_init_calloc_cb... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_cmsgpack.o lua_cmsgpack.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_bit.o lua_bit.c no checking for TLS... yes checking whether a program using ffsl is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua.o lua.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o luac.o luac.c yes checking whether atomic(9) is compilable... no checking whether Darwin OSAtomic*() is compilable... no checking whether to force 32-bit __sync_{add,sub}_and_fetch()... no checking whether to force 64-bit __sync_{add,sub}_and_fetch()... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o print.o print.c no checking whether Darwin OSSpin*() is compilable... ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o strbuf.o fpconv.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o lua_cjson.o lua_struct.o lua_cmsgpack.o lua_bit.o # DLL needs all object files ranlib liblua.a no checking for stdbool.h that conforms to C99... cc -o lua lua.o liblua.a -lm yes checking for _Bool... liblua.a(loslib.o): In function `os_tmpname': loslib.c:(.text+0x35): warning: the use of `tmpnam' is dangerous, better use `mkstemp' cc -o luac luac.o print.o liblua.a -lm make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/lua/src' yes configure: creating ./config.status config.status: creating Makefile config.status: creating doc/html.xsl config.status: creating doc/manpages.xsl config.status: creating doc/jemalloc.xml config.status: creating include/jemalloc/jemalloc_macros.h config.status: creating include/jemalloc/jemalloc_protos.h config.status: creating include/jemalloc/internal/jemalloc_internal.h config.status: creating test/test.sh config.status: creating test/include/test/jemalloc_test.h config.status: creating config.stamp config.status: creating bin/jemalloc.sh config.status: creating include/jemalloc/jemalloc_defs.h config.status: creating include/jemalloc/internal/jemalloc_internal_defs.h config.status: creating test/include/test/jemalloc_test_defs.h config.status: executing include/jemalloc/internal/private_namespace.h commands config.status: executing include/jemalloc/internal/private_unnamespace.h commands config.status: executing include/jemalloc/internal/public_symbols.txt commands config.status: executing include/jemalloc/internal/public_namespace.h commands config.status: executing include/jemalloc/internal/public_unnamespace.h commands config.status: executing include/jemalloc/internal/size_classes.h commands config.status: executing include/jemalloc/jemalloc_protos_jet.h commands config.status: executing include/jemalloc/jemalloc_rename.h commands config.status: executing include/jemalloc/jemalloc_mangle.h commands config.status: executing include/jemalloc/jemalloc_mangle_jet.h commands config.status: executing include/jemalloc/jemalloc.h commands =============================================================================== jemalloc version : 3.6.0-0-g46c0af68bd248b04df75e4f92d5fb804c3d75340 library revision : 1 CC : gcc CPPFLAGS : -D_GNU_SOURCE -D_REENTRANT CFLAGS : -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -fvisibility=hidden LDFLAGS : EXTRA_LDFLAGS : LIBS : -lpthread RPATH_EXTRA : XSLTPROC : /usr/bin/xsltproc XSLROOT : /usr/share/sgml/docbook/xsl-stylesheets PREFIX : /usr/local BINDIR : /usr/local/bin INCLUDEDIR : /usr/local/include LIBDIR : /usr/local/lib DATADIR : /usr/local/share MANDIR : /usr/local/share/man srcroot : abs_srcroot : /opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc/ objroot : abs_objroot : /opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc/ JEMALLOC_PREFIX : je_ JEMALLOC_PRIVATE_NAMESPACE : je_ install_suffix : autogen : 0 experimental : 1 cc-silence : 1 debug : 0 code-coverage : 0 stats : 1 prof : 0 prof-libunwind : 0 prof-libgcc : 0 prof-gcc : 0 tcache : 1 fill : 1 utrace : 0 valgrind : 0 xmalloc : 0 mremap : 0 munmap : 0 dss : 0 lazy_lock : 0 tls : 1 =============================================================================== cd jemalloc && make CFLAGS="-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops " LDFLAGS="" lib/libjemalloc.a make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc' gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/jemalloc.o src/jemalloc.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/arena.o src/arena.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/atomic.o src/atomic.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/base.o src/base.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/bitmap.o src/bitmap.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk.o src/chunk.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk_dss.o src/chunk_dss.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk_mmap.o src/chunk_mmap.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/ckh.o src/ckh.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/ctl.o src/ctl.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/extent.o src/extent.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/hash.o src/hash.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/huge.o src/huge.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/mb.o src/mb.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/mutex.o src/mutex.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/prof.o src/prof.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/quarantine.o src/quarantine.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/rtree.o src/rtree.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/stats.o src/stats.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/tcache.o src/tcache.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/util.o src/util.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/tsd.o src/tsd.c ar crus lib/libjemalloc.a src/jemalloc.o src/arena.o src/atomic.o src/base.o src/bitmap.o src/chunk.o src/chunk_dss.o src/chunk_mmap.o src/ckh.o src/ctl.o src/extent.o src/hash.o src/huge.o src/mb.o src/mutex.o src/prof.o src/quarantine.o src/rtree.o src/stats.o src/tcache.o src/util.o src/tsd.o make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc' make[3]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' CC adlist.o CC ae.o CC anet.o CC dict.o anet.c: In function ‘anetSockName’: anet.c:565: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:563: note: initialized from here anet.c:569: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:567: note: initialized from here anet.c: In function ‘anetPeerToString’: anet.c:543: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:541: note: initialized from here anet.c:547: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:545: note: initialized from here anet.c: In function ‘anetTcpAccept’: anet.c:511: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:509: note: initialized from here anet.c:515: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:513: note: initialized from here CC redis.o CC sds.o CC zmalloc.o CC lzf_c.o CC lzf_d.o CC pqsort.o CC zipmap.o CC sha1.o CC ziplist.o CC release.o CC networking.o CC util.o CC object.o CC db.o CC replication.o CC rdb.o CC t_string.o db.c: In function ‘scanGenericCommand’: db.c:454: warning: ‘pat’ may be used uninitialized in this function db.c:455: warning: ‘patlen’ may be used uninitialized in this function CC t_list.o CC t_set.o CC t_zset.o CC t_hash.o CC config.o CC aof.o CC pubsub.o CC multi.o CC debug.o CC sort.o CC intset.o CC syncio.o CC migrate.o CC endianconv.o CC slowlog.o CC scripting.o CC bio.o CC rio.o CC rand.o CC memtest.o CC crc64.o CC crc32.o CC bitops.o CC sentinel.o CC notify.o CC setproctitle.o CC hyperloglog.o CC latency.o CC sparkline.o CC slots.o CC redis-cli.o CC redis-benchmark.o CC redis-check-dump.o CC redis-check-aof.o LINK redis-benchmark LINK redis-check-dump LINK redis-check-aof LINK redis-server INSTALL redis-sentinel LINK redis-cli Hint: It's a good idea to run 'make test' ;)
[root@node5 codis]# make gotest GOPATH=`godep path`:$GOPATH go test ./pkg/... ./cmd/... ok github.com/CodisLabs/codis/pkg/models 17.324s ok github.com/CodisLabs/codis/pkg/proxy 6.296s ok github.com/CodisLabs/codis/pkg/proxy/redis 1.338s ok github.com/CodisLabs/codis/pkg/proxy/router 0.654s ? github.com/CodisLabs/codis/pkg/utils [no test files] ? github.com/CodisLabs/codis/pkg/utils/assert [no test files] ? github.com/CodisLabs/codis/pkg/utils/atomic2 [no test files] ok github.com/CodisLabs/codis/pkg/utils/bytesize 0.002s ? github.com/CodisLabs/codis/pkg/utils/errors [no test files] ? github.com/CodisLabs/codis/pkg/utils/log [no test files] ? github.com/CodisLabs/codis/pkg/utils/trace [no test files] ? github.com/CodisLabs/codis/cmd/cconfig [no test files] ? github.com/CodisLabs/codis/cmd/proxy [no test files]
執行全部指令后，會在 bin 文件夾內生成 codis-config、codis-proxy、codis-server三個可執行文件。另外, bin/assets 文件夾是 codis-config 的 dashboard http 服務需要的前端資源, 需要和 codis-config 放置在同一文件夾下)
[root@node5 codis]# cd bin/ [root@node5 bin]# ls assets codis-config codis-proxy codis-server
可執行文件使用方式：
[root@node5 bin]# ./codis-config -h
[root@node5 bin]# ./codis-config -h
usage: codis-config [-c <config_file>] [-L <log_file>] [--log-level=<loglevel>] <command> [<args>...] options: -c #配置文件地址 -L #日志輸出文件地址 --log-level=<loglevel> #輸出日志級別 (debug < info (default) < warn < error < fatal) commands: server #redis 服務器組管理 slot #slot 管理 dashboard #啟動 dashboard 服務 action #事件管理 (目前只有刪除歷史事件的日志) proxy #proxy 管理
[root@node5 bin]# ./codis-proxy -h
$ bin/codis-proxy -h usage: codis-proxy [-c <config_file>] [-L <log_file>] [--log-level=<loglevel>] [--cpu=<cpu_num>] [--addr=<proxy_listen_addr>] [--http-addr=<debug_http_server_addr>] options: -c #配置文件地址 -L #日志輸出文件地址 --log-level=<loglevel> #輸出日志級別 (debug < info (default) < warn < error < fatal) --cpu=<cpu_num> #proxy占用的 cpu 核數, 默認1, 最好設置為機器的物理cpu數的一半到2/3左右 --addr=<proxy_listen_addr> #proxy 的 redis server 監聽的地址, 格式 <ip or hostname>:<port>, 如: localhost:9000, :9001 --http-addr=<debug_http_server_addr> #proxy 的調試信息啟動的http server, 可以訪問 http://debug_http_server_addr/debug/vars
[root@node5 bin]# ./codis-server -h
[root@node5 bin]# ./codis-server -h Usage: ./redis-server [/path/to/redis.conf] [options] ./redis-server - (read config from stdin) #指定redis配置文件 ./redis-server -v or --version #顯示版本 ./redis-server -h or --help #獲取幫助 ./redis-server --test-memory <megabytes> #單位為字節 Examples: ./redis-server (run the server with default conf) ./redis-server /etc/redis/6379.conf ./redis-server --port 7777 ./redis-server --port 7777 --slaveof 127.0.0.1 8888 ./redis-server /etc/myredis.conf --loglevel verbose
開始部署：
[root@node5 bin]# cd ..
[root@node5 codis]# vim config.ini
zk=192.168.10.205:2181
dashboard_addr=192.168.10.205:18087
配置並啟動 dashboard：
[root@node5 codis]# cp config.ini bin/ #將修改后的文件復制至bin目錄，因為bin里面你的命令默認讀取當前目錄的config.ini文件，或可以使用 -c指定也可以
[root@node5 bin]# ./codis-config dashboard & #該命令會啟動 dashboard
啟動成功就可以通過 http://ServerIP:18087訪問
初始化 slots：
執行./codis-config slot init，該命令會在zookeeper上創建slot相關信息，如果提示已經初始化可以加 -f 強制
[root@node5 bin]# ./codis-config slot init -f { "msg": "OK", "ret": 0 }
啟動 Codis Redis：
如果已經啟動可以跳過，如果還未啟動就執行：
[root@node5 bin]# /etc/init.d/redisd start
添加 Redis Server Group：
每一個 Server Group 作為一個 Redis 服務器組存在, 只允許有一個 master, 可以有多個 slave, group id 僅支持大於等於1的整數
添加一個master：
[root@node5 bin]# ./codis-config server add 1 localhost:6379 master { "msg": "OK", "ret": 0 }
為master添加一個slave：
[root@node5 bin]# ./codis-config server add 1 192.168.10.205:7009 slave
[root@node5 bin]# ./codis-config server add 1 192.168.10.205:7009 slave 2016/03/22 03:37:27 main.go:154: [PANIC] run sub-command failed [error]: http status code 500, ERR SLAVEOF not allowed in cluster mode. 4 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/utils.go:66 main.callApi 3 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/server_group.go:94 main.runAddServerToGroup 2 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/server_group.go:54 main.cmdServer 1 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/main.go:87 main.runCommand 0 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/main.go:152 main.main ... ... [stack]: 0 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/main.go:154 main.main ... ...
查看列表信息：
[root@node5 bin]# ./codis-config server list [ { "id": 1, "product_name": "test", "servers": [ { "addr": "localhost:6379", "group_id": 1, "type": "master" }, { "addr": "192.168.10.205:7009", "group_id": 1, "type": "slave" } ] } ]
在瀏覽器驗證：
設置 server group 服務的 slot 范圍：
Codis 采用 Pre-sharding 的技術來實現數據的分片, 默認分成 1024 個 slots (0-1023), 對於每個key來說, 通過以下公式確定所屬的 Slot Id : SlotId = crc32(key) % 1024 每一個 slot 都會有一個且必須有一個特定的 server group id 來表示這個 slot 的數據由哪個 server group 來提供：
如:
設置編號為[0, 511]的 slot 由 server group 1 提供服務, 編號 [512, 1023] 的 slot 由 server group 2 提供服務
[root@node5 bin]# ./codis-config slot range-set 0 511 1 online { "msg": "OK", "ret": 0 } [root@node5 bin]# ./codis-config slot range-set 512 1023 2 online { "msg": "OK", "ret": 0 }
啟動 codis-proxy：
[root@node5 bin]# ./codis-proxy -c config.ini -L ./log/proxy.log --cpu=8 --addr=0.0.0.0:19000 --http-addr=0.0.0.0:11000 _____ ____ ____/ / (_) _____ / ___/ / __ \ / __ / / / / ___/ / /__ / /_/ / / /_/ / / / (__ ) \___/ \____/ \__,_/ /_/ /____/
剛啟動的 codis-proxy 默認是處於 offline狀態的, 然后設置 proxy 為 online 狀態, 只有處於 online 狀態的 proxy 才會對外提供服務
[root@node5 bin]# ./codis-config -c config.ini proxy online proxy_1 { "msg": "OK", "ret": 0 }
注：命令格式為./codis-config -c config.ini proxy online <proxy_name> <---- proxy的id, 如 proxy_1，可以在瀏覽器看到