今天在用ansilbe命令 ansible noredis -m shell -a "netstat -nltp|grep redis"查看集群redis的啟動狀態發現:
其中有一台是啟動不了的,非常奇怪都是一樣的配置。
於是我ssh 10.25.115.125過去查看了配置,還有命令,發現一切正常! netstat -nltp|grep 6379也沒發現端口已經被占用,那問題出在哪呢?
於是,我在/etc/redis.conf配置了 logfile "/var/log/redis.log" (汗,發現配置log的重要性了,默認redis配置是不開log的):結果找到問題了
[16560] 14 Sep 17:32:37.724 # Server started, Redis version 2.8.21
[16560] 14 Sep 17:32:37.724 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
[16560] 14 Sep 17:32:37.724 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
[16560] 14 Sep 17:32:37.724 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
[16560] 14 Sep 17:32:37.724 # Short read or OOM loading DB. Unrecoverable error, aborting now.
原來問題出現內存不足。