2

I have a node Redis cluser with 1 master and 2 slaves. Sentinel is running on all of them.

master 172.29.245.6
slave1 172.29.240.163
slave2 172.29.225.104

With my master up, this is the status of my cluster.

Version --> redis-3.2.10-2.el7.x86_64

Master

# Replication
role:master
connected_slaves:2
slave0:ip=172.29.225.104,port=6379,state=online,offset=486,lag=1
slave1:ip=172.29.240.163,port=6379,state=online,offset=633,lag=0
master_repl_offset:925
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:924

Slave1

# Replication
role:slave
master_host:172.29.245.6
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:17719
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

Slave2

# Replication
role:slave
master_host:172.29.245.6
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:20367
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

My aim is that when I shut down the master redis process one of the slaves should become a master.

But when I try to do an automatic failover, it gives the following messages.

Oct 27 14:18:22 redis-2 redis-sentinel[4866]: 4866:X 27 Oct 14:18:22.175 # +new-epoch 27
Oct 27 14:18:22 redis-2 redis-sentinel[4866]: 4866:X 27 Oct 14:18:22.175 # +try-failover master redis-master 172.29.245.6 6379
Oct 27 14:18:22 redis-2 redis-sentinel[4866]: 4866:X 27 Oct 14:18:22.185 # +vote-for-leader 61c51a37f2db1673ddcf5dc3fe6816c9ba83a408 27
Oct 27 14:18:33 redis-2 redis-sentinel[4866]: 4866:X 27 Oct 14:18:33.142 # -failover-abort-not-elected master redis-master 172.29.245.6 6379
Oct 27 14:18:33 redis-2 redis-sentinel[4866]: 4866:X 27 Oct 14:18:33.218 # Next failover delay: I will not start a failover before Fri Oct 27 14:20:23 2017

However when I use the method to manually failover, then it works.

[root@redis-1 ~]# redis-cli -p 26379 SENTINEL failover redis-master
OK

Oct 27 14:19:40 redis-3 redis-sentinel[1211]: 1211:X 27 Oct 14:19:40.503 # -sdown master redis-master 172.29.245.6 6379
Oct 27 14:19:40 redis-3 redis-sentinel[1211]: 1211:X 27 Oct 14:19:40.503 # -odown master redis-master 172.29.245.6 6379
Oct 27 14:24:56 redis-3 redis-sentinel[1211]: 1211:X 27 Oct 14:24:56.982 # +new-epoch 28
Oct 27 14:24:57 redis-3 redis-sentinel[1211]: 1211:X 27 Oct 14:24:57.851 # +config-update-from sentinel 5080eb7a0e27f9fb6ea6a197bf1abd334608943f 172.29.245.6 26379 @ redis-master 172.29.245.6 6379
Oct 27 14:24:57 redis-3 redis-sentinel[1211]: 1211:X 27 Oct 14:24:57.852 # +switch-master redis-master 172.29.245.6 6379 172.29.225.104 6379
Oct 27 14:24:57 redis-3 redis-sentinel[1211]: 1211:X 27 Oct 14:24:57.852 * +slave slave 172.29.240.163:6379 172.29.240.163 6379 @ redis-master 172.29.225.104 6379
Oct 27 14:24:57 redis-3 redis-sentinel[1211]: 1211:X 27 Oct 14:24:57.852 * +slave slave 172.29.245.6:6379 172.29.245.6 6379 @ redis-master 172.29.225.104 6379

I have gone through the documentation and followed it exactly but the automatic failover for me does not work. Any ideas ?

My sentinel.conf on all the nodes look like:

MAster

port 26379
sentinel myid 5080eb7a0e27f9fb6ea6a197bf1abd334608943f
sentinel monitor redis-master 172.29.245.6 6379 1
sentinel down-after-milliseconds redis-master 5000
sentinel failover-timeout redis-master 60000
sentinel auth-pass redis-master XXXXXXXXXXXXXXXXX
# Generated by CONFIG REWRITE
dir "/"
sentinel config-epoch redis-master 29
sentinel leader-epoch redis-master 29
sentinel known-slave redis-master 172.29.225.104 6379
sentinel known-slave redis-master 172.29.240.163 6379
sentinel known-sentinel redis-master 172.29.240.163 26379 61c51a37f2db1673ddcf5dc3fe6816c9ba83a408
sentinel known-sentinel redis-master 172.29.225.104 26379 85e2ce3707304f7b3b4d0f7281dcd6932094c8b6
sentinel current-epoch 29

Slave1

port 26379
sentinel myid 61c51a37f2db1673ddcf5dc3fe6816c9ba83a408
sentinel monitor redis-master 172.29.245.6 6379 1
sentinel down-after-milliseconds redis-master 5000
sentinel failover-timeout redis-master 60000
sentinel auth-pass redis-master XXXXXXXXXXXXXXXX

# Generated by CONFIG REWRITE
dir "/"
sentinel config-epoch redis-master 29
sentinel leader-epoch redis-master 27
sentinel known-slave redis-master 172.29.225.104 6379
sentinel known-slave redis-master 172.29.240.163 6379
sentinel known-sentinel redis-master 172.29.245.6 26379 5080eb7a0e27f9fb6ea6a197bf1abd334608943f
sentinel known-sentinel redis-master 172.29.225.104 26379 85e2ce3707304f7b3b4d0f7281dcd6932094c8b6
sentinel current-epoch 29

Slave2

port 26379
sentinel myid 85e2ce3707304f7b3b4d0f7281dcd6932094c8b6
sentinel monitor redis-master 172.29.245.6 6379 1
sentinel down-after-milliseconds redis-master 5000
sentinel failover-timeout redis-master 60000
sentinel auth-pass redis-master XXXXXXXXXXXX

# Generated by CONFIG REWRITE
dir "/"
sentinel config-epoch redis-master 29
sentinel leader-epoch redis-master 26
sentinel known-slave redis-master 172.29.225.104 6379
sentinel known-slave redis-master 172.29.240.163 6379
sentinel known-sentinel redis-master 172.29.245.6 26379 5080eb7a0e27f9fb6ea6a197bf1abd334608943f
sentinel known-sentinel redis-master 172.29.240.163 26379 61c51a37f2db1673ddcf5dc3fe6816c9ba83a408
sentinel current-epoch 29

My Sentinel config also looks good:

[root@redis-1 ~]# redis-cli -p 26379 info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=redismaster,status=ok,address=172.29.245.6:6379,slaves=2,sentinels=3

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.