1

I am trying to setup a google TCP internal Load Balancer. Instance group behind this lb consists of redis-server processes listening on port 6379. Out of these redis instances, only one of them is master.

Problem: Add a TCP health check to detect the redis master and make lb divert all traffic to redis master only.

Approach: Added a TCP Health Check for the port 6379. In order to send the command role to redis-server process and parse the response, I am using the optional params provided in the health check. Please check the screenshot here.

Result: Health check is failing for all. If I remove the optional request/response params, health check starts passing for all.

Debugging:

  1. Connected to lb using netcat and issued the command role, it sends the response starting with *3(for master) and *5(for slave) as expected.
  2. Logged into instance and stopped redis-server process. Started listening on port 6379 using nc -l -p 6379 to check what exactly is being received at the instance's side in the health check. It does receive role\r\n.
  3. After step 2, restarted redis-server and ran MONITOR command in redis-cli, to watch log of commands received by this process. Here there is no log of role. This means, instance is receiving the data(role\r\n) over tcp but is not received by the process redis-cli(as per MONITOR command) or something else is happening. Please help.
  • Are your Redis clients able to use Redis Sentinels? If so, you could load balance among any of a pool of Redis Sentinels; the client would thus ask the Sentinel for the currrent Redis master -- rather than having your load balancer do the Redis master discovery. – Castaglia Apr 6 '17 at 14:23
  • @Castaglia No. I have explored that approach and kept it as a secondary option. First, I am trying to make lb discover the redis master. – Yadvendar Apr 7 '17 at 7:05
  • Were you able to solve this issue? If so please consider posting an answer, it will certainly benefit the community. – Carlos Aug 18 '17 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.