I have tried every conceivable combination in an attempt to cluster rabbitmq in an AWS environment. But to recap:
- Shutdown and removed erlang and rabbit distribution on my local ubuntu 14
- Tried the auto configuration modules around the web
- The ubuntu 14 version the comes default installed won't cut it.
- The erlang cookies match - this is demonstrated below
The hostname mismatch is the only puzzle. The node itself thinks it's hostname is 'q1' or 'q2' respectively. When I try to set the host name of the container to the private dns name of the host (so it can connect to the other node) the rabbit instance in the container crashes. Not below how
q2 but I shelled into an amazon private dns?
root@q2:~# hostname q2 root@q2:~# exit christian@q2:~$ logout Connection to ip-10-0-3-101.us-west-2.compute.internal closed.
I am using the latest rabbitmq docker image.
docker run -d --restart always --hostname q1 --name rabbitmq -p 4369:4369 -p 15671:15671 -p 25672:25672 -p 15672:15672 -p 5672:5672 -e RABBITMQ_HIPE_COMPILE=1 -e RABBITMQ_ERLANG_COOKIE='ilikecookies' rabbitmq:3-management
The service starts up just fine
root@q1:~# curl -I localhost:15672 HTTP/1.1 200 OK Content-Length: 1419 Content-Type: text/html Date: Fri, 20 Jan 2017 22:46:12 GMT last-modified: Fri, 20 Jan 2017 22:38:45 GMT Server: MochiWeb/1.0 (Any of you quaids got a smint?)
And here is the cookie from host q1
root@q1:~# docker exec -it rabbitmq /bin/bash root@q1:/# cat /var/lib/rabbitmq/.erlang.cookie ilikecookies root@q1:/#
Now I attempt to cluster it (from host q2 with q1 being the master)
root@q2:~# docker exec -it rabbitmq /bin/bash root@q2:/# rabbitmqctl stop_app Stopping node rabbit@q2 ... root@q2:/# rabbitmqctl join_cluster email@example.com Clustering node rabbit@q2 with 'firstname.lastname@example.org' ... Error: unable to connect to nodes ['email@example.com']: nodedown DIAGNOSTICS =========== attempted to contact: ['firstname.lastname@example.org'] email@example.com: * connected to epmd (port 4369) on ip-10-0-3-56.us-west-2.compute.internal * epmd reports node 'rabbit' running on port 25672 * TCP connection succeeded but Erlang distribution failed * suggestion: hostname mismatch? * suggestion: is the cookie set correctly? * suggestion: is the Erlang distribution using TLS? current node details: - node name: 'rabbitmq-cli-41@q2' - home dir: /var/lib/rabbitmq - cookie hash: quN0y0GUm2Zxv8VYc2eX9A== root@q2:/# cat /var/lib/rabbitmq/.erlang.cookie ilikecookies root@q2:/#
The question is HOW do you get these things to cluster? What's the missing ingredient? The error message means nothing around the web. Does anyone have some experience in this?
UPDATE AWS security group for these instances:
Custom TCP Rule TCP 1024 - 65535 0.0.0.0/0