Currenty the architecture is organised like this:
- 192.168.1.10, 192.168.1.11, 192.168.1.12 - mesos master with marathon and consul server
- 192.168.1.21, 192.168.1.22, 192.168.1.23 - three mesos slave with consul agent
Every machine the following configuration:
- its own IP address as DNS server
- HAPROXY with consul-template for automatic generation of haproxy.cfg
- consul-mesos for registering the running docker containers with its host IP address, in order to avoid the service for having 22.214.171.124 address in the DNS when resolving from different machine
The problem that i am having currently is that when i start docker container with a service in a bridged mode, the address is resolved by the DNS as it should be (if you have luigi.service.consul it will be done without a problem), but when i try to
CURL -L http://luigi.service.consul/, i have random
503 error codes, sometimes the address is resolved and sometimes is not.
Any ideas how to investigate this?
I have checked
/etc/resolv.conf and from times to times, I can see that the DNS address is changed to the old DNS IP address (126.96.36.199), should i use the host IP address as DNS IP, or i need to use the consul leader address?
I have tried debugging the tcpflow, when the response is 503 it is like waiting some time, like it is failing to resolve the service.