I am working with some hardware units which uses mobile Internet to communicate to my application server.
Initially, the hardware had a hard-coded IP address (load balancer) in the hardware for communication which was used to route the traffic to my application server; now this creates a single point of failure if the load balancer goes down the whole redundant infrastructure behind it becomes useless since its objective is to share the load, not to maintain the availability.
Now I asked the hardware creator to switch it to using DNS, and by using multiple A records I got better results. In this case I can have applications servers directly working with the hardware; but if one server goes down some hardware units fail to communicate for a while, but it goes to the other server in 20-30 minutes; since DNS failover doesn't work always.
So how can I overcome this problem and keep my application server working somehow all the time?