I'm currently deploying a pair of HA firewall devices that will act as a transparent forward proxy (traffic will be directed through the proxy via routing rather than configuring a proxy URL on the client machines) for outbound traffic. I have the high availability configuration in place and working and I can see that the session state is being shared across both devices. When failover is triggered, the passive device will assume the IP addresses (actually the whole network adaptor is moved across as this is on AWS) of the previously active instance.
Client - - > firewall/proxy - - > web server
As a test, I setup a web server and created a large html file. I then used a client machine to retrieve this file using wget and curl (via my proxies) and during the file download I performed a manual failover. When I performed the failover the wget (same happened with curl) download got stuck. I then added connection timeouts and the wget command timed out and then restarted the download which worked fine although I could see that a new session was created. One thing to note is that this is a Cloud setup where failover times are a lot slower than on-premise high-spec devices so it can take between 15 - 60 seconds for failover to complete. I'm trying to ensure that my deployment will not impact highly on applications which will mainly be sending HTTP traffic.
Is it reasonable to expect a HTTP download to continue after a failover if session state is synchronized across the HA devices or should the client use timeouts and retries to start the download again?
Am I likely to need application teams to change their timeout and retry settings? What is considered normal for timeout and retry settings for applications that regularly send API requests? I'm hoping to avoid application teams having to change anything on their end once I deploy this.
Is there a way to prevent wget or curl from hanging during download when the connection is temporarily interrupted for up to a minute and automatically continue after the connection is restored on the device that has assumed the active role? I know you can terminate the request and have it continue the download from where it left off but that's not what application teams will be doing.
I'm basically replacing NAT Gateways on AWS with a HA pair of Nextgen firewalls with session synchronization and inspection capabilities and I dont want this to cause any operational issues.