I am planing to add more web application servers to support increasing clients, deploying HAproxy and Keepalived for load balancing and High availability.
My server usage has the following characteristic:
- Jobs are not CPU intensive. Message are JSON text less than 100 character.
- Users will send message to server through Client device Y. Usually 4-5 messages per day
- Client devices X keep waiting message from server. If message is available at server, client device X must be able to get it within 2 seconds. Otherwise, this message is outdated.
For this reason,
- Client devices X is using long polling HTTP connection in order to be responsive. Each connection will last for 5 seconds and reconnect.
- Client devices X and Client devices Y are connected to same server, so X and Y can send message easily
If there are over 60,000 Client devices X connecting to server, my load balancer or router will be running out of TCP port. What is the best way to scale up for , say, 20,000 users?
My server is running on Ubuntu server, using tomcat and Java Servlet.