We are trying to improve the average latency of Java proxy application.
The Java application receives a message from a client and proxies it to a necessary server.
The protocol is binary, asynchronous.
The topology is:
Client (1) - Java proxy (2) - Server (3)
Call flow is
(1)-(2)-(3)-(2)-(1)
The requirement is 2 ms average latency overhead for one direction hop for Java proxy.
We've noticed that if we increase the number of clients, latency overhead is decreased linearly.
Let's say,
1 client -> Java proxy latency overhead = 6 ms.
2 clients -> Java proxy latency overhead = 3 ms.
4 clients -> Java proxy latency overhead = 1.5 ms.
On the other hand we must comply with latency requirement even with one client.
The same behavior we observer, if we replace Java implementation of proxy with Erlang implementation.
Can we do anything for low level OS tuning?
Java proxy host is 2 Nehalem CPU X5550 2.67GHz, 16 hw threads in total; Red Hat Enterprise Linux Server release 5.6