It sounds like you're doing premature optimization. If you already have a load that warrants that architecture, then you should also have facts about CPU load etc from your current servers.
So basically the data is going through 3 machines , HaProxy, NGIX and Finally CF. Is Latency a big issue here?
Normally, no. Each layer adds a little bit of latency of course, but usually it's less than 10 milliseconds, even with general-purpose servers and software.
- Keep it simple.
- Keep it standardized.
How about at first doing:
Internet -> | nginx -> CF |
And then later on:
Internet -> | nginx(SSL)| -> | -> | CF |
------------- | ------
| -> | CF |
And lastly, to fully answer your question: Yes your setup does make sense; when the SSL decryption alone is more than a single server can handle.
Essentially you're suggesting a modified version of what Willy Tarreau wrote about in this excellent overview of load balancing technique. In many cases it might be simpler to just use firewall hashing, or routing tricks, or DNS Round Robin at the very front (in front of the SSL accelerators) instead of using HAPRoxy in TCP mode as you suggest -- maybe consider that.