Suppose I have a little web infrastructure for serving clients that contains nginx as a frontend server, couple of backends (upstreams), a messaging system, db and so on.
This stuff is located on two dedicated servers. There are LAN between servers and also each server has at least one public IP address. And all the public addresses are in the same broadcasting network segment (connected via switch or in VLAN - doesn't matter). Latency on both this networks is the same.
And I have two choices to communicate between the parts of this system: use local IPs throughout all configs or use public IPs instead. In first case I will have a little bit more complicated infrastructure ('caused by two networks). In second case my system will be more transparent but I'm worried about the privacy of the network communications.
It there exists some rules or best practices for using one approach or another?