I'm using 'socat' on Linux Centos 7 to diagnose some problems. Server and clients are all linux centos7, they all have multiple interfaces. Server 10.1.1.1 is on Switch 1, 10.1.1.2 is on Switch 2. Switch 1 and Switch 2 are Layer3 switches, directly connected, running Cisco NX-OS. This is situation I can't explain:
server 10.1.1.1 (connect to Switch 1, UDP packet sender) to send data via STDIN:
socat STDIO UDP4-DATAGRAM:22.214.171.124:12345
client 10.1.1.2 (connect to Switch 2, UDP packet receiver):
socat UDP4-RECVFROM:12345,ip-add-membership=126.96.36.199:10.1.1.2,fork,reuseaddr -
The client receives server's UDP packets, until I start another receiver on the same machine as the server 10.1.1.1:
socat UDP4-RECVFROM:12345,ip-add-membership=188.8.131.52:10.1.1.1,fork,reuseaddr -
The moment I start this second receiver, client on 10.1.1.2 stops receiving any data from server. I also use tcpdump and see that no packet has been received on that port on 10.1.1.2. In addition, from then onwards, even if I restart the server and clients, 10.1.1.2 does not receives anymore multicast packets from 10.1.1.1 anymore for THAT multicast prefix (239.2.2.x).
Is this a problem in switch routing? or I missed some parameters on the 'socat' command line? I checked that rp_filter=0 on that interface (10.1.1.x).
EDIT: Update the question to include more network setup. as more suspicion on this as a routing problem.
UPDATE: we found that the 10.1.1.x VLAN on the Layer3 switch does not have PIM enable. That probably means multicast routing is not enabled. That said, I'm not sure why the first time 10.1.1.2 on the different switch would receive data in the first place.