How to make a TCP dump where it is guaranteed that all the packets that really pass through the network are captured, and nothing is missed?
We have an issue with 3rd party vendor who provides a solution on top of SCTP stack, which he also implements.
Under quite high throughput (52 000 messages/sec, average message size is 500 bytes) the SCTP link breaks.
We believe that the bug is in the vendor SCTP stack.
But the vendor says, this happens because SCTP stack sends a message, doesn't receive ACK on it, sends a number of retransmits, doesn't receive ACKs on them as well and closes the SCTP link.
So the vendor says, this is the network which is guilty, because it loses packets.
In the TCP dumps on both sides, client and server we see that the original messages reaches the server and see that the server doesn't answer with ACK. But the vendor says that TCP dumps are not reliable, that when capturing a TCP dump, some packets could be not captured, because libpcap library works only within one hardware thread, its power can be not enough to log all the packets.
52 000 messages/sec, average message size is 500 bytes, so 26 MB/sec in total, 4 SCTP links are used.
Hardware: CPU E5-2670, 2.6 GHz, 8 HW threads
Network: 10 GBit, the traffic is between HP blades, which are located in one rack.