You don't say what model switch you are using but my guess is that it is a low-cost unmanaged switch. In the conditions you are describing one of two things can be happening when one of your systems goes highly active:
- You are saturating an link -- most likely the uplink to the internet but not necessarily -- and some packets from all systems aren't getting through.
- Packets are getting dropped because buffer space inside the switch is becoming exhausted. This is much less likely but needs to be considered if the first reason doesn't pan out.
Consider what happens if you have two servers connected to one switch with Gigabit Ethernet and the uplink you have to your ISP is 100Mbit. (or less). If each server may be trying to upload at 60Mbit/second at the same time something has to give. Ideally your switch will sense a buffer usage threshold being exceeded and start sending PAUSE frames back to the servers. Done right this kind of flow control is very effective. If you have a more complex network than what I described here it can make things worse.
You can get managed switches that give you the capability to manage buffers and bandwidth. You just have to pay more and learn to use them. Cisco, Dell, Delta, SMC, and Accton all have similar models in that range.