For over 3 years I have routinely observed the following: while our server is busy compressing backups (7z, all cores, idle priority), the first attempt to RDP to any other computer on the local network results in a timeout. The second attempt then connects and opens immediately. Cancelling and quickly retrying the connection does not help, I have to wait out the 60 seconds.
When I record the exchange with Wireshark on both sides of the connection, I see that around 6kB of data is exchanged within the first 400ms, then the communication stalls. After 60 seconds the remote endpoint sends a RST,ACK and the RDP dialog keeps spinning for 30 more seconds before declaring a timeout. There is nothing obviously wrong in either of the captures and they don't seem to be doing any related network communication while this is happening.
What's odd is that if I do cancel and retry the connection after waiting a bit, it eventually connects. I still see the RST packet for the first connection show up on the 60-second mark. The timeout seems to be a global thing.
So... any clue what I'm dealing with here?