You may need to enable RDP keep-alive. Without this setting, a typical stateful device (firewall) connection that usually has a timeout will destroy the TCP connection for the RDP session when the timeout elapses.
When this setting is enabled, the server keep-alive prevents the stateful device from timing out the session. This setting also helps to ensure that the remote desktop session will change the state to disconnected in the event that it is actually disconnected. Many people set this to one minute.
Machine > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Connections
"This policy setting allows you to enter a keep-alive interval to ensure that the session state on the RD Session Host server is consistent with the client state.
After an RD Session Host server client loses the connection to an RD Session Host server, the session on the RD Session Host server might remain active instead of changing to a disconnected state, even if the client is physically disconnected from the RD Session Host server. If the client logs on to the same RD Session Host server again, a new session might be established (if the RD Session Host server is configured to allow multiple sessions), and the original session might still be active.
If you enable this policy setting, you must enter a keep-alive interval. The keep-alive interval determines how often, in minutes, the server checks the session state. The range of values you can enter is 1 to 999,999.
If you disable or do not configure this policy setting, a keep-alive interval is not set and the server will not check the session state."