4

I am using Powershell to pull data from Exchange via EWS API.

Import-Module "C:\ews_api\Microsoft.Exchange.WebServices.dll"
$ExchangeServer = 'EXCHANGE'
$ExchangeService = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService([Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2013_SP1)
$ExchangeService.Url = "https://$ExchangeServer/ews/exchange.asmx"
$SearchRootFolder = New-Object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Calendar)
$SearchFolderView =  New-Object Microsoft.Exchange.WebServices.Data.FolderView([int]::MaxValue)
$SearchResult = $ExchangeService.FindFolders($SearchRootFolder, $SearchFolderView)

Executing the last line of code will cause a HTTP connection against the Exchange server, which will be left in the TCP state TIME_WAIT for 2 minutes with Windows default settings. Hitting the Exchange server with a lot of queries will result in many TIME_WAIT sessions existing at the same time.

Get-NetTCPConnection | Group-Object State

   Count    Name
   -----    ----
      28    Listen
      10    Bound
       5    Established
       6    CloseWait
     189    TimeWait

There seems to be a limit around 190 TIME_WAIT sessions in Windows, because around that mark I am getting the following error in Powershell.

Microsoft.Exchange.WebServices.Data.ServiceRequestException: The request failed. Unable to connect to the remote server
---> System.Net.WebException: Unable to connect to the remote server
---> System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted 10.50.222.87:443

I thought that Windows can allocate a lot more ephemeral ports than that for TCP sessions? And indeed when using another Powershell code sample, I experience no problems.

$connections = @()
for ($i = 0; $i -lt 1000; $i++) {
    $connections += [System.Net.Sockets.TcpClient]::new($ExchangeServer, 443)
}

A thousand connections are opened without problem. However they are in TCP state ESTABLISHED and not in TIME_WAIT. Why am I only running into this issue on TIME_WAIT session states and is there a way to increase this limit?

Get-NetTCPConnection | Group-Object State

   Count    Name
   -----    ----
      28    Listen
    1010    Bound
    1003    Established
       8    CloseWait
1
  • There is a limit, but it is way more than 190. This may be a .NET HttpClient setting, or something more specific to the consumer (PowerShell).
    – Greg Askew
    May 15 at 17:45

2 Answers 2

2

I'm unable to find any documentation about such a limit, but you can modify the amount of time a socket remains in TIME_WAIT state:

https://learn.microsoft.com/en-us/biztalk/technical-guides/settings-that-can-be-modified-to-improve-network-performance

The article indeed recommends lowering it from the default (120) to 30 seconds in order to improve network performance:

enter image description here

2
  • +1 this somewhat betters the situation. Though I want to query a lot of folders and would prefer not to throttle myself, only making under 200 requests every 30 seconds is quite a limitation. Also the registry key needs a reboot and affects everyone on the server, which might be not acceptable on some servers May 15 at 12:26
  • Also, please note that a connection is defined by source IP, destination IP, source port and destination port; in this case, the destination IP and port on the Exchange server are always the same.... so what is causing the socket conflict appears to be the source IP/port. I suggest applying the same setting to the machine your are querying Exchange from, and/or to run your queries from multiple machines.
    – Massimo
    May 15 at 12:34
1

I was not able to reproduce the issue on a fresh Windows Server 2022 installation. First I thought it was related to some Group Policy setting, however this was not the case.

I since found out that something else was limiting the ports that can be allocated per user. In our case it was the Palo Alto Networks Terminal Server Agent. This agent is responsible for identifying users on terminal servers so that firewall rules can apply on a user basis.

Unfortunate side effect of that was that the ports per user are limited. For me the solution is to use another server where this agent is not present.

enter image description here

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .