I get the following warning as a System event: "TCP/IP has reached the security limit imposed on the number of concurrent TCP connect attempts." on o box running Win XP SP3. How can I fix this issue? Also how do I diagnose which apps are opening all these connections.
It could either be that you are actually wilfully establishing more than ten connections simultaneously (the actual establishing part that is, not total connections established) or you have some malware on the system.
As Izzy wrote this is supposedly a security feature to prevent malware from bogging down the system with incomplete connection attempts - so you may want to thouroughly check the health of your system first and try to find out what is causing this warning (try Malwarebyte's Antimalware if you want my personal favourite).
Some P2P applications might 'cause this, but most if you keep them updated should handle this by now. Here's the Microsoft article and here's some quotes from it, ending with an instruction in how you might find out the culprit:
The TCP/IP stack in Windows XP with Service Pack 2 (SP2) installed limits the number of concurrent, incomplete outbound TCP connection attempts. When the limit is reached, subsequent connection attempts are put in a queue and resolved at a fixed rate so that there are only a limited number of connections in the incomplete state. During normal operation, when programs are connecting to available hosts at valid IP addresses, no limit is imposed on the number of connections in the incomplete state. When the number of incomplete connections exceeds the limit, for example, as a result of programs connecting to IP addresses that are not valid, connection-rate limitations are invoked, and this event is logged.
At the command prompt, type Netstat –no Find the process with a large number of open connections that are not yet established. These connections are indicated by the TCP state SYN_SENT in the State column of the Active Connections information. Note the process identification number (PID) of the process in the PID column. Press CTRL+ALT+DELETE and then click Task Manager.
To see what is connecting, try NETSTAT from the command line, with its different switches for added effect.
This is a security feature of XP (SP2 onwards), and a pretty useless one at that. You need to edit the tcpip.sys file.
Windws XP SP2 introduces a few new twists to TCP/IP in order to babysit users and "reduce the threat" of worms spreading fast without control. In one such attempt, the devs seem to have limited the number of possible TCP connection attempts per second to 10 (from unlimited in SP1). This argumentative feature can possibly affect server and P2P programs that need to open many outbound connections at the same time.
This site provides an automatic (and manual method if you're worried about the Event ID 4226 Patcher application) for editing the tcpip.sys file. I used the automated (application) many years ago and it worked like a charm.
Try eEye's BIOT. Will work with XP SP3.
Bypassing Incomplete Outbound TCP Connection Limit (BIOT) is utility software for Windows XP SP2 and Windows Server 2003 SP1/SP2 which bypasses the incomplete outbound TCP connection limit. BIOT overwrites the TCP/IP connection limit in kernel memory, leaving the system file unmodified.
Another utility for this: http://www.softpedia.com/get/Tweak/Network-Tweak/TCP-Z.shtml