3

Ubuntu 11.04, node.js 0.6.16, nginx 1.0.8

I started debugging the following error that we were logging from the front end quite frequently while under heavy load (~ 3500 users actively long polling / making api calls):

parsererror:Error: jQuery171045845469435053_1347386706235 was not called",

All of the calls generating this error are long polls going through an nginx proxy to a node.js server.

While this is occurring, I see the following errors appear in following places on the server:

In /var/log/syslog I see bursts of these every few minutes:

[6356088.813219] TCP: Possible SYN flooding on port 8894. Sending cookies.

In the nginx error.log, I see a bit more frequent bursts of the following two errors:

2012/09/11 18:13:05 [error] 25104#0: *1229679431 upstream timed out (110: Connection timed out) while connecting to upstream,

and

2012/09/11 18:10:38 [error] 25103#0: *1229382210 recv() failed (104: Connection reset by peer) while reading response header from upstream, 

================================

I have tuned (among others) the following system parameters over the past few months while our traffic was steadily increasing:

  1. set ulimit very high (16000 I think) for the user running the processes
  2. set the nginx worker_connections to 16024
  3. set the sysctl net.ipv4.tcp_max_syn_backlog and net.core.somaxconn to 4096

=================================

This seems to suggest that the node.js server is closing connections ungracefully or being forced to close them by the system. How can I test my hypothesis?

0

You must log in to answer this question.

Browse other questions tagged .