Using logstash 1.4.2 pushing to an elastic search 1.1.1 instance on a separate server. Pulling log items from AWS SQS. Everything works fine the first several hours or so, but then the following begins to appear in the logstash log.

:message=>"Error reading SQS queue.", :error=>#<OpenSSL::SSL::SSLError: Socket closed>

As soon as the first of these errors appears the rate in which logstash empties the queue slows. Eventually it stops completely as more errors appear. The java process hosting logstash continues to operate without issue.

CPU consumption stays around 30% on an m3.xlarge instance, so it appears to have enough horsepower.

Does this sound like a bug, or is there something I need to do to either prevent socket loss or get logstash to reopen them?


There are these errors that may be related with your problem:

Error establishing SSL connections: OpenSSL::SSL::SSLError: Socket closed

I have been intermittently seeing "SSLError: Socket closed" upon connect, trying to establish an SSL socket connection. Hard to tell whether it was just my connection or not. Then I noticed that it happens without fail upon accept when running WEBrick in https mode--except from Safari. I mean, Webrick starts up okay, but when I navigate to https://localhost:3000 in my browser (though, strangely, Safari works fine), it fails with the following backtrace...

See more: https://jira.codehaus.org/browse/JRUBY-6346

This is the more specific:

SSL errors using tcp ssl input

I've configured tcp with ssl input on logstash. I 'm using nxlog to forward logs to logstash. I've configured tcp input on several ports. Everything works,I can get logs, but in log file I see errors like following...

See more: https://logstash.jira.com/browse/LOGSTASH-1095

In logstash the issue is still open.

I hope this help.

| improve this answer | |
  • Thanks for those, but the closest thing I found was this: logstash.jira.com/browse/LOGSTASH-1968. Basically the AWS Ruby SDK can't keep up. One of the commenters on that thread uploaded a new input script that uses the AWS Java SDK. I modified it a bit and added it to my installation. Throughput increased and system load dropped through the floor (from 30% to 1-2%). It's been running this way for over 18 hours now and the problem in my original question has not reappeared. – Todd Rowan Nov 14 '14 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.