Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

My first question here - hope I'm in the right place.

In essence, I have a Java app that occasionally seems to have difficulty connecting and I've run out of ideas for where to look. Hoping for some inspiration & advice.

In detail then...

Background:

  • Two 2K8r2 physical servers (call them 01 and 02) that sit on corporate LAN and are domain joined.

  • Each server hosts a Java app that is used to make secure connections to an internet hosted web service for the transfer of financial data.

  • Java apps are triggered by Windows Task Scheduler and run under a domain based service account which has local admin rights on the server.

  • When triggered, the Java app connects to an internet based web service and uploads/downloads files depending on execution script.

  • The Java app is configured to connect via our MS proxy (TMG 2010) to the web service over HTTPS.

  • The web service requires client authentication and a cert is intalled in the Java cert store on each server.

  • Our proxy is configured to NOT perform HTTPS inspection and NOT require authentication for connections to the web service.

  • Each server has two tasks; Up and Down. The Up tasks run regularly (every 20 mins between 7:00am and 7:00pm) while the Down tasks run either once a day or just in the morning.

The problem:

Most of the time all runs normally. All the tasks report occasional connection failures. This is thought to be down to the prevailing network conditions and brittleness of the Java app.

Less frequently (once every 1 to 2 weeks), there will be an uninterrupted sequence of connection failures - usually by the Up task on server 01 - which can go on for hours.

I receive an email alert on each failure and I have run remote console sessions to the server during one of these failures to try and work out what is going wrong.

I have noticed the following symptoms which are only present during a sequence of failures:

  • Browsing (via IE) to our corporate SharePoint Intranet results in a logon prompt, even though I'm logged into the server with my normal Domain account.

  • Attempting to browse to an external website fails - no logon prompt, just an IE error saying it can't connect.

I find it hard to believe that the sustained connection failures are due to prevailing network conditions - dropped packets etc.

My gut reaction is that there is something wrong with server 01, since server 02 is not as badly affected.

Sample error messages:

  • Connection failed 'Error when logging on - 'java.lang.RuntimeException: Error while silently connecting: org.w3c.www.protocol.http.HttpException: java.io.IOException: Invalid response from proxy: null''

  • Connection failed 'Error when logging on - 'java.lang.RuntimeException: Error while silently connecting: org.w3c.www.protocol.http.HttpException: java.net.ConnectException: Connection timed out: connect''

  • Connection failed 'Error when logging on - 'java.lang.RuntimeException: Error while silently connecting: org.w3c.www.protocol.http.HttpException: java.net.SocketException: Software caused connection abort: recv failed''

Edit: Debug level log file info:

2013-02-01 08:43:52,249 DEBUG - PKCS11SendRequest:sendRequest:Connecting to 'https://webserviceurl.com:443/path1/path2' ...
2013-02-01 08:43:52,249 DEBUG - PKCS11SendRequest:sendRequest: Protocol: 'https'
2013-02-01 08:43:52,249 DEBUG - PKCS11SendRequest:sendRequest: Host: 'webserviceurl.com, port 443
2013-02-01 08:43:52,249 DEBUG - PKCS11SendRequest:sendRequest: ServletPath: '/path1/path2'
2013-02-01 08:43:52,249 DEBUG - PKCS11SendRequest:sendRequest: ServletSessionID: ''
2013-02-01 08:43:52,249 DEBUG - PKCS11SendRequest:sendRequest: opening connection
2013-02-01 08:43:52,249 DEBUG - IaikSSLPkcs11SecurityProvider:getSecureRandom Entry  Not using the PKCS11 device as it is too slow.
2013-02-01 08:43:52,249 DEBUG - IaikSSLPkcs11SecurityProvider:getSecureRandom Exit
2013-02-01 08:43:52,249 DEBUG - PKCS11SendRequest:sendRequest: setting sslcontext
2013-02-01 08:43:52,264 DEBUG - PKCS11SendRequest:sendRequest: configuring rest of connection
2013-02-01 08:43:52,311 DEBUG - PKCS11SendRequest:sendRequest: getting connection OutputStream
2013-02-01 08:43:52,311 DEBUG - PKCS11SendRequest:sendRequest: got connection OutputStream
2013-02-01 08:43:52,311 DEBUG - PKCS11SendRequest:sendRequest: sending request object
2013-02-01 08:43:52,311 DEBUG - PKCS11SendRequest:sendRequest: flushing and closing connection OutputStream
2013-02-01 08:43:52,327 DEBUG - PKCS11SendRequest:sendRequest: Exception incurred 'java.lang.RuntimeException: Error while silently connecting: org.w3c.www.protocol.http.HttpException: java.io.IOException: Invalid response from proxy: null
    at org.w3c.www.protocol.http.HttpURLConnection.error(Unknown Source)
    at org.w3c.www.protocol.http.HttpURLConnection.checkReply(Unknown Source)
    at org.w3c.www.protocol.http.HttpURLConnection.getResponseCode(Unknown Source)
    at au.com.sienna.gwhttpclient.mailbox.n.a(Unknown Source)
    at au.com.sienna.gwhttpclient.mailbox.GatewareHttpMailboxBean.a(Unknown Source)
    at au.com.sienna.gwhttpclient.mailbox.GatewareHttpMailboxBean.a(Unknown Source)
    at au.com.sienna.gwhttpclient.mailbox.GatewareHttpMailboxBean.logon(Unknown Source)
    at au.com.sienna.gwhttpclient.mailbox.GatewareHttpMailboxBean.logon(Unknown Source)
    at au.com.sienna.gwhttpclient.APPNAME.APPNAME.h(Unknown Source)
    at au.com.sienna.gwhttpclient.APPNAME.APPNAME.if(Unknown Source)
    at au.com.sienna.gwhttpclient.APPNAME.APPNAME.main(Unknown Source)
'
2013-02-01 08:43:52,327 DEBUG - PKCS11SendRequest:sendRequest: No responseMsg object was received, setting up unknown error
2013-02-01 08:43:52,327 DEBUG - PKCS11SendRequest:sendRequest: exit
2013-02-01 08:43:52,327 DEBUG - GatewareHttpMailboxBean:sendRequest: No responseMsg object was received, setting up unknown error
Connection failed 'Error when logging on - 'java.lang.RuntimeException: Error while silently connecting: org.w3c.www.protocol.http.HttpException: java.io.IOException: Invalid response from proxy: null''
share|improve this question
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.