Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I just completed a migration from Windows server 2008 R2 to Ubuntu 10.04. I manage a java application (Java 6, Tomcat) that is having some performance issues. I'd like to use JMX to try and troubleshoot, but I can't seem to get jvisualvm to connect.

If I do a ps -ef | grep "java", I see the following parameters.

-Dcom.sun.management.jmxremote.port=8084 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.authenticate=false

Netstat shows that port 8084 is listening on 0.0.0.0. In my config, JMX is setup to bind to the FQDN of the server (we use a private DNS server). My firewall (IPTABLES/UFW) is setup to allow all outgoing traffic, and to allow incoming traffic on port 8084.

The server itself is virtual with two NICs, a public and a private. The public NIC's gateway is disabled so that connections can only come in on the private side.

When I try to connect jvisualvm to my app server using JMX, I get the following error in jvisualvm.

Cannot connect to [FQDN OMITTED]:8084 using server:jmx:rmi:///jndi/rmi://[FQDN OMITTED]:8084/jmxrmi

If I look in the jvisualvm log, I see the following trace.

NFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]: connect(service:jmx:rmi:///jndi/rmi://[FQDN OMITTED]:8084/jmxrmi)
java.io.EOFException: SSL peer shut down incorrectly
                at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
                at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:789)
Caused: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
                at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:808)
                at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1120)
                at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:623)
                at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
                at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
                at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
                at java.io.DataOutputStream.flush(DataOutputStream.java:106)
                at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
Caused: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: 
                javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
                at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
                at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
                at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
                at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
                at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
Caused: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: 
                javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
                at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
                at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
                at javax.naming.InitialContext.lookup(InitialContext.java:392)
                at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
                at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
                at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:257)
Caused: java.io.IOException: Failed to retrieve RMIServer stub
                at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
                at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.tryConnect(JmxModelImpl.java:451)
[catch] at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.connect(JmxModelImpl.java:395)
                at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.connect(JmxModelImpl.java:216)
                at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.<init>(JmxModelImpl.java:205)
                at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:61)
                at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:42)
                at com.sun.tools.visualvm.core.model.ModelFactory.getModel(ModelFactory.java:111)
                at com.sun.tools.visualvm.tools.jmx.JmxModelFactory.getJmxModelFor(JmxModelFactory.java:69)
                at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.addJmxApplication(JmxApplicationProvider.java:267)
                at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.createJmxApplication(JmxApplicationProvider.java:185)
                at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationImpl(JmxApplicationsSupport.java:283)
                at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationInteractive(JmxApplicationsSupport.java:261)
                at com.sun.tools.visualvm.jmx.impl.AddJMXConnectionAction$1.run(AddJMXConnectionAction.java:80)
                at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:577)
                at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1030)

Does anyone have any ideas?

share|improve this question
add comment

1 Answer 1

up vote 1 down vote accepted

The problem is that even if JMX is configured to listen on port 8084, after the initial connection is made, the JMX host will dynamically pick a different port for the rest of the "conversation". My firewall allowed traffic through 8084, but was blocking the subsequent traffic.

There are two options.

  1. Allow inbound traffic on all ephemeral ports (yikes!).
  2. Develop a JMX wrapper that uses a single port for communication.
share|improve this answer
    
If you are on Java 7 update 4 or later there is hope with the flag `` -Dcom.sun.management.jmxremote.rmi.port=7091`` See this blog post: hirt.se/blog/?p=289 –  davey Oct 9 '13 at 21:55
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.