Cannot connect to remote JMX

Question

I have a tomcat application on the remote host and need to connect it by JConsole. Application starts with params:

IP=`ifconfig eth0 | grep 'inet addr:' | cut -d ':' -f2 | cut -d ' ' -f1`

-Dcom.sun.management.jmxremote
-Djava.rmi.server.hostname=$IP
-Dcom.sun.management.jmxremote.port=9999
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false

Port 9999 is open, IP value is valid, I checked it. I can access it by telnet (telnet <my_host> <my_port>). Netstat:

netstat -a | grep LISTEN
tcp        0      0 *:9999                  *:*                     LISTEN

But I can't connect it via jconsole, I allways get same error:

Connection Failed: Retry?

But netstat -a shows that the connection is ESTABLISHED

I tried different addresses:

<my_host>:<my_port>

service:jmx:rmi://<my_host>:<my_port>/jndi/rmi://<my_host>:<my_port>/jmxrmi

service:jmx:rmi:///jndi/rmi://<my_host>:<my_port>/jmxrmi

I also tried to add files .../conf/remote.users and .../remote.acl and write pathes to that files in prorerties -Dcom.sun.management.jmxremote.password.file and -Dcom.sun.management.jmxremote.access.file but it had no effect.

When I deploy this app on my local machine, i can connect to it at "localhost:9999"

Help somebody, what could be the problem?

First check connectivity using command `telnet `, if you not able to connect then check firewall setting between these two machines. — Mahendra, Mar 14 '16 at 09:24
Also echo the IP `echo $IP` after IP=`ifconfig eth0 | grep 'inet addr:' | cut -d ':' -f2 | cut -d ' ' -f1` to verify valid IP value. — Mahendra, Mar 14 '16 at 09:26
Try check that the `$IP` you're using is not somehow translated in the machine's `/etc/hosts` to internal loopback address? — Avihoo Mamka, Mar 14 '16 at 10:26

Mahendra · Accepted Answer · 2016-03-14T13:17:15.817

If you are able to telnet <my_host> <my_port> then issue must be because of Firewall, since in JMX once handshake done on configured port i.e then a new port is assigned for further communication and probably your Firwall is not allowing to open a new port.

To cross check simply try to run below Java JMX Client which will try to display HeapMemoryUsage.

import java.util.Set;

import javax.management.MBeanServerConnection;
import javax.management.ObjectName;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;

public class JMXDemo {

    private static final String HOST = ""; // configure host <my_host> here
    private static final String PORT = ""; // configure port <my_port> here

    private static void testJMX() throws Exception {
        JMXServiceURL url = new JMXServiceURL("service:jmx:rmi://" + HOST + "/jndi/rmi://" + HOST + ":" + PORT
                + "/jmxrmi");

        JMXConnector jmxConnector = JMXConnectorFactory.connect(url);
        try {
            MBeanServerConnection mbeanServerConnection = jmxConnector.getMBeanServerConnection();

            ObjectName mbeanName = new ObjectName("java.lang:type=Memory");
            javax.management.openmbean.CompositeDataSupport obj = null;

            obj = (javax.management.openmbean.CompositeDataSupport) mbeanServerConnection.getAttribute(mbeanName,
                    "HeapMemoryUsage");
            Set<String> keySet = obj.getCompositeType().keySet();

            for (String key : keySet) {
                System.out.print(key + "=" + obj.get(key) + ", ");
            }
        } finally {
            jmxConnector.close();
        }
        System.out.println("\n==========================");
    }

    public static void main(String args[]) throws Exception {
        testJMX();
    }

}

To Enable JMX logging Create a logging.properties file

handlers= java.util.logging.ConsoleHandler
.level=ALL

java.util.logging.FileHandler.pattern = jmx.log
java.util.logging.FileHandler.limit = 50000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter = java.util.logging.XMLFormatter

java.util.logging.ConsoleHandler.level = FINEST
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter

// Use FINER or FINEST for javax.management.remote.level - FINEST is
// very verbose...
//
javax.management.level=FINEST
javax.management.remote.level=FINEST

java.security.debug=all

Compile the java code and run using below command

 java -Djava.util.logging.config.file=./logging.properties JMXDemo

This will print so many logs on console, you can grep "port".

It's blocked in `JMXConnector jmxConnector = JMXConnectorFactory.connect(url);` — Kirill, Mar 14 '16 at 12:24
JMXConnector jmxConnector = JMXConnectorFactory.connect(url); throws: `Exception in thread "main" java.rmi.ConnectException: Connection refused to host: 192.168.123.32; nested exception is: ` `java.net.ConnectException: Connection timed out` — Kirill, Mar 14 '16 at 12:34
I am trying this. For that we have to [enable the JMX logging](https://blogs.oracle.com/jmxetc/entry/tracing_jmx_what_s_going) also TCP logging — Mahendra, Mar 14 '16 at 13:11
You're right! It was a firewall. This is a test machine and I can just turn it off. — Kirill, Mar 14 '16 at 14:14
Your example has been extremely helpful in uncovering my issue. I was tunneling the JMX port over SSH and presuming the RMI port would tunnel too, but the RMI host it's trying to connect to is trying to bypass the SSH tunnel, which of course is blocked. — Coder Guy, Dec 06 '17 at 14:26

score 4 · Answer 2 · edited Jun 22 '17 at 00:31

Write a basic Java program:

import java.util.*;
import java.io.*;

class testCode{
  public static void main(String args[]){
    System.out.println("just killing time...do not enter a value and kill me ");
    Scanner sc = new Scanner(System.in);
    sc.nextInt();
  }    
}

Compile and then run it with the following parameters:

java -Dcom.sun.management.jmxremote=true \
 -Dcom.sun.management.jmxremote.port=12345 \
 -Dcom.sun.management.jmxremote.authenticate=false \
 -Dcom.sun.management.jmxremote.ssl=false \
 -Djava.rmi.server.hostname=10.1.10.167 \
 testCode

Replace rmi.server.hostname with the IP of the machine where Java code is being executed, then use jconsole to remote connect with 10.1.10.167:12345. No user id and password required.

Thanks a lot! I have multiple networks, this option solved my problem: -Djava.rmi.server.hostname. — hao, Oct 20 '17 at 02:16

score 1 · Answer 3 · answered Mar 26 '16 at 13:00

try all these flags:

-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.port=9999 
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false 
-Djava.rmi.server.hostname=$IP

if Dcom.sun.management.jmxremote.local.only is true, it will block remote connections.

Cannot connect to remote JMX

3 Answers3

Linked