How to remote debug HBase coprocessor - RegionObserver?
I'm using Cloudera Quickstart.
There is a question about it, but there is no information about where to put that exec command.
Also I tried this instructions, but HBase doesn't start with that.
So what is the right way to remote debug coprocessors?
You didn't mention what IDE you use, but here is how I got the setup for IntelliJ.
Select Run/Debug -> Add new configuration (+ sign) -> Remote -> fill out the hostname where the regionserver runs (leave it to localhost, if you run HBase locally). In my case, the hostname is skdevhad6.dev.dszn.cz.
Next, edit the hbase-env.sh file and find the HBASE_OPTS environment value setting. You should add there:
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=0.0.0.0:5005
This will ensure that the JDB will listen on all interfaces (0.0.0.0) on port 5005 using JDB transport protocol.
This is, how the HBASE_OPTS might look afterwards:
export HBASE_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=0.0.0.0:5005 -XX:+UseG1GC -XX:MaxGCPauseMillis=50 -XX:-ResizePLAB -XX:ParallelGCThreads=24 -XX:-OmitStackTraceInFastThrow -XX:+ParallelRefProcEnabled"
Preferably, if you have multiple HBase region servers, apply those settings to them too, so that you can pick the region you want to debug later (just add another Remote debug settings in IntelliJ for that).
Now, restart all the region servers to which you have applied the updated HBASE_OPTS version. If the restart was successful, go ahead and run the Remote debug session from IntelliJ.
If you see something like this, it means the debugger is attached and you may place you breakpoint wherever you want.
Go ahead and try it :) HF!
If the setup fail and you can't connect to the remote JDB, check that the region server actually listens on the port you specified on all interfaces using:
$ ss -tunlp | grep 5005 # where 5005 is the port it should listen to
tcp LISTEN 0 1 *:5005 *:* users:(("java",pid=28364,fd=5))
If it does, check the firewall of the server:
$ iptables -L
If the incoming traffic to the port is restricted, then you have found the problem.
