1

I tried Connecting the AWS Neptune with this Java code and got the error , NoHostAvailable Exception

approach 1:

public static void main(String[] args) throws Exception {
        Cluster.Builder builder = Cluster.build();
        builder.addContactPoint("endpoint");
        builder.port(8182);
        builder.enableSsl(true);
        builder.keyStore("pem-file");
        Cluster cluster = builder.create();
        GraphTraversalSource g = traversal().withRemote(DriverRemoteConnection.using(cluster));
        System.out.println(g.V().limit(10).toList());
        cluster.close();
       }}

approach 2:

    Cluster cluster = Cluster.build("endpoint").
                enableSsl(true).keyStore("pem").
                    handshakeInterceptor( r -> {
                    NeptuneNettyHttpSigV4Signer sigV4Signer = null;
                    try {
                     sigV4Signer = new NeptuneNettyHttpSigV4Signer("us-east-2", new 
       DefaultAWSCredentialsProviderChain());
                    } catch (NeptuneSigV4SignerException e) {
                        e.printStackTrace();
                    }
                    try {
                        sigV4Signer.signRequest(r);
                    } catch (NeptuneSigV4SignerException e) {
                        e.printStackTrace();
                    }
                    return r;
                }).create();
        Client client=Cluster.open("src\\conf\\remote-objects.yaml").connect();
        client.submit("g.V().limit(10).toList()").all().get();

what ever I do, I am getting this error:

Sep 02, 2021 3:18:34 PM io.netty.channel.ChannelInitializer exceptionCaught
    WARNING: Failed to initialize a channel. Closing: 
    java.lang.RuntimeException: java.lang.NullPointerException
 
 
 
 org.apache.tinkerpop.gremlin.driver.Channelizer$AbstractChannelizer.initChannel(Channelizer.java:117)
    Caused by: org.apache.tinkerpop.gremlin.driver.exception.NoHostAvailableException: All hosts 
    are considered unavailable due to previous exceptions. Check the error log to find the actual 
    reason.

I need the code or the document to connect my Gremlin code in .java file to AWS neptune. I am struggling and tried various number of ways, 1.created EC2 instance and did installed maven and apache still got error and code is running in Server(EC2), i want code to present in IntelliJ

it would be more helpful, if I get the Exact Code any way. what should be added in remote-objects.yaml.

if we require Pem-file to access Amazon Neptune, please help with the creation of it.

Kelvin Lawrence
  • 14,674
  • 2
  • 16
  • 38
Harish
  • 11
  • 3
  • Is your EC2 instance in the same VPC as Neptune or at least has access to that VPC? Is IAM Authentication enabled on the Neptune cluster? For the most part connecting to Neptune is no different than connecting to any other Gremlin Server once you have the VPC and SigV4 (if needed) configured. – Kelvin Lawrence Sep 06 '21 at 18:20
  • yes, EC2 instance is in same VPC. I have used the process https://docs.aws.amazon.com/neptune/latest/userguide/iam-auth-connecting-gremlin-java.html . using TinkerPop 3.4.11 or higher – Harish Sep 07 '21 at 04:54
  • I just want to know, whether we can connect gremlin query in java code ( intelliJ) to get /read/update data on the AWS neptune instance ?. if we can how to do exactly ? just assume i have just AWS neptune cluster with active VPC and IAM disabled and how to connect to java code in my localmachine intelliJ. please let me know all exact procedure to start from this state to establishing connection between intelliJ and AWS neptune – Harish Sep 07 '21 at 10:28
  • I will add an answer soon with an example but there are two parts to it. As well as the Java code your local machine will need access to the VPC. There are many ways to do that such as SSH tunnel or load balancer. – Kelvin Lawrence Sep 07 '21 at 11:10
  • To answer your basic question - yes all of this is possible and regularly done. – Kelvin Lawrence Sep 07 '21 at 11:16
  • I tried the above procedure, got the error: Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: org.apache.tinkerpop.gremlin.driver.exception.NoHostAvailableException: All hosts are considered unavailable due to previous exceptions. Check the error log to find the actual reason.at org.apache.tinkerpop.gremlin.driver.Client.submit(Client.java:258) org.apache.tinkerpop.gremlin.driver.exception.NoHostAvailableException: All hosts are considered unavailable due to previous exceptions. Check the error log to find the actual reason. – Harish Sep 08 '21 at 06:24

1 Answers1

0

Assuming SSL is enabled but IAM is not, in terms of Java code, this is all you need to create the connection.

    Cluster.Builder builder = Cluster.build();
    builder.addContactPoint("localhost");
    builder.port(8182);
    builder.enableSsl(true);
    builder.serializer(Serializers.GRAPHBINARY_V1D0);

    cluster = builder.create();
    drc = DriverRemoteConnection.using(cluster);
    g = traversal().withRemote(drc);

You may need to add an entry to your /etc/hosts file to get the SSL certs to resolve correctly such as:

127.0.0.1 localhost my-neptune-cluster.us-east-1.neptune.amazonaws.com

If you find that using localhost with SSL enabled does not work then use the actual Neptune cluster DNS name and make the edit to your /etc/hosts file.

The last thing you will need to do is create access to the Neptune VPC from your local machine. One way is using an SSH tunnel as explained in this post

Kelvin Lawrence
  • 14,674
  • 2
  • 16
  • 38
  • I tried above method and got error: Caused by: org.apache.tinkerpop.gremlin.driver.exception.NoHostAvailableException: All hosts are considered unavailable due to previous exceptions. Check the error log to find the actual reason. Security Inbound rules are 1. type-HTTPS ,protocol-TCP port-, source - xx.xxx.xxx.xxx/xx, 2. type-Custom TCP ,protocol-TCP port-8182, source - xxx.xx.xxx.xxx/xx. 3. 1. type-custom TCP ,protocol-TCP port-8182, source - .xx.xxx.xxx./xx. please let me know if we have to do any modification to the inbound rules, – Harish Sep 08 '21 at 06:46
  • error at this line: client.submit("g.V()"); – Harish Sep 08 '21 at 06:57
  • Your security group will need to allow traffic on port 8182. With respect to the Gremlin, you are mixing the two ways of sending a query. `submit` is only used when sending queries as text. When using Java and a `DriverRemoteConnection` you need to do something like `result = g.V().toList()` . No need to use submit. – Kelvin Lawrence Sep 08 '21 at 12:14
  • security group is allowing the traffic on port 8182 – Harish Sep 08 '21 at 15:02
  • if I used the above process instead of g.V().toList() i used g.V().limit(2) and got output : [GraphStep(vertex,[]), RangeGlobalStep(0,2)], if i try to use g.V().toList(), i am getting NoHostAvailableException. traffic is on port 8182 is working fine – Harish Sep 08 '21 at 15:05
  • Without a terminal step such as `next` or `toList` your query is not sent to the server. You are just seeing the toString of the bytecode that would have been sent. – Kelvin Lawrence Sep 08 '21 at 15:16
  • can you curl the Neptune endpoint's status API from your local machine? If not your networking is still the issue. `curl https://neptune-host:8182/status` Did you make the edit I suggested to your `/etc/hosts` file ? – Kelvin Lawrence Sep 08 '21 at 15:17
  • when i do the curl operation from my local machine ( if i have edit the /etc/hosts ), I am getting desired output. – Harish Sep 08 '21 at 15:26
  • and from your code are you using `localhost` or the server DNS name? To get SSL to resolve it needs to be the latter (so use whatever worked for curl in other words). – Kelvin Lawrence Sep 08 '21 at 15:27
  • I am using localhost, im not sure, where to find the DNS name – Harish Sep 08 '21 at 15:28
  • `localhost` will not resolve the SSL certificate most likely. Try using the real server DNS name. – Kelvin Lawrence Sep 08 '21 at 15:29
  • okay, i will look into EC2 servers and check for DNS names – Harish Sep 08 '21 at 15:30
  • it would be more helpful, if you let me know where we can find the DNS names, VPCID is DNS name ? – Harish Sep 08 '21 at 15:36
  • I mean the DNS name of the Neptune endpoint. I assume you had to use that for curl as well? Just copy it from the Neptune AWS console/web page or get it from the CLI using `aws neptune describe-db-clusters | grep -i endpoint` Just do not use the `https://` part, replace with `wss://` – Kelvin Lawrence Sep 08 '21 at 15:41
  • I found the DNS name, i tried using it instead of Localhost, I am able to see the output [GraphStep(vertex,[]), RangeGlobalStep(0,2)], which is not desired, i have edited localhost details like 127.0.0.1 DNS name endpoint – Harish Sep 08 '21 at 16:00
  • You need the terminal step I mentioned above like `next()` or `toList()`. You may want to study the Apache TinkerPop documentation a little as these are all Gremlin basic items not specific to Neptune or any other DB - https://tinkerpop.apache.org/docs/current/reference/ – Kelvin Lawrence Sep 08 '21 at 17:23