0

I am trying to execute query from Athena to show data from Neptune database. I am using Athena Neptune connector to connect to Neptune DB and show the data in Athena query editor.

However I get error when I run the following query

SELECT * FROM  "datasource/datacatalog".<dbname>.<tablename> limit 10;

Error Message:

 Encountered an exception[java.lang.NullPointerException] from your LambdaFunction[neptune_connector lambda function name] executed in context[S3SpillLocation{bucket='S3 bucketname', key='<file prefix name/2e6fedb0-9366-4d83-8a69-20472d7ff850/', directory=true}]

I have done so far.

  1. Created a new data catalog and database and table (manually, componenttype: vertex).
  2. Connected datacatalog with neptune connector lambda
  3. Full access to Athena,S3,Neptune and Glue has been to the role under which Neptune connector lambda runs.
  4. An existing S3 bucket name has been provided in spillbucket variable for connector lambda.

References:

  1. Amazon Athena Neptune connector - https://github.com/awslabs/aws-athena-query-federation/tree/master/athena-neptune
  2. Connecting to the data source - https://docs.aws.amazon.com/athena/latest/ug/connect-to-a-data-source-lambda.html#connect-to-a-data-source-lambda-connecting

However I am able to see table info when I run the following query but the actual select query does not work.

describe `datasource/datacatlogname`.<dbname>.<tablename>;

Cloudwatch logs:

java.lang.NullPointerException: java.lang.NullPointerException
java.lang.NullPointerException  at
com.amazonaws.athena.connectors.neptune.propertygraph.PropertyGraphHandler.executeQuery(PropertyGraphHandler.java:112)
at
com.amazonaws.athena.connectors.neptune.NeptuneRecordHandler.readWithConstraint(NeptuneRecordHandler.java:113)
at
com.amazonaws.athena.connector.lambda.handlers.RecordHandler.doReadRecords(RecordHandler.java:19 at
com.amazonaws.athena.connector.lambda.handlers.RecordHandler.doHandleRequest(RecordHandler.java:158)
at
com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:138)
at
com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:103)
user2026504
  • 69
  • 1
  • 9

2 Answers2

0

Check if the lambda configuration is setup properly to access the Neptune database. Make sure the lambda security group attached is properly setup to access the database

The connector is available in Serverless Application Reppository, try deploying from there

Abhishek Mishra
  • 26
  • 2
  • I have existing lambdas which connect to Neptune for data insert/fetch. I have deployed connector lambda in the same VPC/Subnets and has same SG. I am using my own cloudformation yaml file to deploy lambda and it uses same role as existing lambdas which has access to Neptune. I won't able to deploy the code from Serverless Application Repository due to permission issue. However I have used source code available on AWS git to create jar file and deploy it. – user2026504 Jun 27 '22 at 21:16
  • this can only happen if component type is not read correct at this code: String type = recordsRequest.getSchema().getCustomMetadata().get("componenttype"); TableSchemaMetaType tableSchemaMetaType = TableSchemaMetaType.valueOf(type.toUpperCase()); There is a log entry logger.debug("readWithConstraint: enter - " + recordsRequest.getSplit()); can you check details in cloudwatch – Abhishek Mishra Jun 28 '22 at 03:16
0

I was able to find out the issue. When I was creating the table manually in console I was inserting separatorChar and componenttype under Serde parameters.

https://github.com/awslabs/aws-athena-query-federation/blob/master/athena-neptune/docs/aws-glue-sample-scripts/manual/sample-cli-script.sh

It should be under table properties.

enter image description here

user2026504
  • 69
  • 1
  • 9