Load data in to Amazon Neptune DB

Question

I have used the below code to load the data in to neptune db access.

curl -X POST \
    -H 'Content-Type: application/json' \
    https://your-neptune-endpoint:port/loader -d '
    {
      "source" : "s3://bucket-name/object-key-name",
      "format" : "format",
      "iamRoleArn" : "arn:aws:iam::account-id:role/role-name",
      "region" : "region",
      "failOnError" : "FALSE",
      "parallelism" : "MEDIUM",
      "updateSingleCardinalityProperties" : "FALSE",
      "queueRequest" : "TRUE",
      "dependencies" : ["load_A_id", "load_B_id"]
    }'

i have an csv file in s3 bucket which i am trying to add in the neptune DB. This gave me the response 200. But I could not se any data in my neptune DB instance in aws. Where do i view the uploaded data in aws neptune DB?

And also does loading data in to neptune DB always needs an s3 in the middle?

I am getting the below error while getting the status of the job id that i have inserted. Invalid header, ~id is required field What does it mean? — JhansiRani Kolapalli, Sep 16 '20 at 10:34

Kelvin Lawrence · Answer 1 · 2020-09-16T12:23:48.373

I am assuming from your description that you are trying to load property graph data and that the format is "csv". Each CSV file requires a specific header row.

For vertices it must include ~id and as a best practice should include ~label for the vertex ID and label.

For edges it must contain ~id, ~from and ~to. Once again it is a best practice to include a column for ~label.

~from and ~to are the IDs of the vertices connected by an edge

You can have additional entries in the header row that define the names of the properties.

You should have at least one CSV file for the vertices and at least one other for the edges. Larger CSV files are more efficient than lots of small ones.

For Property Graph (Gremlin) data, the bulk loader does require the files to be in an S3 bucket.

You can also add data to the graph using Gremlin queries of course.

There is some fairly detailed documentation here https://docs.aws.amazon.com/neptune/latest/userguide/bulk-load-tutorial-format.html

is there an api to get records in aws neptune db without using gremlin? — JhansiRani Kolapalli, Sep 17 '20 at 05:34
The bulk loader API is the alternative to using Gremlin queries to load property graph data. — Kelvin Lawrence, Sep 17 '20 at 14:55

score 0 · Answer 2 · answered Jan 05 '22 at 14:56

The result 200 to the post request don't provide the import status, it's only confirming that the load job has been started or queued.

The HTTP response will also contains the load-id that you can use to retrieve the status.

Here is a C# sample code to retrieve this id :

var response = client.PostAsJsonAsync("loader", request).Result;
response.EnsureSuccessStatusCode();
string loadId = response.Content.ReadFromJsonAsync<NeptuneLoadResponse>().Result.payload.loadId;

Once you got this id, you can query the real status of the import by calling the following REST route :

https://<your-neptune-endpoint>:<port>/loader/<load-id>?details=TRUE&errors=TRUE

Load data in to Amazon Neptune DB

2 Answers2