Titan with dynamodb

Question

I am trying to upload a social networking friends list data to titan with dynamodb. The dataset is from https://archive.org/download/friendster-dataset-201107 with the details about the dataset discussed in https://archive.org/details/friendster-dataset-201107 I am new to this area and need suggestions about selecting the best strategy to upload the data at a rapid rate.

Answer 1

You can write your own parsers to read the files. Here is how I did it for a similar project. Everything was done in Java. We had millions of vertices in json format and one json per line in a file.

First step is to create the vertex labels and property keys and their explicit types. Then create indexes on your desired properties.

Then we read one line at a time and created the vertex. We committed every 5000 vertices, for example.

Look also into bulk loading. http://s3.thinkaurelius.com/docs/titan/1.0.0/bulk-loading.html