graph builder of dse configuration

Question

It seems like messing with the configuration, like trying to tune the number of thread readers for vertices & edges, cause a lot of unexplained exceptions, also there is an issue with trying to set the batch size. It seems to only work with the default settings produced by the executable. I've got a lot of exception, while trying "to play" with those values, some of them are [1] CAS exception of cassandra, something about the inability to create more partition keys. [2] Cassandra timeout during write query at consistency ONE

and more. As there is no reference within the documentation about how to solve those issues, I don't know how to continue. It seems like everything there is very delicate and shaky, so any little change causes tons of exceptions.

This is for using graph-loader-6.0.1 and dse 6.0.0 or dse 6.0.1

For example : com.datastax.dsegraphloader.exception.LoadingException: com.datastax.driver.core.exceptions.InvalidQueryException: Resource limited exceeded on added vertices, properties and edges. Maximum of 100000 allowed. Please split transaction into multiple smaller ones and retry.

This is what I get when I try to use some config.

this is the groovy file for configuration :

// config
config preparation: false
config create_schema: false
config load_new: true
config load_edge_threads: 5
config load_vertex_threads: 5
config batch_size: 5000

// orders
inputfiledir = '/home/dseuser/'
profileInput = File.text(inputfiledir + "soc-pokec-profiles.txt").
        delimiter("\t").header('user_id','public','completion_percentage','gender','region','last_login','registration','age',
'body','I_am_working_in_field','spoken_languages','hobbies','I_most_enjoy_good_food','pets','body_type',
'my_eyesight','eye_color','hair_color','hair_type','completed_level_of_education','favourite_color',
'relation_to_smoking','relation_to_alcohol','sign_in_zodiac','on_pokec_i_am_looking_for','love_is_for_me',
'relation_to_casual_sex','my_partner_should_be','marital_status','children','relation_to_children','I_like_movies',
'I_like_watching_movie','I_like_music','I_mostly_like_listening_to_music','the_idea_of_good_evening',
'I_like_specialties_from_kitchen','fun','I_am_going_to_concerts','my_active_sports','my_passive_sports','profession',
'I_like_books','life_style','music','cars','politics','relationships','art_culture','hobbies_interests',
'science_technologies','computers_internet','education','sport','movies','travelling','health','companies_brands',
'holder1','holder2')

relationInput = File.text(inputfiledir + "soc-pokec-relationships.txt").
        delimiter("\t").header('auser','buser')

profileInput = profileInput.transform {
                if (it['completion_percentage'] == 'null') { it.remove('completion_percentage')};
                if (it['gender'] == 'null') { it.remove('gender')};
                if (it['last_login'] == 'null') { it.remove('last_login')};
                if (it['registration'] == 'null') { it.remove('registration')};
                if (it['age'] == 'null') { it.remove('age')};
                it
               }

load(profileInput).asVertices {
        label "user"
        key "user_id"
}

load(relationInput).asEdges {
        label "relation"
        outV "auser", {
                label "user"
                key "user_id"
        }
        inV "buser", {
                label "user"
                key "user_id"
        }
}

I tried to use the soc-pokec (social network) from stanford (available in web). I had to loose most of the config to solve the issue. Note that there is totally no correlation between the numbers in the exception, and the settings I mad in the config.