Indexing fails on enabling force index in Titan/Janus

Question

I've written a JUnit Test to check against the generate-modern.groovy graph if marko exists.
My gremlin query being

"g.V().has('name','marko')";

As you can see in the generate-modern.groovy file that indexing is already applied on the name property of the person. I later made the following

query.force-index=true

property true in the dynamodb.properties file which blocks whole graph scan thereby making indexing mandatory. However it throws me the following exception

org.janusgraph.core.JanusGraphException: Could not find a suitable index to answer graph query and graph scans are disabled: [(name = marko)]:VERTEX

The above exception is raised from the following StandardJanusGraphTx class's method

    @Override
    public Iterator<JanusGraphElement> execute(final GraphCentricQuery query, final JointIndexQuery indexQuery, final Object exeInfo, final QueryProfiler profiler) {
        Iterator<JanusGraphElement> iter;
        if (!indexQuery.isEmpty()) {
            List<QueryUtil.IndexCall<Object>> retrievals = new ArrayList<QueryUtil.IndexCall<Object>>();
            for (int i = 0; i < indexQuery.size(); i++) {
                final JointIndexQuery.Subquery subquery = indexQuery.getQuery(i);

                retrievals.add(new QueryUtil.IndexCall<Object>() {
                    @Override
                    public Collection<Object> call(int limit) {
                        final JointIndexQuery.Subquery adjustedQuery = subquery.updateLimit(limit);
                        try {
                            return indexCache.get(adjustedQuery, new Callable<List<Object>>() {
                                @Override
                                public List<Object> call() throws Exception {
                                    return QueryProfiler.profile(subquery.getProfiler(), adjustedQuery, q -> indexSerializer.query(q, txHandle));
                                }
                            });
                        } catch (Exception e) {
                            throw new JanusGraphException("Could not call index", e.getCause());
                        }
                    }
                });
            }


            List<Object> resultSet = QueryUtil.processIntersectingRetrievals(retrievals, indexQuery.getLimit());
            iter = com.google.common.collect.Iterators.transform(resultSet.iterator(), getConversionFunction(query.getResultType()));
        } else {
            if (config.hasForceIndexUsage()) throw new JanusGraphException("Could not find a suitable index to answer graph query and graph scans are disabled: " + query);
            log.warn("Query requires iterating over all vertices [{}]. For better performance, use indexes", query.getCondition());

            QueryProfiler sub = profiler.addNested("scan");
            sub.setAnnotation(QueryProfiler.QUERY_ANNOTATION,indexQuery);
            sub.setAnnotation(QueryProfiler.FULLSCAN_ANNOTATION,true);
            sub.setAnnotation(QueryProfiler.CONDITION_ANNOTATION,query.getResultType());

            switch (query.getResultType()) {
                case VERTEX:
                    return (Iterator) getVertices().iterator();

                case EDGE:
                    return (Iterator) getEdges().iterator();

                case PROPERTY:
                    return new VertexCentricEdgeIterable(getInternalVertices(),RelationCategory.PROPERTY).iterator();

                default:
                    throw new IllegalArgumentException("Unexpected type: " + query.getResultType());
            }
        }

        return iter;
    }

};

As you can observe from the method that the exception is raised when the JointIndexQuery object is empty(arrayList being empty) and force index is true.
The problem is why the list is empty? when we have specified the indexing query against the name property in the generate-modern.groovy while querying from a JUnit Test.This works fine meaning the list is not empty when the same data is being preloaded into the gremlin server with the same file.

Could you provide your entire failing JUnit test? There's a lot of things that could be happening here. If you're simply using the `generate-modern.groovy` script and trying to load that outside the server start, you'll have issues, as it is set up to run as a hook on the server lifecycle and won't actually generate anything. If that's not the case, the script won't create an index if you've created any vertex labels on the graph you're using in your test. — pantalohnes, Jul 28 '17 at 12:07
No I am loading the script at the creation of server . I have verified this fact as when I switch force index property to false my test case passes successfully. — Abhishek Raj, Jul 28 '17 at 13:10
Success after turning `force-index` to false indicates to me that the simply indexes aren't getting created then. Do you already have data in the graph that you're trying to load this into? https://github.com/pluradj/titan-tp3-driver-example/blob/master/scripts/generate-modern.groovy#L13 assumes that you're using an empty graph with no pre-existing vertex labels. — pantalohnes, Jul 28 '17 at 13:14
Yes the graph is loaded properly . I ran the code under debugging mode and found that it never went to the else part of StandardJanusGraphTx class's method . This proves that indexes are getting created — Abhishek Raj, Jul 28 '17 at 13:28
What is the result of using the management class to get the index status? — pantalohnes, Jul 28 '17 at 13:30
The one that you get from `graph.openManagement()`. `def mgmt = graph.openManagement(); def propertyKey = mgmt.getPropertyKey('keyname'); mgmt.getGraphIndex('indexName').getIndexStatus(propertyKey)` — pantalohnes, Jul 28 '17 at 14:05

score 1 · Accepted Answer · answered Aug 03 '17 at 19:10

The personByName index definition uses a label constraint.

def personByName = mgmt.buildIndex("personByName", Vertex.class).addKey(name).indexOnly(person).buildCompositeIndex()

In order to take advantage of that index, you must use the label and the property. For example:

g.V().has('person', 'name', 'marko')

You can read more about this in the JanusGraph documentation http://docs.janusgraph.org/latest/indexes.html#_label_constraint

Indexing fails on enabling force index in Titan/Janus

1 Answers1