Exclude document(s) if condition true

Question

I have three fields in an entity:

1. establishmentNameEn
2. IsTelPublishDa
3. isTelSecret

I have fuzzy search on establishmentNameEn. And now i want to apply condition to exclude document(s) if field IsTelPublishDa value is 0 or isTelSecret value is 1.

My final query is: (+establishmentNameEn:kamran~1 +(-IsTelPublishDa:[0 TO 0] -isTelSecret:[1 TO 1]))

But it is not returning result.

Query code:

private org.apache.lucene.search.Query excludeDoc(QueryBuilder queryBuilder) {
    List<org.apache.lucene.search.Query> queries = new ArrayList<>();
    queries.add(queryBuilder.keyword().onField("IsTelPublishDa").matching(0).createQuery());
    queries.add(queryBuilder.keyword().onField("isTelSecret").matching(1).createQuery());

    BooleanQuery.Builder builder = new BooleanQuery.Builder();
    for (Query qu : queries) {
        builder.add(qu, BooleanClause.Occur.MUST_NOT);
    }

    return builder.build();
}

Main method:

Query fuzzyQuery = queryBuilder.keyword().fuzzy().withEditDistanceUpTo(1).onField("establishmentNameEn").matching(word).createQuery();
luceneQuery.add(fuzzyQuery);
luceneQuery.add(excludeDoc(queryBuilder));
BooleanQuery.Builder builder = new BooleanQuery.Builder();
for (Query qu : luceneQuery) {
    builder.add(qu, BooleanClause.Occur.MUST);
}

Answer 1

This will never match anything, because the boolean query only contains negative clauses:

    BooleanQuery.Builder builder = new BooleanQuery.Builder();
    for (Query qu : queries) {
        builder.add(qu, BooleanClause.Occur.MUST_NOT);
    }

    return builder.build();

That's quite confusing, but that's how Lucene works, and you're using a low-level Lucene API when you're using BooleanQuery.Builder.

Solution #1

If you want to avoid that kind of surprise in the future, make sure you always have positive clauses in your query. For example, refactor your code to add the "MUST_NOT" clause to the top-level boolean query:

// Main code
BooleanQuery.Builder builder = new BooleanQuery.Builder();
builder.add(queryBuilder.keyword().fuzzy().withEditDistanceUpTo(1).onField("establishmentNameEn").matching(word).createQuery(), BooleanClause.Occur.MUST);
builder.add(excludedDoc(queryBuilder), BooleanClause.Occur.MUST_NOT);

private org.apache.lucene.search.Query excludedDoc(QueryBuilder queryBuilder) {
    BooleanQuery.Builder builder = new BooleanQuery.Builder();
    builder.add(queryBuilder.keyword().onField("IsTelPublishDa").matching(0).createQuery(), BooleanClause.Occur.SHOULD);
    builder.add(queryBuilder.keyword().onField("isTelSecret").matching(1).createQuery(), BooleanClause.Occur.SHOULD);

    return builder.build();
}

Solution #2

Alternatively, you can just keep your code as is, but use the Hibernate Search DSL instead of BooleanQuery.Builder. The Hibernate Search DSL "fixes" some of the most confusing aspects of Lucene, so that this query will work as expected (matching all documents except those that match the clauses):

    BooleanJunction<?> booleanJunction = queryBuilder.bool();
    for (Query qu : queries) {
        booleanJunction.mustNot(qu);
    }
    return booleanJunction.createQuery();

More details...

If you want to know why exactly this doesn't work...

Boolean queries will not match anything by default, unless a (positive) clause matches a document, in which case matching documents will be filtered out based on other (positive or negative) clauses.

So in your case, the query doesn't match anything, and then it's filtered out with the "must not" clauses, so it still doesn't match anything.

Just adding a MatchAllDocs clause would make it work as expected:

    BooleanQuery.Builder builder = new BooleanQuery.Builder();
    builder.add(new MatchAllDocsQuery(), BooleanClause.Occur.MUST);
    for (Query qu : queries) {
        builder.add(qu, BooleanClause.Occur.MUST_NOT);
    }

    return builder.build();