Calcite SQLParser is threadsafe .?

Question

while running a test like this, getting into different exceptions with each test run.

private static void testInParallelCaliciteParser() {
    SqlParser parser = SqlParser.create("select * from test",
            SqlParser.configBuilder().setConformance(SqlConformanceEnum.DEFAULT).build());
    ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(2);
    Runnable runnableTask = () -> {
        Instant startTime = Instant.now();
        try {
            //synchronized (parser) {                   
                SqlNode parseQuery = parser.parseQuery("select COUNT(*) FROM common.test_table");
            //}
            System.out.println(new Date() + " [" + Thread.currentThread().getName()
                    + "] after parsing qry, time taken in millis >> "
                    + Duration.between(startTime, Instant.now()).toMillis());
        } catch (SqlParseException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    };
    IntStream.range(0, 4).forEach(i -> executor.submit(runnableTask));
}

here is the exception.

org.apache.calcite.sql.parser.SqlParseException: Encountered "elect" at line 1, column 1. Was expecting one of: "ABS" ... "ALTER" ... "ARRAY" ... "AVG" ... "CALL" ... "CARDINALITY" ... "CASE" ... "CAST" ... "CEIL" ... "CEILING" ... "CHAR_LENGTH" ... "CHARACTER_LENGTH" ... "CLASSIFIER" ... "COALESCE" ... "COLLECT" ... "CONVERT" ... "COUNT" ... "COVAR_POP" ... "COVAR_SAMP" ... "CUME_DIST" ... "CURRENT" ... "CURRENT_CATALOG" ... "CURRENT_DATE" ... "CURRENT_DEFAULT_TRANSFORM_GROUP" ... "CURRENT_PATH" ... "CURRENT_ROLE" ... "CURRENT_SCHEMA" ... "CURRENT_TIME" ... "CURRENT_TIMESTAMP" ... "CURRENT_USER" ... "CURSOR" ... "DATE" ... "DELETE" ... "DENSE_RANK" ... "DESCRIBE" ... "ELEMENT" ... "EVERY" ... "EXISTS" ... "EXP" ... "EXPLAIN" ... "EXTRACT" ... "FALSE" ... "FIRST_VALUE" ... "FLOOR" ... "FUSION" ... "GROUPING" ... "HOUR" ... "INSERT" ... "INTERSECTION" ... "INTERVAL" ... "JSON_ARRAY" ... "JSON_ARRAYAGG" ... "JSON_EXISTS" ... "JSON_OBJECT" ... "JSON_OBJECTAGG" ... "JSON_QUERY" ... "JSON_VALUE" ... "LAG" ... "LAST_VALUE" ... "LEAD" ... "LEFT" ... "LN" ... "LOCALTIME" ... "LOCALTIMESTAMP" ... "LOWER" ... "MATCH_NUMBER" ... "MAX" ... "MERGE" ... "MIN" ... "MINUTE" ... "MOD" ... "MONTH" ... "MULTISET" ... "NEW" ... "NEXT" ... "NOT" ... "NTH_VALUE" ... "NTILE" ... "NULL" ... "NULLIF" ... "OCTET_LENGTH" ... "OVERLAY" ... "PERCENT_RANK" ... "PERIOD" ... "POSITION" ... "POWER" ... "PREV" ... "RANK" ... "REGR_COUNT" ... "REGR_SXX" ... "REGR_SYY" ... "RESET" ... "RIGHT" ... "ROW" ... "ROW_NUMBER" ... "RUNNING" ... "SECOND" ... "SELECT" ... "SESSION_USER" ... "SET" ... "SOME" ... "SPECIFIC" ... "SQRT" ... "STDDEV_POP" ... "STDDEV_SAMP" ... "SUBSTRING" ... "SUM" ... "SYSTEM_USER" ... "TABLE" ... "TIME" ... "TIMESTAMP" ... "TRANSLATE" ... "TRIM" ... "TRUE" ... "TRUNCATE" ... "UNKNOWN" ... "UPDATE" ... "UPPER" ... "UPSERT" ... "USER" ... "VALUES" ... "VAR_POP" ... "VAR_SAMP" ... "WITH" ... "YEAR" ... <UNSIGNED_INTEGER_LITERAL> ... <APPROX_NUMERIC_LITERAL> ... <DECIMAL_NUMERIC_LITERAL> ... <BINARY_STRING_LITERAL> ... <QUOTED_STRING> ... <PREFIXED_STRING_LITERAL> ... <UNICODE_STRING_LITERAL> ... "(" ... <LBRACE_D> ... <LBRACE_T> ... <LBRACE_TS> ... <LBRACE_FN> ... "?" ... "+" ... "-" ... <BRACKET_QUOTED_IDENTIFIER> ... <QUOTED_IDENTIFIER> ... <BACK_QUOTED_IDENTIFIER> ... ... <UNICODE_QUOTED_IDENTIFIER> ...

at org.apache.calcite.sql.parser.impl.SqlParserImpl.convertException(SqlParserImpl.java:365)
at org.apache.calcite.sql.parser.impl.SqlParserImpl.normalizeException(SqlParserImpl.java:150)
at org.apache.calcite.sql.parser.SqlParser.handleException(SqlParser.java:148)
at org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:163)
at org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:178)
at com.apporchid.cloudseer.db.qry.helper.TestMultiThreadSafe.lambda$5(TestMultiThreadSafe.java:142)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)

Caused by: org.apache.calcite.sql.parser.impl.ParseException: Encountered "elect" at line 1, column 1. Was expecting one of: "ABS" ... "ALTER" ... "ARRAY" ... "AVG" ... "CALL" ... "CARDINALITY" ... "CASE" ... "CAST" ... "CEIL" ... "CEILING" ... "CHAR_LENGTH" ... "CHARACTER_LENGTH" ... "CLASSIFIER" ... "COALESCE" ... "COLLECT" ... "CONVERT" ... "COUNT" ... "COVAR_POP" ... "COVAR_SAMP" ... "CUME_DIST" ... "CURRENT" ... "CURRENT_CATALOG" ... "CURRENT_DATE" ... "CURRENT_DEFAULT_TRANSFORM_GROUP" ... "CURRENT_PATH" ... "CURRENT_ROLE" ... "CURRENT_SCHEMA" ... "CURRENT_TIME" ... "CURRENT_TIMESTAMP" ... "CURRENT_USER" ... "CURSOR" ... "DATE" ... "DELETE" ... "DENSE_RANK" ... "DESCRIBE" ... "ELEMENT" ... "EVERY" ... "EXISTS" ... "EXP" ... "EXPLAIN" ... "EXTRACT" ... "FALSE" ... "FIRST_VALUE" ... "FLOOR" ... "FUSION" ... "GROUPING" ... "HOUR" ... "INSERT" ... "INTERSECTION" ... "INTERVAL" ... "JSON_ARRAY" ... "JSON_ARRAYAGG" ... "JSON_EXISTS" ... "JSON_OBJECT" ... "JSON_OBJECTAGG" ... "JSON_QUERY" ... "JSON_VALUE" ... "LAG" ... "LAST_VALUE" ... "LEAD" ... "LEFT" ... "LN" ... "LOCALTIME" ... "LOCALTIMESTAMP" ... "LOWER" ... "MATCH_NUMBER" ... "MAX" ... "MERGE" ... "MIN" ... "MINUTE" ... "MOD" ... "MONTH" ... "MULTISET" ... "NEW" ... "NEXT" ... "NOT" ... "NTH_VALUE" ... "NTILE" ... "NULL" ... "NULLIF" ... "OCTET_LENGTH" ... "OVERLAY" ... "PERCENT_RANK" ... "PERIOD" ... "POSITION" ... "POWER" ... "PREV" ... "RANK" ... "REGR_COUNT" ... "REGR_SXX" ... "REGR_SYY" ... "RESET" ... "RIGHT" ... "ROW" ... "ROW_NUMBER" ... "RUNNING" ... "SECOND" ... "SELECT" ... "SESSION_USER" ... "SET" ... "SOME" ... "SPECIFIC" ... "SQRT" ... "STDDEV_POP" ... "STDDEV_SAMP" ... "SUBSTRING" ... "SUM" ... "SYSTEM_USER" ... "TABLE" ... "TIME" ... "TIMESTAMP" ... "TRANSLATE" ... "TRIM" ... "TRUE" ... "TRUNCATE" ... "UNKNOWN" ... "UPDATE" ... "UPPER" ... "UPSERT" ... "USER" ... "VALUES" ... "VAR_POP" ... "VAR_SAMP" ... "WITH" ... "YEAR" ... <UNSIGNED_INTEGER_LITERAL> ... <APPROX_NUMERIC_LITERAL> ... <DECIMAL_NUMERIC_LITERAL> ... <BINARY_STRING_LITERAL> ... <QUOTED_STRING> ... <PREFIXED_STRING_LITERAL> ... <UNICODE_STRING_LITERAL> ... "(" ... <LBRACE_D> ... <LBRACE_T> ... <LBRACE_TS> ... <LBRACE_FN> ... "?" ... "+" ... "-" ... <BRACKET_QUOTED_IDENTIFIER> ... <QUOTED_IDENTIFIER> ... <BACK_QUOTED_IDENTIFIER> ... ... <UNICODE_QUOTED_IDENTIFIER> ...

at org.apache.calcite.sql.parser.impl.SqlParserImpl.generateParseException(SqlParserImpl.java:32563)
at org.apache.calcite.sql.parser.impl.SqlParserImpl.jj_consume_token(SqlParserImpl.java:32377)
at org.apache.calcite.sql.parser.impl.SqlParserImpl.SqlStmt(SqlParserImpl.java:3679)
at org.apache.calcite.sql.parser.impl.SqlParserImpl.SqlStmtEof(SqlParserImpl.java:3691)
at org.apache.calcite.sql.parser.impl.SqlParserImpl.parseSqlStmtEof(SqlParserImpl.java:197)
at org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:161)
... 7 more

org.apache.calcite.sql.parser.SqlParseException: Non-query expression encountered in illegal context at org.apache.calcite.sql.parser.impl.SqlParserImpl.convertException(SqlParserImpl.java:365) at org.apache.calcite.sql.parser.impl.SqlParserImpl.normalizeException(SqlParserImpl.java:150) at org.apache.calcite.sql.parser.SqlParser.handleException(SqlParser.java:148) at org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:163) at org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:178) at com.apporchid.cloudseer.db.qry.helper.TestMultiThreadSafe.lambda$5(TestMultiThreadSafe.java:142) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.calcite.runtime.CalciteException: Non-query expression encountered in illegal context at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:457) at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:550) at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:857) at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:842) at org.apache.calcite.sql.parser.impl.SqlParserImpl.checkNonQueryExpression(SqlParserImpl.java:309) at org.apache.calcite.sql.parser.impl.SqlParserImpl.Expression3(SqlParserImpl.java:14388) at org.apache.calcite.sql.parser.impl.SqlParserImpl.Expression2b(SqlParserImpl.java:14057) at org.apache.calcite.sql.parser.impl.SqlParserImpl.Expression2(SqlParserImpl.java:14098) at org.apache.calcite.sql.parser.impl.SqlParserImpl.Expression(SqlParserImpl.java:14029) at org.apache.calcite.sql.parser.impl.SqlParserImpl.LeafQueryOrExpr(SqlParserImpl.java:14006) at org.apache.calcite.sql.parser.impl.SqlParserImpl.QueryOrExpr(SqlParserImpl.java:13478) at org.apache.calcite.sql.parser.impl.SqlParserImpl.OrderedQueryOrExpr(SqlParserImpl.java:481) at org.apache.calcite.sql.parser.impl.SqlParserImpl.SqlStmt(SqlParserImpl.java:3653) at org.apache.calcite.sql.parser.impl.SqlParserImpl.SqlStmtEof(SqlParserImpl.java:3691) at org.apache.calcite.sql.parser.impl.SqlParserImpl.parseSqlStmtEof(SqlParserImpl.java:197) at org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:161) ... 7 more

Is there any reason you can't construct a new parser object for each thread? — Michael Mior, Jun 30 '20 at 14:50
Also, if you're asking about exceptions you're encountering, it would be helpful to post these exceptions. — Michael Mior, Jun 30 '20 at 14:50
This doesn't answer your question about thread-safety of the parser, but is there any reason you can't just construct a new parser on each thread? I don't think the memory overhead will be significant. — Michael Mior, Jul 01 '20 at 13:51
@Michael Mior, that is causing me little bit of extra time each time — Suresh Kumar Vepari, Jul 02 '20 at 00:32

score 0 · Answer 1 · answered Jul 02 '20 at 19:03

Thread safety is not an explicit design goal of the parser so you shouldn't count on it being thread-safe. If you are finding this to be a serious problem, I would suggest starting a conversation on the Calcite mailing list (dev@calcite.apache.org) to discuss possible solutions which may require changes to Calcite itself.

Calcite SQLParser is threadsafe .?

1 Answers1