Errors trying to start ZetaSQL planner

Question

I'm trying to run a Beam pipeline with SQL transforms, parsed with ZetaSQL. I begin with setting options with

options.setPlannerName("org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner");

When I try creating my SqlTransform with any given query, I get

java.util.ServiceConfigurationError: org.apache.beam.repackaged.sql.com.google.zetasql.ClientChannelProvider: Provider org.apache.beam.repackaged.sql.com.google.zetasql.JniChannelProvider could not be instantiated
    at java.util.ServiceLoader.fail (ServiceLoader.java:232)
    at java.util.ServiceLoader.access$100 (ServiceLoader.java:185)
    at java.util.ServiceLoader$LazyIterator.nextService (ServiceLoader.java:384)
    at java.util.ServiceLoader$LazyIterator.next (ServiceLoader.java:404)
    at java.util.ServiceLoader$1.next (ServiceLoader.java:480)
    at org.apache.beam.repackaged.sql.com.google.zetasql.ClientChannelProvider.loadChannel (ClientChannelProvider.java:31)
...
    at org.apache.beam.sdk.extensions.sql.SqlTransform.expand (SqlTransform.java:82)
    at org.apache.beam.sdk.Pipeline.applyInternal (Pipeline.java:539)
    at org.apache.beam.sdk.Pipeline.applyTransform (Pipeline.java:473)
    at org.apache.beam.sdk.values.PCollection.apply (PCollection.java:357)
...

I've added the following relevant dependencies to my POM in maven:

        <dependency>
            <groupId>org.apache.beam</groupId>
            <artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
            <version>2.15.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.beam</groupId>
            <artifactId>beam-sdks-java-extensions-sql</artifactId>
            <version>2.16.0</version>
        </dependency>

Is there something else I'm missing here?

score 1 · Answer 1 · answered Dec 05 '19 at 03:30

Unfortunately the ZetaSQL planner will today not work from a MAC, or older versions of Linux. Please see the comment from Rui in:

ZetaSQL Sample Using Apache beam

Looks like this PR maybe useful here ( I have not dug deeply into this to confirm):

https://github.com/google/zetasql/pull/3

As a workaround could you try on a newer version of linux? Maybe in a container?

score 0 · Answer 2 · answered Apr 07 '20 at 22:00

This is resolved in the latest version of ZetaSQL, which will be used with Beam 2.21. You can also try pulling down a newer version of ZetaSQL (at least 2020.03.2):

    <dependency>
        <groupId>com.google.zetasql</groupId>
        <artifactId>zetasql-jni-channel</artifactId>
        <version>2020.03.2</version>
    </dependency>

Errors trying to start ZetaSQL planner

2 Answers2