Apache Beam 2.1.0 with Google DatastoreIO calls Guava Preconditions checkArgument on non-existing function in GAE

Question

When building a dataflow template which should read from datastore I get the following error in stackdriver logs (from Google App Engine):

java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;I)V at org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read.withQuery(DatastoreV1.java:494) .... my code

This happens in a line where a read from Datastore would be generated. The pom dependency

<!-- https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-io-google-cloud-platform -->
<dependency>
    <groupId>org.apache.beam</groupId>
    <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
    <version>2.1.0</version>
</dependency>

References

<!-- https://mvnrepository.com/artifact/com.google.guava/guava -->
<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>20.0</version>
</dependency>

But this version does not contain a method checkArgument(String string) in class Preconditions, nor does any other version I looked at. As mentioned above, the template should be built inside a GAE flexible environment project and later executed, but the template generation fails.

If I let a main function generate the template locally, it works fine, but as soon as the project is in GAE, it fails.

Any Input is highly appreciated

EDIT: the dependency tree for com.google.guava:

[INFO] xy.company_name.test:bcc.dataflow.project_name:war:0.0.3
[INFO] \- org.apache.beam:beam-runners-google-cloud-dataflow-java:jar:2.1.0:compile
[INFO]    +- org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:jar:2.1.0:compile
[INFO]    |  \- com.google.cloud.bigdataoss:gcsio:jar:1.4.5:compile
[INFO]    |     \- (com.google.guava:guava:jar:18.0:compile - omitted for conflict with 20.0)
[INFO]    +- org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.1.0:compile
[INFO]    |  +- io.grpc:grpc-core:jar:1.2.0:compile
[INFO]    |  |  \- (com.google.guava:guava:jar:19.0:compile - omitted for conflict with 20.0)
[INFO]    |  +- com.google.api:gax-grpc:jar:0.20.0:compile
[INFO]    |  |  +- io.grpc:grpc-protobuf:jar:1.2.0:compile
[INFO]    |  |  |  +- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO]    |  |  |  \- io.grpc:grpc-protobuf-lite:jar:1.2.0:compile
[INFO]    |  |  |     \- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO]    |  |  +- com.google.api:api-common:jar:1.1.0:compile
[INFO]    |  |  |  \- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO]    |  |  +- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO]    |  |  \- com.google.api:gax:jar:1.3.1:compile
[INFO]    |  |     \- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO]    |  +- com.google.cloud:google-cloud-core-grpc:jar:1.2.0:compile
[INFO]    |  |  +- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO]    |  |  \- com.google.protobuf:protobuf-java-util:jar:3.2.0:compile
[INFO]    |  |     \- (com.google.guava:guava:jar:18.0:compile - omitted for conflict with 19.0)
[INFO]    |  +- com.google.cloud.datastore:datastore-v1-proto-client:jar:1.4.0:compile
[INFO]    |  |  \- (com.google.guava:guava:jar:18.0:compile - omitted for conflict with 19.0)
[INFO]    |  +- io.grpc:grpc-all:jar:1.2.0:runtime
[INFO]    |  |  \- io.grpc:grpc-protobuf-nano:jar:1.2.0:runtime
[INFO]    |  |     \- (com.google.guava:guava:jar:19.0:runtime - omitted for duplicate)
[INFO]    |  +- com.google.cloud:google-cloud-core:jar:1.0.2:compile
[INFO]    |  |  \- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO]    |  +- com.google.cloud.bigtable:bigtable-protos:jar:0.9.7.1:compile
[INFO]    |  |  \- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO]    |  +- com.google.cloud.bigtable:bigtable-client-core:jar:0.9.7.1:compile
[INFO]    |  |  +- com.google.auth:google-auth-library-appengine:jar:0.6.1:compile
[INFO]    |  |  |  \- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO]    |  |  \- (com.google.guava:guava:jar:19.0:compile - omitted for duplicate)
[INFO]    |  \- com.google.guava:guava:jar:20.0:compile
[INFO]    +- com.google.auth:google-auth-library-oauth2-http:jar:0.7.1:compile
[INFO]    |  \- (com.google.guava:guava:jar:19.0:compile - omitted for conflict with 20.0)
[INFO]    \- com.google.cloud.bigdataoss:util:jar:1.4.5:compile
[INFO]       \- (com.google.guava:guava:jar:18.0:compile - omitted for conflict with 20.0)

UPDATE:

After Adding

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>20.0</version>
        </dependency>
    </dependencies>
</dependencyManagement>

And updating a function handling DatastoreEntities it seems to work again! Sorry for bothering, sometimes it just helps to structure the problem and stackoverflow is a great help to do so.

score 0 · Answer 1 · answered Sep 20 '17 at 20:58

0

As noted here that corresponds to Preconditions.checkArgument(boolean, String, int). The Z is a boolean, and the Ljava/lang/String; is a String, and the I is an integer. This method should exist in Guava 20.0.

answered Sep 20 '17 at 20:58

Ben Chambers

6,070
11
16

Thank you for your answer! Any suggestions why the template generation fails in Google AppEngine and does not locally? Is there a way to check which version is used in GAE? – Malte Sep 21 '17 at 09:14
Updated the question and it seems like its working again. Thank you Ben – Malte Sep 21 '17 at 11:07

Apache Beam 2.1.0 with Google DatastoreIO calls Guava Preconditions checkArgument on non-existing function in GAE

1 Answers1