I integrated spring-cloud-sleuth with GCP support into an application. Under load the app suddenly stops reporting any spans until it is restarted.
The only tracing relevant log i can see is the following exception:
Unexpected error flushing spans java.lang.IllegalStateException: timeout waiting for onClose. timeoutMs=5000, resultSet=false
at zipkin2.reporter.stackdriver.internal.AwaitableUnaryClientCallListener.await(AwaitableUnaryClientCallListener.java:49)
at zipkin2.reporter.stackdriver.internal.UnaryClientCall.doExecute(UnaryClientCall.java:50)
at zipkin2.Call$Base.execute(Call.java:380)
at zipkin2.Call$Mapping.doExecute(Call.java:237) at zipkin2.Call$Base.execute(Call.java:380)
at zipkin2.reporter.AsyncReporter$BoundedAsyncReporter.flush(AsyncReporter.java:285)
at zipkin2.reporter.AsyncReporter$Flusher.run(AsyncReporter.java:354)
at java.base/java.lang.Thread.run(Unknown Source)
This exception happens a few times around the time the traces end and then never aggain (as if something permanently breaks)
I read in a spring-cloud-gcp issue (see here) that this can be related to to few executer threads so i already configured the number of threads to 8 (from 4).
