Quarkus Vertx timeout in JWT autorization phase in POST request upload file

Question

I'm building a project based on quarkus and microprofile with the following extensions: rest-client,health, resteasy-jsonb, metrics, openapi,fault,jdbc-postgres,hibernate-orm,jwt,mongodb-client,kotlin,resteasy-jsonb.

In one of the REST Resources I have an upload method which handle csv file uploads, due to the file size and connection bandwidth the file upload process takes more than two seconds; after that time the following exception arise:

2020-03-13 10:58:45,435 WARNING [io.ver.cor.imp.BlockedThreadChecker] (vertx-blocked-thread-checker) Thread Thread[vert.x-eventloop-thread-1,5,main]=Thread[vert.x-eventloop-thread-1,5,main] has been blocked for 2590 ms, time limit is 2000 ms: io.vertx.core.VertxException: Thread blocked
    at com.sun.crypto.provider.AESCrypt.<clinit>(AESCrypt.java:299)
    at com.sun.crypto.provider.AESCipher.<init>(AESCipher.java:185)
    at com.sun.crypto.provider.AESCipher$General.<init>(AESCipher.java:54)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.security.Provider$Service.newInstance(Provider.java:1595)
    at javax.crypto.Cipher.chooseProvider(Cipher.java:859)
    at javax.crypto.Cipher.init(Cipher.java:1395)
    at javax.crypto.Cipher.init(Cipher.java:1326)
    at org.jose4j.jwe.SimpleAeadCipher.getInitialisedCipher(SimpleAeadCipher.java:55)
    at org.jose4j.jwe.SimpleAeadCipher.encrypt(SimpleAeadCipher.java:70)
    at org.jose4j.jwe.SimpleAeadCipher.isAvailable(SimpleAeadCipher.java:131)
    at org.jose4j.jwe.AesGcmKeyEncryptionAlgorithm.isAvailable(AesGcmKeyEncryptionAlgorithm.java:123)
    at org.jose4j.jwa.AlgorithmFactory.isAvailable(AlgorithmFactory.java:86)
    at org.jose4j.jwa.AlgorithmFactory.registerAlgorithm(AlgorithmFactory.java:71)
    at org.jose4j.jwa.AlgorithmFactoryFactory.initialize(AlgorithmFactoryFactory.java:94)
    at org.jose4j.jwa.AlgorithmFactoryFactory.<init>(AlgorithmFactoryFactory.java:45)
    at org.jose4j.jwa.AlgorithmFactoryFactory.<clinit>(AlgorithmFactoryFactory.java:36)
    at org.jose4j.jws.JsonWebSignature.getAlgorithm(JsonWebSignature.java:253)
    at org.jose4j.jws.JsonWebSignature.getAlgorithm(JsonWebSignature.java:231)
    at org.jose4j.jws.JsonWebSignature.verifySignature(JsonWebSignature.java:205)
    at org.jose4j.jwt.consumer.JwtConsumer.processContext(JwtConsumer.java:222)
    at org.jose4j.jwt.consumer.JwtConsumer.process(JwtConsumer.java:433)
    at io.smallrye.jwt.auth.principal.DefaultJWTTokenParser.parse(DefaultJWTTokenParser.java:86)
    at io.quarkus.smallrye.jwt.runtime.auth.MpJwtValidator.authenticate(MpJwtValidator.java:55)
    at io.quarkus.smallrye.jwt.runtime.auth.MpJwtValidator.authenticate(MpJwtValidator.java:28)
    at io.quarkus.smallrye.jwt.runtime.auth.MpJwtValidator_ClientProxy.authenticate(MpJwtValidator_ClientProxy.zig:134)
    at io.quarkus.security.runtime.QuarkusIdentityProviderManagerImpl.handleProvider(QuarkusIdentityProviderManagerImpl.java:121)
    at io.quarkus.security.runtime.QuarkusIdentityProviderManagerImpl.authenticate(QuarkusIdentityProviderManagerImpl.java:89)
    at io.quarkus.security.runtime.IdentityProviderManagerCreator_ProducerMethod_ipm_91f102be1b2a781216db8a81e6ab4b9b1a84f03c_ClientProxy.authenticate(IdentityProviderManagerCreator_ProducerMethod_ipm_91f102be1b2a781216db8a81e6ab4b9b1a84f03c_ClientProxy.zig:105)
    at io.quarkus.smallrye.jwt.runtime.auth.JWTAuthMechanism.authenticate(JWTAuthMechanism.java:50)
    at io.quarkus.smallrye.jwt.runtime.auth.JWTAuthMechanism_ClientProxy.authenticate(JWTAuthMechanism_ClientProxy.zig:72)
    at io.quarkus.vertx.http.runtime.security.HttpAuthenticator.attemptAuthentication(HttpAuthenticator.java:66)
    at io.quarkus.vertx.http.runtime.security.HttpAuthenticator_ClientProxy.attemptAuthentication(HttpAuthenticator_ClientProxy.zig:150)
    at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$1.handle(HttpSecurityRecorder.java:36)
    at io.quarkus.vertx.http.runtime.security.HttpSecurityRecorder$1.handle(HttpSecurityRecorder.java:25)
    at io.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1034)
    at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:131)
    at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:130)
    at io.vertx.ext.web.impl.RouterImpl.handle(RouterImpl.java:54)
    at io.vertx.ext.web.impl.RouterImpl.handle(RouterImpl.java:36)
    at io.quarkus.vertx.http.runtime.VertxHttpRecorder$1.handle(VertxHttpRecorder.java:91)
    at io.quarkus.vertx.http.runtime.VertxHttpRecorder$1.handle(VertxHttpRecorder.java:82)
    at io.vertx.core.http.impl.WebSocketRequestHandler.handle(WebSocketRequestHandler.java:50)
    at io.vertx.core.http.impl.WebSocketRequestHandler.handle(WebSocketRequestHandler.java:32)
    at io.vertx.core.http.impl.Http1xServerConnection.handleMessage(Http1xServerConnection.java:136)
    at io.vertx.core.net.impl.VertxHandler$$Lambda$226/1718108476.handle(Unknown Source)
    at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369)
    at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:43)
    at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:232)
    at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:173)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
    at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
    at io.netty.handler.codec.http.websocketx.extensions.WebSocketServerExtensionHandler.channelRead(WebSocketServerExtensionHandler.java:102)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
    at io.vertx.core.http.impl.Http1xUpgradeToH2CHandler.channelRead(Http1xUpgradeToH2CHandler.java:109)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
    at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:328)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:315)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:429)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
    at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)

Is it possible to modify this 2 seconds value for a greater one?

score 3 · Accepted Answer · edited Mar 13 '20 at 21:28

3

This happens because the Vert.x's event loop is blocked for more than 2 seconds, which is configurable. You can change this using following parameter:

quarkus.vertx.max-event-loop-execute-time

edited Mar 13 '20 at 21:28

James Z

12,209
10
24
44

answered Mar 13 '20 at 16:53

Rafał Maciak

196
5

Please note that if you find yourself needing to change the above-mentioned config parameter then you are probably misusing Quarkus. Event loop thread execution time must remain very low otherwise you might block execution of other requests and your overall app throughput will be low and not scalable. – S. D. Aug 24 '22 at 17:24

S. D. · Answer 2 · 2022-08-25T10:16:14.433

Your long-running file upload blocks the event loop (at the core of Quarkus) and you should not block the event-loop thread.

You can follow this guide to use multipart-form file uploads and then, on your REST resource, implement something as follows:

  @POST
  @Consumes(MediaType.MULTIPART_FORM_DATA)
  @Produces(MediaType.APPLICATION_JSON)
  public Uni<Response> upload(@MultipartForm MultipartBody multipartBody) {
    Uni<String> objName = someService.doSomething(multipartBody);

    return objName.onItem()
          .transform(name -> URI.create("/resource/" + name))
        .onItem()
          .transform(Response::created)
        .onItem()
          .transform(Response.ResponseBuilder::build);
  }

More help could be found here: https://quarkus.io/guides/getting-started-reactive#reactive-resource

Quarkus Vertx timeout in JWT autorization phase in POST request upload file

2 Answers2