We're using Azure Functions with Java worker handling incoming HTTP calls. Our functions were working just fine for more than 2 weeks properly handling incoming calls as presented on the screenshot, but for some reason they suddenly stopped recognizing the class having methods handling incoming HTTP triggers, resulting in ClassNotFoundException being transformed to 500 Server Error HTTP response. The client-side invoking the function made a number of retries according to its logic but didn't succeed and stopped trying after some time. Today, we manually restarted the client app calling the azure function via HTTP triggers without redeploying Azure functions or making any kind of configuration change to it and it started working again just fine.
We found following issue reported on GitHub of Azure/azure-functions-java-worker, which is currently open:
https://github.com/Azure/azure-functions-java-worker/issues/696.
We believe this might be exactly the same issue as we're facing, therefore we updated the issue asking for advise.
Screenshot of failed request count vs total request count
Our environment setup:
Runtime version: 4.22.0.20804
FUNCTIONS_EXTENSION_VERSION: ~4
OS: Windows
Location: North Europe
Trigger: HTTP request
Java: 17
Application logs from sample failure:
2023-07-08T17:39:14.6216938Z ERROR Result: Failure
Exception: ClassNotFoundException: XXX (Class with method handling HTTP Trigger)
Stack: java.lang.ClassNotFoundException: XXX (Class with method handling HTTP Trigger)
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at com.microsoft.azure.functions.worker.broker.FunctionDefinition.getContainingClass(FunctionDefinition.java:41)
at com.microsoft.azure.functions.worker.broker.FunctionDefinition.<init>(FunctionDefinition.java:28)
at com.microsoft.azure.functions.worker.broker.JavaFunctionBroker.loadMethod(JavaFunctionBroker.java:60)
at com.microsoft.azure.functions.worker.handler.FunctionLoadRequestHandler.execute(FunctionLoadRequestHandler.java:40)
at com.microsoft.azure.functions.worker.handler.FunctionLoadRequestHandler.execute(FunctionLoadRequestHandler.java:13)
at com.microsoft.azure.functions.worker.handler.MessageHandler.handle(MessageHandler.java:44)
at com.microsoft.azure.functions.worker.JavaWorkerClient$StreamingMessagePeer.lambda$onNext$0(JavaWorkerClient.java:94)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
2023-07-08T17:39:14.6217914Z ERROR Executed 'Functions.XXX' (Failed, Id=f92f258e-2a68-490e-a844-81fc0bbad2eb, Duration=4ms)
2023-07-08T17:39:14.6241065Z ERROR Result: Failure
Exception: ClassNotFoundException: XXX (Class with method handling HTTP Trigger)
Stack: java.lang.ClassNotFoundException: XXX (Class with method handling HTTP Trigger)
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at com.microsoft.azure.functions.worker.broker.FunctionDefinition.getContainingClass(FunctionDefinition.java:41)
at com.microsoft.azure.functions.worker.broker.FunctionDefinition.<init>(FunctionDefinition.java:28)
at com.microsoft.azure.functions.worker.broker.JavaFunctionBroker.loadMethod(JavaFunctionBroker.java:60)
at com.microsoft.azure.functions.worker.handler.FunctionLoadRequestHandler.execute(FunctionLoadRequestHandler.java:40)
at com.microsoft.azure.functions.worker.handler.FunctionLoadRequestHandler.execute(FunctionLoadRequestHandler.java:13)
at com.microsoft.azure.functions.worker.handler.MessageHandler.handle(MessageHandler.java:44)
at com.microsoft.azure.functions.worker.JavaWorkerClient$StreamingMessagePeer.lambda$onNext$0(JavaWorkerClient.java:94)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Azure exception call stack included in GitHub thread reply by amcsnoctuai, not including here as it would exceed the total allowed body length.
Unfortunately, the application affected by the issue is currently in production, hence we would much rather implement any kind of workaround hopefully preventing it from happening again, than wait for the GitHub issue to be resolved.
If any of you faced this kind of issue or know of possible workarounds, we would be grateful for advice.
