0

I am trying to run a project written in Java and using Spring framework on Linux, for the purpose of extending some functionalities. The project is originally developed on Windows platform, however, I am trying to run (and continue working) on Ubuntu 15.10. The platform is Tomcat 8.0.30, PostgreSQL 9.4, JDK 1.8.0_73, Spring 4.2.0. Project is built under Spring Tool Suite, using Gradle 2.5. Everything is mostly under default configuration, except for necessary paths. The backend in java is a REST service, I haven't even started to go through the code yet.

The problem is, that the entire thing fails to run on this, particular, machine (I'll call it desktop). Build is successful, and everything is ok, until war is deployed on tomcat. Depending on the circumstances, on this machine, the deploy will produce a Spring circular reference, on an Oauth bean instantiation, or will pass normally, until the first request from the frontend, which is login, and it will produce a Stack overflow error, over and over (there are no other requests possible in the beginning).

This would not be so strange, if the exactly same code, with the exactly same setup on several different machines works without a glitch. What I have tried is:

  • Ubuntu 12.04, same java, older postgresql (9.1 I think) - works on first deploy
  • MacOS, latest java, latest postgresql - works on first deploy
  • Ubuntu 12.04 (other), same java, older postgresql - works on first deploy
  • Windows 7 x64, same setup, postgresql on linux - works on first deploy

To expand a bit, I recently upgraded the desktop, beforehand it was running ubuntu 14.10, with exactly the same effects, except I managed to actually run the project on the desktop by creating a new user, and running under that. Afterwards, it worked under my regular user, but only if run under STS, deployed on the external tomcat 8.0.30. If run under the Pivotal TC server that comes with the STS, I'd get the same errors. If manually deployed war to tomcat 8.0.30, again, errors. These "hacks" don't work anymore with the new version.

I also tried querying the project from the frontend running on other machines, and the result is the same, so it is purely something about Java and/or Tomcat. I also tried to dockerize tomcat (latest 8.0.32 tag, with openjdk-8, and also with 1.8.0_73), but with no luck. Transferring compiled war, which worked on a laptop, to a desktop - again gave no positive results.

This is a part of the log:

05-Mar-2016 02:21:32.868 SEVERE [http-nio-8080-exec-2] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [dispatcher] in context with path [/backend] threw exception [Filter execution threw an exception] with root cause
java.lang.StackOverflowError
at org.springframework.beans.factory.support.AbstractBeanFactory.transformedBeanName(AbstractBeanFactory.java:1084)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:238)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:196)
at org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:35)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:187)
at com.sun.proxy.$Proxy100.loadClientByClientId(Unknown Source)
at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy100.loadClientByClientId(Unknown Source)
at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy100.loadClientByClientId(Unknown Source)
at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy100.loadClientByClientId(Unknown Source)
at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy100.loadClientByClientId(Unknown Source)
at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy100.loadClientByClientId(Unknown Source)
at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy100.loadClientByClientId(Unknown Source)
at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy100.loadClientByClientId(Unknown Source)
at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy100.loadClientByClientId(Unknown Source)
at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)

... it goes on and on

The current status is: builds are clean, deploys are clean, always given the error from the log.

I am not familiar with Spring yet, so I cannot make out anything useful from the logs. I am, though, positive, that the services are installed correctly and are running as they should. The environments are as configured as close to defaults as possible to eliminate difference as the cause.

Any hints or inputs on what might be the cause (and solution) to this will be highly appreciated. I'll try to provide any information that might be useful as soon as possible.

Thank you!

Guy
  • 46,488
  • 10
  • 44
  • 88
user2750477
  • 91
  • 1
  • 9
  • Show us the loadClientByClientId method. – František Hartman Mar 05 '16 at 08:46
  • Also if you set logging to debug for org.springframework then have a look what is different when it works and when it doesn't.. – František Hartman Mar 05 '16 at 08:48
  • If the application is running smoothly on other OS then its most likely an issue with the memory allocation. Try to increase the JVM memory using -Xms and -Xmx options. – Ayman Mar 05 '16 at 09:01
  • Thank you all for your inputs, Ayman, I will try the memory increase as soon as I get to the desktop. frant.hartm, there is no trace of the mentioned method. I grepped the entire source, not one mention. I remember noticing it before in the logs, and not finding its source anywhere, so I assumed it must be a spring thing. – user2750477 Mar 05 '16 at 14:41
  • 1
    Found some hints in this [old issue](https://github.com/spring-projects/spring-security-oauth/issues/173) and [this question](http://stackoverflow.com/q/31798631/3080094). Also, source of Spring security oath [loadClientByClientId](https://github.com/spring-projects/spring-security-oauth/blob/master/spring-security-oauth2/src/main/java/org/springframework/security/oauth2/provider/ClientDetailsService.java#L34) – vanOekel Mar 05 '16 at 15:52
  • Finally got back. I am happy to report that the issue has been resolved after a few hours of tinkering. – user2750477 Mar 06 '16 at 21:46
  • The links you sent me were crucial, as they confirmed some ideas I got previously from a colleague. Can't imagine how I didn't come across them, as I was trying to solve this for weeks. Anyways, as soon as I got from the link it was a known issue, and basically a race condition, following a hint given to me earlier, I created singletons to instantiate clientDetailsService manually, along with a couple others that might be the issue. Works like a charm. I can't thank you enough. Really. – user2750477 Mar 06 '16 at 21:51
  • @user2750477 Glad I could help, please consider [answering your own question](http://blog.stackoverflow.com/2011/07/its-ok-to-ask-and-answer-your-own-questions/) with the answer describing the crucial information from the links and a small (code) example of what you did to resolve the problem. Also, if you use the "@user" in the comments, I receive a notification :-) – vanOekel Mar 07 '16 at 21:23

1 Answers1

2

Ok, so, very useful stuff in the comments, but to recap...

To resolve this, it was very useful for me to know that the issue is not code-related, but is a known (although rare) issue. To be precise, the order of the initialization was the culprit, and trying to access objects that are yet to be initialized (where proxy objects are then created).

My solution isn't stylish, and probably not by-the-book, but it did solve the issue for me. Instead of relying on @Autowired and proxies, I created singletons for each troublesome class, thus ensuring that the configuration is set properly, and the object is available at the very first request for it - from wherever it might come.

The links in the comment contain much more detailed explanation about the nature of the problem. Thank you all for your help!

PS. somewhere, while trying to find a solution to this problem (and following the links), I also encountered a post about "AnnotationConfigWebApplicationContext - Exception encountered during context initialization - cancelling refresh attempt org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'oauth2TokenController': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field:... it goes on.. Error creating bean with name 'might be solvable in the same manner, however, haven't tried.

user2750477
  • 91
  • 1
  • 9