I'm using spring batch to read a csv file and perform a feign call for every row of the file.
This is the core class for my Job:
package it.nexi.digitalreview.batch.config;
import it.nexi.digitalreview.batch.core.config.JobCompletionNotificationListener;
import it.nexi.digitalreview.batch.core.domain.job.NexiJobInterface;
import it.nexi.digitalreview.batch.manager.*;
import it.nexi.digitalreview.batch.manager.factory.BatchSetupFactory;
import it.nexi.digitalreview.batch.manager.factory.Setup;
import it.nexi.digitalreview.batch.model.DataInput;
import it.nexi.digitalreview.batch.model.DataOutput;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParametersIncrementer;
import org.springframework.batch.core.JobParametersValidator;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.core.step.tasklet.TaskletStep;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.FileSystemResource;
import org.springframework.stereotype.Component;
@Component
public class NexiJob implements NexiJobInterface {
@Autowired
private BatchSetupFactory<DataInput, DataOutput> batchSetupFactory;
@Autowired
private JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
public Job build(String templateId, ByteArrayResource inputResource, FileSystemResource failsResource) {
Setup<DataInput, DataOutput> setup = batchSetupFactory.getSetup(templateId);
if(setup == null) return null;
AbstractNotificationProcessor<DataInput, DataOutput> abstractNotificationProcessor = setup.getAbstractNotificationProcessor();
AbstractCsvNotificationMultiFormatWriter<DataOutput> abstractCsvNotificationMultiFormatWriter = setup.getAbstractCsvNotificationMultiFormatWriter();
AbstractCsvNotificationReader<DataInput> abstractCsvNotificationReader = setup.getAbstractCsvNotificationReader();
AbstractFileVerificationSkipper abstractFileVerificationSkipper = setup.getAbstractFileVerificationSkipper();
AbstractCsvNotificationWriter<DataOutput> abstractCsvNotificationWriter = setup.getAbstractCsvNotificationWriter();
JobCompletionNotificationListener jobCompletionNotificationListener = setup.getJobCompletionNotificationListener();
abstractCsvNotificationReader.setResource(inputResource);
abstractCsvNotificationWriter.setResource(failsResource);
TaskletStep csvFileToNotificatorStep = stepBuilderFactory.get("csvFileToNotificatorStep")
.<DataInput, DataOutput>chunk(10)
.reader(abstractCsvNotificationReader)
.faultTolerant()
.skipPolicy(abstractFileVerificationSkipper)
.processor(abstractNotificationProcessor)
.writer(abstractCsvNotificationMultiFormatWriter)
.build();
return jobBuilderFactory.get("csvFileToNotificatorJob")
.incrementer(new RunIdIncrementer())
.listener(jobCompletionNotificationListener)
.flow(csvFileToNotificatorStep)
.end()
.build();
}
@Override
public String getName() {
return this.getName();
}
@Override
public boolean isRestartable() {
return false;
}
@Override
public void execute(JobExecution execution) {
this.execute(execution);
}
@Override
public JobParametersIncrementer getJobParametersIncrementer() {
return this.getJobParametersIncrementer();
}
@Override
public JobParametersValidator getJobParametersValidator() {
return this.getJobParametersValidator();
}
public BatchSetupFactory<DataInput, DataOutput> getBatchSetupFactory() {
return batchSetupFactory;
}
}
What I'm noticing, is that at some time (after about 300 to 400k rows), the heap is no more freed from the gc. That's what I see about heap and cpu:
As you can see, the gc worked fine at a first time, just before 17:00, but then it started to degrade ending up filling the whole heap.
(there were 3 runs of the batch, this explain the heap profile between 17:30-18:00 and 19:15-20:00. In those sections the batch was idle, but still keeping objects in memory)
I performed an heap dump with the heap at maximum, and found out what's following:
It looks like jetty is causing the memory leak. I searched the web and out this was a common problem that's been solved with newer versions of jetty. I tried to update but nothing changed.
That's the dependency tree:
it.nexi.digitalreview.batch:send-notifications-from-csv:jar:2.0.9.RELEASE
+- org.springframework.boot:spring-boot-starter-jdbc:jar:1.5.3.RELEASE:compile
| +- org.springframework.boot:spring-boot-starter:jar:1.5.3.RELEASE:compile
| | +- org.springframework.boot:spring-boot:jar:1.5.3.RELEASE:compile
| | +- org.springframework.boot:spring-boot-autoconfigure:jar:1.5.3.RELEASE:compile
| | +- org.springframework.boot:spring-boot-starter-logging:jar:1.5.3.RELEASE:compile
| | | +- ch.qos.logback:logback-classic:jar:1.1.11:compile
| | | | \- ch.qos.logback:logback-core:jar:1.1.11:compile
| | | +- org.slf4j:jul-to-slf4j:jar:1.7.25:compile
| | | \- org.slf4j:log4j-over-slf4j:jar:1.7.25:compile
| | +- org.springframework:spring-core:jar:4.3.8.RELEASE:compile
| | \- org.yaml:snakeyaml:jar:1.17:compile
| +- org.apache.tomcat:tomcat-jdbc:jar:8.5.14:compile
| | \- org.apache.tomcat:tomcat-juli:jar:8.5.14:compile
| \- org.springframework:spring-jdbc:jar:4.3.7.RELEASE:compile
| +- org.springframework:spring-beans:jar:4.3.7.RELEASE:compile
| \- org.springframework:spring-tx:jar:4.3.7.RELEASE:compile
+- junit:junit:jar:4.12:compile
| \- org.hamcrest:hamcrest-core:jar:1.3:compile
+- it.icbpi.digitalreview.archcomps:nexi-core-pt:jar:1.1.2.RELEASE:compile
| +- it.icbpi.digitalreview.archcomps:nexi-core-pt-om:jar:1.1.2.RELEASE:compile
| | \- it.icbpi.digitalreview.archcomps:nexi-core-abs-om:jar:1.1.2.RELEASE:compile
| | \- org.projectlombok:lombok:jar:1.16.16:compile
| \- it.icbpi.digitalreview.archcomps:nexi-core-abs:jar:1.1.2.RELEASE:compile
| +- org.springframework.ws:spring-ws-support:jar:2.4.0.RELEASE:compile
| | +- org.springframework.ws:spring-ws-core:jar:2.4.0.RELEASE:compile
| | | \- org.springframework:spring-oxm:jar:4.3.7.RELEASE:compile
| | +- org.springframework.ws:spring-xml:jar:2.4.0.RELEASE:compile
| | \- org.springframework:spring-jms:jar:4.3.7.RELEASE:compile
| +- org.springframework.ws:spring-ws-security:jar:2.4.0.RELEASE:compile
| | +- org.apache.ws.security:wss4j:jar:1.6.19:compile
| | | +- org.apache.santuario:xmlsec:jar:1.5.8:compile
| | | \- org.opensaml:opensaml:jar:2.5.1-1:compile
| | | \- org.opensaml:openws:jar:1.4.2-1:compile
| | | \- org.opensaml:xmltooling:jar:1.3.2-1:compile
| | | \- joda-time:joda-time:jar:2.9.9:compile
| | +- org.apache.wss4j:wss4j-ws-security-dom:jar:2.1.4:compile
| | | +- org.apache.wss4j:wss4j-ws-security-common:jar:2.1.4:compile
| | | | +- org.opensaml:opensaml-saml-impl:jar:3.1.1:compile
| | | | | +- org.opensaml:opensaml-profile-api:jar:3.1.1:compile
| | | | | | \- org.opensaml:opensaml-core:jar:3.1.1:compile
| | | | | +- org.opensaml:opensaml-saml-api:jar:3.1.1:compile
| | | | | | +- org.opensaml:opensaml-xmlsec-api:jar:3.1.1:compile
| | | | | | \- org.opensaml:opensaml-soap-api:jar:3.1.1:compile
| | | | | +- org.opensaml:opensaml-security-impl:jar:3.1.1:compile
| | | | | | \- org.opensaml:opensaml-security-api:jar:3.1.1:compile
| | | | | | \- org.cryptacular:cryptacular:jar:1.0:compile
| | | | | +- org.opensaml:opensaml-xmlsec-impl:jar:3.1.1:compile
| | | | | \- net.shibboleth.utilities:java-support:jar:7.1.1:compile
| | | | +- org.opensaml:opensaml-xacml-impl:jar:3.1.1:compile
| | | | | \- org.opensaml:opensaml-xacml-api:jar:3.1.1:compile
| | | | +- org.opensaml:opensaml-xacml-saml-impl:jar:3.1.1:compile
| | | | | \- org.opensaml:opensaml-xacml-saml-api:jar:3.1.1:compile
| | | | +- org.jasypt:jasypt:jar:1.9.2:compile
| | | | \- org.apache.geronimo.specs:geronimo-javamail_1.4_spec:jar:1.7.1:compile
| | | \- net.sf.ehcache:ehcache:jar:2.10.4:runtime
| | \- org.springframework.security:spring-security-core:jar:4.1.4.RELEASE:compile
| | \- aopalliance:aopalliance:jar:1.0:compile
| +- org.hibernate.javax.persistence:hibernate-jpa-2.1-api:jar:1.0.0.Final:compile
| +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
| +- com.netflix.hystrix:hystrix-core:jar:1.5.6:compile
| | \- org.hdrhistogram:HdrHistogram:jar:2.1.9:compile
| +- com.netflix.feign:feign-core:jar:8.18.0:compile
| +- com.hazelcast:hazelcast-spring:jar:3.7.7:compile
| | \- com.hazelcast:hazelcast:jar:3.7.7:compile
| +- com.hazelcast:hazelcast-client:jar:3.7.7:compile
| +- org.springframework.cloud:spring-cloud-starter-sleuth:jar:1.1.3.RELEASE:compile
| | +- org.springframework.boot:spring-boot-starter-actuator:jar:1.5.3.RELEASE:compile
| | | \- org.springframework.boot:spring-boot-actuator:jar:1.5.3.RELEASE:compile
| | \- org.springframework.cloud:spring-cloud-sleuth-core:jar:1.1.3.RELEASE:compile
| | \- org.aspectj:aspectjrt:jar:1.8.10:compile
| +- commons-io:commons-io:jar:2.4:compile
| +- org.jolokia:jolokia-core:jar:1.3.6:compile
| | \- com.googlecode.json-simple:json-simple:jar:1.1.1:compile
| +- io.springfox:springfox-swagger2:jar:2.6.1:compile
| | +- io.swagger:swagger-annotations:jar:1.5.10:compile
| | +- io.swagger:swagger-models:jar:1.5.10:compile
| | +- io.springfox:springfox-spi:jar:2.6.1:compile
| | | \- io.springfox:springfox-core:jar:2.6.1:compile
| | +- io.springfox:springfox-schema:jar:2.6.1:compile
| | +- io.springfox:springfox-swagger-common:jar:2.6.1:compile
| | +- io.springfox:springfox-spring-web:jar:2.6.1:compile
| | +- org.springframework.plugin:spring-plugin-core:jar:1.2.0.RELEASE:compile
| | +- org.springframework.plugin:spring-plugin-metadata:jar:1.2.0.RELEASE:compile
| | \- org.mapstruct:mapstruct:jar:1.0.0.Final:compile
| +- io.springfox:springfox-swagger-ui:jar:2.6.1:compile
| +- org.springframework.amqp:spring-rabbit:jar:1.7.2.RELEASE:compile
| | +- org.springframework:spring-messaging:jar:4.3.7.RELEASE:compile
| | +- com.rabbitmq:http-client:jar:1.1.1.RELEASE:compile
| | +- com.rabbitmq:amqp-client:jar:4.0.2:compile
| | +- org.springframework.retry:spring-retry:jar:1.2.0.RELEASE:compile
| | \- org.springframework.amqp:spring-amqp:jar:1.7.2.RELEASE:compile
| +- com.h2database:h2:jar:1.4.195:compile
| +- org.springframework.session:spring-session:jar:1.3.0.RELEASE:compile
| +- org.springframework.security.oauth:spring-security-oauth2:jar:2.0.13.RELEASE:compile
| | +- org.springframework.security:spring-security-config:jar:4.1.4.RELEASE:compile
| | +- org.springframework.security:spring-security-web:jar:4.1.4.RELEASE:compile
| | \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
| +- org.springframework.cloud:spring-cloud-starter-hystrix-dashboard:jar:1.2.6.RELEASE:compile
| +- org.springframework.cloud:spring-cloud-starter-hystrix:jar:1.2.6.RELEASE:compile
| | +- com.netflix.hystrix:hystrix-metrics-event-stream:jar:1.5.6:compile
| | | \- com.netflix.hystrix:hystrix-serialization:jar:1.5.6:runtime
| | | +- com.fasterxml.jackson.module:jackson-module-afterburner:jar:2.8.8:runtime
| | | \- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.8.7:runtime
| | \- com.netflix.hystrix:hystrix-javanica:jar:1.5.6:compile
| +- org.springframework.cloud:spring-cloud-netflix-hystrix-dashboard:jar:1.2.6.RELEASE:compile
| | +- org.springframework.boot:spring-boot-starter-freemarker:jar:1.5.3.RELEASE:compile
| | | +- org.freemarker:freemarker:jar:2.3.26-incubating:compile
| | | \- org.springframework:spring-context-support:jar:4.3.7.RELEASE:compile
| | +- org.webjars:jquery:jar:2.1.1:compile
| | \- org.webjars:d3js:jar:3.4.11:compile
| \- it.icbpi.digitalreview.archcomps:nexi-core-abs-jwt-validator:jar:1.1.2.RELEASE:compile
| \- it.nexi:plugin-api-gateway:jar:1.0.0-RELEASE:compile
| +- org.springframework.cloud:spring-cloud-starter-netflix-zuul:jar:1.4.4.RELEASE:compile
| | +- org.springframework.cloud:spring-cloud-starter-netflix-hystrix:jar:1.4.4.RELEASE:compile
| | +- org.springframework.cloud:spring-cloud-starter-netflix-ribbon:jar:1.4.4.RELEASE:compile
| | +- org.springframework.cloud:spring-cloud-starter-netflix-archaius:jar:1.4.4.RELEASE:compile
| | \- com.netflix.zuul:zuul-core:jar:1.3.0:compile
| +- org.springframework.cloud:spring-cloud-starter-config:jar:1.2.3.RELEASE:compile
| | \- org.springframework.cloud:spring-cloud-config-client:jar:1.2.3.RELEASE:compile
| +- org.springframework.cloud:spring-cloud-starter-netflix-eureka-client:jar:1.4.4.RELEASE:compile
| | +- org.springframework.cloud:spring-cloud-netflix-eureka-client:jar:1.2.6.RELEASE:compile
| | +- com.netflix.eureka:eureka-client:jar:1.4.12:compile
| | | +- com.netflix.netflix-commons:netflix-eventbus:jar:0.3.0:runtime
| | | | +- com.netflix.netflix-commons:netflix-infix:jar:0.3.0:runtime
| | | | | +- commons-jxpath:commons-jxpath:jar:1.3:runtime
| | | | | +- org.antlr:antlr-runtime:jar:3.4:runtime
| | | | | | \- org.antlr:stringtemplate:jar:3.2.1:runtime
| | | | | \- com.google.code.gson:gson:jar:2.8.0:runtime
| | | | \- org.apache.commons:commons-math:jar:2.2:runtime
| | | +- com.google.inject:guice:jar:4.0:runtime
| | | \- com.netflix.governator:governator-api:jar:1.12.10:runtime
| | +- com.netflix.eureka:eureka-core:jar:1.4.12:compile
| | | +- com.netflix.governator:governator:jar:1.12.10:runtime
| | | | \- com.netflix.governator:governator-core:jar:1.12.10:runtime
| | | | +- com.google.inject.extensions:guice-multibindings:jar:4.0:runtime
| | | | \- com.google.inject.extensions:guice-grapher:jar:4.0:runtime
| | | | \- com.google.inject.extensions:guice-assistedinject:jar:4.0:runtime
| | | \- org.codehaus.woodstox:woodstox-core-asl:jar:4.4.1:runtime
| | | +- javax.xml.stream:stax-api:jar:1.0-2:runtime
| | | \- org.codehaus.woodstox:stax2-api:jar:3.1.4:runtime
| | \- com.netflix.ribbon:ribbon-eureka:jar:2.2.0:compile
| +- io.jsonwebtoken:jjwt:jar:0.9.1:compile
| \- org.springframework.cloud:spring-cloud-starter-bus-amqp:jar:1.2.2.RELEASE:compile
| +- org.springframework.cloud:spring-cloud-starter-stream-rabbit:jar:1.1.3.RELEASE:compile
| | \- org.springframework.cloud:spring-cloud-stream-binder-rabbit:jar:1.1.3.RELEASE:compile
| | +- org.springframework.cloud:spring-cloud-stream-codec:jar:1.1.2.RELEASE:compile
| | | \- com.esotericsoftware:kryo-shaded:jar:3.0.3:compile
| | | +- com.esotericsoftware:minlog:jar:1.3.0:compile
| | | \- org.objenesis:objenesis:jar:2.1:compile
| | +- org.springframework.boot:spring-boot-starter-amqp:jar:1.5.3.RELEASE:compile
| | \- org.springframework.integration:spring-integration-amqp:jar:4.3.8.RELEASE:compile
| \- org.springframework.cloud:spring-cloud-bus:jar:1.2.2.RELEASE:compile
| +- org.springframework.cloud:spring-cloud-stream:jar:1.1.2.RELEASE:compile
| | +- org.springframework.boot:spring-boot-starter-validation:jar:1.5.3.RELEASE:compile
| | | \- org.apache.tomcat.embed:tomcat-embed-el:jar:8.5.14:compile
| | +- org.springframework.integration:spring-integration-jmx:jar:4.3.8.RELEASE:compile
| | +- org.springframework:spring-tuple:jar:1.0.0.RELEASE:compile
| | \- org.springframework.integration:spring-integration-tuple:jar:1.0.0.RELEASE:compile
| \- org.springframework.integration:spring-integration-core:jar:4.3.8.RELEASE:compile
+- org.springframework.cloud:spring-cloud-starter-feign:jar:1.2.6.RELEASE:compile
| +- org.springframework.cloud:spring-cloud-starter:jar:1.1.8.RELEASE:compile
| | +- org.springframework.cloud:spring-cloud-context:jar:1.1.8.RELEASE:compile
| | \- org.springframework.security:spring-security-rsa:jar:1.0.3.RELEASE:compile
| | \- org.bouncycastle:bcpkix-jdk15on:jar:1.55:compile
| | \- org.bouncycastle:bcprov-jdk15on:jar:1.55:compile
| +- org.springframework.cloud:spring-cloud-netflix-core:jar:1.2.6.RELEASE:compile
| +- org.springframework:spring-web:jar:4.3.7.RELEASE:compile
| | +- org.springframework:spring-aop:jar:4.3.7.RELEASE:compile
| | \- org.springframework:spring-context:jar:4.3.7.RELEASE:compile
| +- org.springframework.cloud:spring-cloud-commons:jar:1.1.8.RELEASE:compile
| | \- org.springframework.security:spring-security-crypto:jar:4.1.4.RELEASE:compile
| +- io.github.openfeign:feign-core:jar:9.3.1:compile
| | \- org.jvnet:animal-sniffer-annotation:jar:1.0:compile
| +- io.github.openfeign:feign-slf4j:jar:9.3.1:compile
| | \- org.slf4j:slf4j-api:jar:1.7.25:compile
| +- io.github.openfeign:feign-hystrix:jar:9.3.1:compile
| +- org.springframework.cloud:spring-cloud-starter-ribbon:jar:1.2.6.RELEASE:compile
| | +- com.netflix.ribbon:ribbon:jar:2.2.0:compile
| | | +- com.netflix.ribbon:ribbon-transport:jar:2.2.0:runtime
| | | | +- io.reactivex:rxnetty-contexts:jar:0.4.9:runtime
| | | | \- io.reactivex:rxnetty-servo:jar:0.4.9:runtime
| | | +- javax.inject:javax.inject:jar:1:runtime
| | | \- io.reactivex:rxnetty:jar:0.4.9:runtime
| | | +- io.netty:netty-codec-http:jar:4.0.27.Final:runtime
| | | | +- io.netty:netty-codec:jar:4.0.27.Final:runtime
| | | | \- io.netty:netty-handler:jar:4.0.27.Final:runtime
| | | \- io.netty:netty-transport-native-epoll:jar:4.0.27.Final:runtime
| | | +- io.netty:netty-common:jar:4.0.27.Final:runtime
| | | +- io.netty:netty-buffer:jar:4.0.27.Final:runtime
| | | \- io.netty:netty-transport:jar:4.0.27.Final:runtime
| | +- com.netflix.ribbon:ribbon-core:jar:2.2.0:compile
| | | \- commons-lang:commons-lang:jar:2.6:compile
| | +- com.netflix.ribbon:ribbon-httpclient:jar:2.2.0:compile
| | | +- commons-collections:commons-collections:jar:3.2.2:runtime
| | | +- com.sun.jersey:jersey-client:jar:1.19.1:runtime
| | | | \- com.sun.jersey:jersey-core:jar:1.19.1:runtime
| | | | \- javax.ws.rs:jsr311-api:jar:1.1.1:runtime
| | | +- com.sun.jersey.contribs:jersey-apache-client4:jar:1.19.1:runtime
| | | +- com.netflix.servo:servo-core:jar:0.10.1:runtime
| | | | \- com.netflix.servo:servo-internal:jar:0.10.1:runtime
| | | \- com.netflix.netflix-commons:netflix-commons-util:jar:0.1.1:runtime
| | +- com.netflix.ribbon:ribbon-loadbalancer:jar:2.2.0:compile
| | | \- com.netflix.netflix-commons:netflix-statistics:jar:0.1.1:runtime
| | \- io.reactivex:rxjava:jar:1.1.10:compile
| \- org.springframework.cloud:spring-cloud-starter-archaius:jar:1.2.6.RELEASE:compile
| +- com.netflix.archaius:archaius-core:jar:0.7.4:compile
| | \- com.google.code.findbugs:jsr305:jar:3.0.1:runtime
| +- commons-configuration:commons-configuration:jar:1.8:compile
| +- com.fasterxml.jackson.core:jackson-annotations:jar:2.8.7:compile
| +- com.fasterxml.jackson.core:jackson-core:jar:2.8.7:compile
| \- com.google.guava:guava:jar:18.0:compile
+- it.nexi.digitalreview.batch:core:jar:2.0.0.RELEASE:compile
| +- com.auth0:java-jwt:jar:3.4.0:compile
| +- it.nexi.digitalreview.batch:rest-notificator:jar:2.0.0.RELEASE:compile
| | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
| | \- it.icbpi.digitalreview.common:notificator-domain:jar:8.0.15.RELEASE:compile
| | \- it.icbpi.digitalreview.common:rest-firebase:jar:8.0.15.RELEASE:compile
| \- com.jcraft:jsch:jar:0.1.50:compile
+- org.springframework.boot:spring-boot-starter-web:jar:1.5.3.RELEASE:compile
| +- org.hibernate:hibernate-validator:jar:5.3.5.Final:compile
| | +- javax.validation:validation-api:jar:1.1.0.Final:compile
| | +- org.jboss.logging:jboss-logging:jar:3.3.1.Final:compile
| | \- com.fasterxml:classmate:jar:1.3.3:compile
| +- com.fasterxml.jackson.core:jackson-databind:jar:2.8.7:compile
| \- org.springframework:spring-webmvc:jar:4.3.7.RELEASE:compile
| \- org.springframework:spring-expression:jar:4.3.7.RELEASE:compile
+- org.springframework.boot:spring-boot-starter-jetty:jar:2.3.0.RELEASE:compile
| +- jakarta.servlet:jakarta.servlet-api:jar:4.0.3:compile
| +- jakarta.websocket:jakarta.websocket-api:jar:1.1.2:compile
| +- org.eclipse.jetty:jetty-servlets:jar:9.4.4.v20170414:compile
| | +- org.eclipse.jetty:jetty-continuation:jar:9.4.4.v20170414:compile
| | +- org.eclipse.jetty:jetty-http:jar:9.4.4.v20170414:compile
| | +- org.eclipse.jetty:jetty-util:jar:9.4.4.v20170414:compile
| | \- org.eclipse.jetty:jetty-io:jar:9.4.4.v20170414:compile
| +- org.eclipse.jetty:jetty-webapp:jar:9.4.4.v20170414:compile
| | +- org.eclipse.jetty:jetty-xml:jar:9.4.4.v20170414:compile
| | \- org.eclipse.jetty:jetty-servlet:jar:9.4.4.v20170414:compile
| | \- org.eclipse.jetty:jetty-security:jar:9.4.4.v20170414:compile
| | \- org.eclipse.jetty:jetty-server:jar:9.4.4.v20170414:compile
| +- org.eclipse.jetty.websocket:websocket-server:jar:9.4.4.v20170414:compile
| | +- org.eclipse.jetty.websocket:websocket-common:jar:9.4.4.v20170414:compile
| | | \- org.eclipse.jetty.websocket:websocket-api:jar:9.4.4.v20170414:compile
| | +- org.eclipse.jetty.websocket:websocket-client:jar:9.4.4.v20170414:compile
| | | \- org.eclipse.jetty:jetty-client:jar:9.4.4.v20170414:compile
| | \- org.eclipse.jetty.websocket:websocket-servlet:jar:9.4.4.v20170414:compile
| +- org.eclipse.jetty.websocket:javax-websocket-server-impl:jar:9.4.4.v20170414:compile
| | +- org.eclipse.jetty:jetty-annotations:jar:9.4.4.v20170414:compile
| | | +- org.eclipse.jetty:jetty-plus:jar:9.4.4.v20170414:compile
| | | | \- org.eclipse.jetty:jetty-jndi:jar:9.4.4.v20170414:compile
| | | +- org.ow2.asm:asm:jar:5.1:compile
| | | \- org.ow2.asm:asm-commons:jar:5.1:compile
| | | \- org.ow2.asm:asm-tree:jar:5.1:compile
| | \- org.eclipse.jetty.websocket:javax-websocket-client-impl:jar:9.4.4.v20170414:compile
| \- org.glassfish:jakarta.el:jar:3.0.3:compile
+- org.springframework.boot:spring-boot-starter-batch:jar:1.5.3.RELEASE:compile
| \- org.springframework.batch:spring-batch-core:jar:3.0.7.RELEASE:compile
| +- com.ibm.jbatch:com.ibm.jbatch-tck-spi:jar:1.0:compile
| | \- javax.batch:javax.batch-api:jar:1.0:compile
| +- com.thoughtworks.xstream:xstream:jar:1.4.9:compile
| | +- xmlpull:xmlpull:jar:1.1.3.1:compile
| | \- xpp3:xpp3_min:jar:1.1.4c:compile
| +- org.codehaus.jettison:jettison:jar:1.2:compile
| \- org.springframework.batch:spring-batch-infrastructure:jar:3.0.7.RELEASE:compile
+- org.springframework.boot:spring-boot-starter-data-jpa:jar:1.5.3.RELEASE:compile
| +- org.springframework.boot:spring-boot-starter-aop:jar:1.5.3.RELEASE:compile
| | \- org.aspectj:aspectjweaver:jar:1.8.10:compile
| +- org.hibernate:hibernate-core:jar:5.0.12.Final:compile
| | +- org.javassist:javassist:jar:3.21.0-GA:compile
| | +- antlr:antlr:jar:2.7.7:compile
| | +- org.jboss:jandex:jar:2.0.0.Final:compile
| | +- dom4j:dom4j:jar:1.6.1:compile
| | \- org.hibernate.common:hibernate-commons-annotations:jar:5.0.1.Final:compile
| +- org.hibernate:hibernate-entitymanager:jar:5.0.12.Final:compile
| +- javax.transaction:javax.transaction-api:jar:1.2:compile
| +- org.springframework.data:spring-data-jpa:jar:1.10.8.RELEASE:compile
| | +- org.springframework.data:spring-data-commons:jar:1.12.8.RELEASE:compile
| | +- org.springframework:spring-orm:jar:4.3.7.RELEASE:compile
| | \- org.slf4j:jcl-over-slf4j:jar:1.7.25:compile
| \- org.springframework:spring-aspects:jar:4.3.7.RELEASE:compile
+- org.hsqldb:hsqldb:jar:2.3.3:compile
+- com.oracle:ojdbc6:jar:11.2.0.4:compile
+- org.springframework.boot:spring-boot-configuration-processor:jar:1.5.3.RELEASE:compile
| \- com.vaadin.external.google:android-json:jar:0.0.20131108.vaadin1:compile
+- org.apache.commons:commons-lang3:jar:3.5:compile
\- org.apache.httpcomponents:httpclient:jar:4.5.3:compile
+- org.apache.httpcomponents:httpcore:jar:4.4.6:compile
\- commons-codec:commons-codec:jar:1.10:compile
any clue on how to solve my problem?
------UPDATE------
I think the problem belongs to my ItemPrecessor. Here a snippet of wahat it does:
public class ICNotificationProcessor<I,O> extends AbstractNotificationProcessor<NotificationDataModel,NotificationDataModelWithState> {
private static final Logger logger = LoggerFactory.getLogger(ICNotificationProcessor.class);
private CustomNotificatorServiceClient customNotificatorServiceClient;
private Tracer tracer;
private String serviceId;
public ICNotificationProcessor(CustomNotificatorServiceClient customNotificatorServiceClient, Tracer tracer, String serviceId){
this.customNotificatorServiceClient = customNotificatorServiceClient;
this.tracer = tracer;
this.serviceId = serviceId;
}
@Override
public NotificationDataModelWithState process(final NotificationDataModel notificationDataModel) {
Span span = tracer.createSpan(notificationDataModel.getIdAsString());
long traceId = span.getTraceId();
long spanId = span.getSpanId();
MDC.put(NexiBaseServiceConst.Mdc.TRACE_ID, String.valueOf(traceId));
MDC.put(NexiBaseServiceConst.Mdc.SPAN_ID, String.valueOf(spanId));
//perform feign call with traceId and spanId in request
tracer.close(span);
return notificationDataModelWithState;
}
}
Looks like that's not the correct way to handle spans.
