2

I'm new to spring XD ...

I'm trying to create a job that will read a csv file and use spring-data-cassandra to insert data in a Cassandra database.

Everything is fine in my batch project. I can package my batch and deploy it to spring xd into "/module/job/myjob" (I called it myjob).

in "myjob" directory there is 2 folders : config (with myjob.xml) and lib (with all dependencies).

But when I try in spring xd shell to create my job : job create --definition myjob --name test1

I got this error :

Command failed org.springframework.xd.rest.client.impl.SpringXDException: 
Unexpected exception parsing XML document from URL 
file:/home/fmarchand/programs/spring-xd/spring-xd-1.0.0.RC1/xd/
modules/job/myjob/config/myjob.xml]; 
nested exception is java.lang.IllegalStateException: Unable to load schema 
mappings from location [META-INF/spring.schemas]

And in spring-xd logs :

Caused by: java.io.FileNotFoundException: /home/fmarchand/programs/spring-xd/spring-xd-1.0.0.RC1/xd/lib/spring-cql-1.0.1.RELEASE.jar (No such file or directory)

I tried to put spring-cql-1.0.1.RELEASE.jar in XD_HOME/lib (I don't like to put jars there but it asked me to do so) then the error changed :

 Command failed org.springframework.xd.rest.client.impl.SpringXDException: Unexpected 
 exception parsing XML document from URL [file:/home/fmarchand/programs/spring-xd/spring-xd-
 1.0.0.RC1/xd/modules/job/myjob/config/myjob.xml]; nested exception is 
 org.springframework.beans.FatalBeanException: Class 
 [org.springframework.data.cassandra.config.xml.CassandraNamespaceHandler] for namespace 
 [http://www.springframework.org/schema/data/cassandra] does not implement the 
 [org.springframework.beans.factory.xml.NamespaceHandler] interface

I missed something for sure !

Update : If I put myjob.xml in $XD_HOME/modules/job and jars in $XD_HOME/lib it works ...

Update 2 : now it works with /modules/job/myjob/config|lib. I did a cherry picking on all jars I had to put in the module library directory. I have another issue now which is a performance one. I will post another question for that.

Thx

Community
  • 1
  • 1
Franck MARCHAND
  • 41
  • 5

1 Answers1

2

This is likely because some dependency of spring-cql appears twice (once in myjob/lib and once in Spring XD itself). While we will likely have a better story for those scenarios in 1.x, one thing you can do is find inspiration in the build file of the PMML module. See how if computes the diff between the module dependencies and XD Dirt dependencies and only includes those that are necessary. There is not always a correct solution to complex classloading schemes like that, but this could help. Of course, you could also arrive at the same setup "by hand", by cherry picking the jars you need.

Also, your update comment feels weird. Whether the xml definition of the job is in job/myjob.xml or job/myjob/config/myjob.xml should have absolutely zero impact and behave the same. If you can share your exact setup, we'd be interested to look into it.

Hope this helps

ebottard
  • 1,997
  • 2
  • 12
  • 14