0

Under normal circumstances, I can save a ComputationGraph (a Convolutional Neural Network) to a file and load it in a later run and it works fine.

However, when I include it in a jar and I try to load it, it fails.

Is there some way to load ComputationGraph objects from inside of a Jar as a resource?

I got it working for the Word2Vec objects, but it fails on ComputationGraph objects.

Thank you

Arthur Dunbar
  • 23
  • 5

1 Answers1

0

It's just a zip file (or should be!). I'm not sure if you're using our ModelSerializer (this is the only thing you should be using for computation graph or multi layer network) - you can use an InputStream or a File when you call:

ModelServer.restoreComputationGraph

on an InputStream. That InputStream could be a classpath resource input stream (using something like spring's ClassPathResource) or calling a YourClass.class.getResourceAsStream(..) - any of those should work fine for you.

Adam Gibson
  • 3,055
  • 1
  • 10
  • 12
  • I am using model serializer... it's possible I screwed something up with the resource loading. I'm using it as a file... it didn't cause any problems before, but I remember that being finicky now – Arthur Dunbar Jul 24 '18 at 02:57
  • There's nothing that should be "finnicky". Everything has a cause, I'm not sure what problems you're having, but it can't be that unique. Could you update your question with a stack trace so I can understand what an actual "cause" would be? Like I said, it's just a zip file. Loading a zip file from a jar file is a fairly standard operation in java. I also said you could extract it as well using ClassPathResource. – Adam Gibson Jul 24 '18 at 03:01
  • Turns out it didn't actually work with Word2Vec either. The error is something along the lines of not being able to find /path/to/jar!model.model or /path/to/jar!/model.model. I tried using the streams, but as it turns out ModelSerializer doesn't have a stream version of loading Word2Vec objects, and the deprecated one for WordVectors doesn't work for it either. – Arthur Dunbar Jul 24 '18 at 21:42
  • This still isn't useful. I need a real stack trace.I can't read your computer. Also, a few things there: ModelSerializer isn't even used with word2vec. I'm not sure where you got that idea to begin with. Beyond that, make sure your version of dl4j is up to date. There's zero reason why input streams shouldn't work. – Adam Gibson Jul 24 '18 at 23:42
  • I'm sorry you're right, not ModelSerializer WordVectorSerializer. There were the options for methods that I saw that returned Word2Vec. readWord2Vec(File file), readWord2VecModel(File file), readWord2VecModel(File file, boolean extendedModel), readWord2VecModel(String path), readWord2VecModel(String path, boolean extendedModel), loadFullModel(String path), readWord2VecFromText(File vectors, File hs, File h_codes, File h_points, VectorsConfiguration configuration) and fromPair(org.nd4j.linalg.primitives.Pair pair). None of which have InputStream as a type. – Arthur Dunbar Jul 25 '18 at 01:08
  • For InputStream as an argument I saw, WordVectors loadTxtVectors(InputStream stream, boolean skipFirstLine), ParagraphVectors readParagraphVectors(InputStream stream), ParagraphVectors readParagraphVectorsFromText(InputStream stream), and one that returned a String, a float, and a VocabCache object. I need a Word2Vec object. – Arthur Dunbar Jul 25 '18 at 01:11
  • I tried static WordVectors loadTxtVectors(InputStream stream, boolean skipFirstLine) (even though it was deprecated) because WordVectors is an Interface used by Word2Vec, but it didn't work. – Arthur Dunbar Jul 25 '18 at 01:17
  • My computer is at work, so I can't give the stack trace until tomorrow. – Arthur Dunbar Jul 25 '18 at 01:18
  • Thanks no problem! Just give me something I can look at so we can get your situation fixed. – Adam Gibson Jul 25 '18 at 02:54
  • So, I tried the stream thing for the ComputationGraph object and it worked, so that was the problem there. I could not find a stream version for Word2Vec objects. (I don't have a stack trace because the stream version worked for ComputationGraph, and I don't know what, if any, method to try for Word2Vec objects. I'm guessing it's not the File object one I was using though) – Arthur Dunbar Jul 25 '18 at 21:03
  • You're confusing 2 different things. Please decouple problems with word2vec from this question. If your computation graph worked, could you please mark this answer as accepted and raise a separate issue on word2vec? – Adam Gibson Jul 26 '18 at 01:46