I created a RNN program which creates an RNN model, trains it and predicts using the model using deeplearning4j. It runs with a good speed(30 minutes)but when it is added to the spark pipeline, it takes about 10 hours to run.Can you give me reason for that?
Asked
Active
Viewed 174 times
0
Thamali Wijewardhana
- 464
- 2
- 7
- 22
-
Show some relevant code part which you think may be problamatic – Ajay Pandya Mar 25 '16 at 06:15
-
The only change which I have done is adding the program to the Spark pipeline. – Thamali Wijewardhana Mar 25 '16 at 06:34
-
This is to broad. Are you running Spark in standalone, on Hadoop,.... If on Hadoop which scheduler etc. Based on your question the best tip is probably: https://cwiki.apache.org/confluence/display/SPARK/Profiling+Spark+Applications+Using+YourKit – CAFEBABE Mar 25 '16 at 09:52
-
Please join the community on Gitter: https://gitter.im/deeplearning4j/deeplearning4j – racknuf Apr 02 '16 at 23:22