0

I have a Large-Scale Gradient Descent optimization problem that I am running using Matlab. The code has got two parts:

  1. A Sequential update part that fires every iteration that updates the parameter vector.
  2. A validation error computation part that fires every 10 iterations or so using the parameter value at the end of the corresponding iteration in which its fired.

The way that I am running this now is to do (1) and (2) sequentially. But (2) takes a lot of time and its not the core part of my routine - I made it just to check the progress and plot the error of my model. Is it possible in Matlab to run (2) in a parallel manner to (1) ? Please note that (1) cannot be run in parallel since it performs sequential update. So a simple 'parfor' usage is not a solution, unless there is a really smart way of doing that.

Swami
  • 71
  • 1
  • 10
  • do you know how many iterations in total your process (1) will run ? (or is conditioned to some kind of convergence ...) – Hoki Sep 27 '14 at 17:12

1 Answers1

0

I don't think Matlab has any way of multi-threading outside of the (rather restricted) parallel computing toolbox. There is a work over which may help you though:

Open 2 sessions of Matlab, sessions A and B (or instances, or workspaces, however you call it)

  • Matlab session A:
    • Calculate the 10 iterations of your sequential process (1)
    • Saves the result in a file (adequately and uniquely named)
    • Goes on to calculate the next 10 iterations (back to the top of this loop basically)

In parralel:

  • Matlab session B:
    • Check periodically for the existence of the file written by process A (define a timer that will do that at the time interval which make sense for your process, a few seconds or a few minutes ...)
    • If the file exist => load it then do the validation computation (your process (2)) and display/report the results.

note: This only works if process (1) doesn't need the result of process (2) to run its iterations, but if it is the case I don't know how you could parallelise anyway.

If you have multiple cores on your machine that should run smoothly, if you have a single core then the 2 sessions will have to share and you will see a performance impact.

Hoki
  • 11,637
  • 1
  • 24
  • 43
  • I think this is a smart way to do it. Thanks! As for your previous comment, It is indeed conditioned on convergence but right now I am setting an iteration limit and looking at how the function behaves for different parameter configurations. – Swami Sep 28 '14 at 17:17