0

I built a small video frame analysis app with desktop Java 8. On each frame, I extract data (5 doubles now, but could expand to a 1920x1080x3 OpenCV Mat in the future). I would like to store this data into a database (Java DB, for example) to perform some time-series analysis, and periodically return the results to the user.

I am worried about hard-drive access times if I write to the database and run the app on a single thread, and the best solution that occured to me would be to implement the producer/consumer pattern with multithreading. The examples I found all implement 3 threads:

  1. the main thread
  2. the producer thread
  3. the consumer thread

Is there an advantage in doing that compared to a 2 thread implementation?

  1. main and producer thread
  2. consumer thread

And is that the right way to handle real-time data with a database?

Raoul
  • 1,872
  • 3
  • 26
  • 48

2 Answers2

4

It's limiting to use a fixed number of threads. My PC has (only) 8 cores, your intensive sounding app is not going to use half of them, indeed probably only the consumer is the intensive one, so maybe 12.5%. You'll have to have several of each thread to get the most out of the CPU, and then you'll spend a lot of effort managing threads.

The alternative is to use one of various existing systems for executing work in the background. For example ThreadPoolExecutor With that you can just throw lots of work at it (Runnables) and it will queue work up, and execution can be scaled to suit the hardware it's running on by customizing the number of worker threads.

Or if you're using Swing, then SwingWorker. The advantage of this is you can do some work on a background thread and post the results on the foreground (main/UI) thread easily.

weston
  • 54,145
  • 21
  • 145
  • 203
  • I find your answer enlightening. For my own understanding, are you talking about giving the OS a chance to balance the workload the best way, hence create many threads, but leave their distribution to good frameworks? I'm comparing to this discussion here: [http://stackoverflow.com/questions/9799708/assign-each-thread-a-cpu-core](http://stackoverflow.com/questions/9799708/assign-each-thread-a-cpu-core) – peter_the_oak Mar 06 '15 at 08:44
  • @peter_the_oak Well, you could make X threads, or you could get a thread pool to create X threads. The OS doesn't know the difference. The advantage of the `ThreadPoolExecuter` is it has been written to keep those threads busy processing work from a queue. Without making use of that you've got a lot of work to do, and it's tricky work too. – weston Mar 06 '15 at 08:55
1

Your question is rather conceptional, so I think it belongs here: Programmers

But as one short hint from my experience, you separate the producer from the main because your main control may freeze if something goes wrong with the producer. Things like frozen forms, not responding controls etc. may be the result. Give your system a chance to reestablish by command.

Community
  • 1
  • 1
peter_the_oak
  • 3,529
  • 3
  • 23
  • 37