2

I am using GraphicsMagick for node. I basically crop the photos and retrieve the exif data of the photos uploaded by the user. I don't want to block the flow of request waiting for these task to be completed, therefore I need to use asynchronous functions to be able to do so. And I think I should be able to as these are I/O operations which node.js makes async itself.

But as I see all the functions in GraphicsMagick for node are synchronous functions. So I am not being able to sure as to how to achieve what I am looking for.

One idea that comes to my mind is to write a function with callback and have the GraphicsMagick processing done inside it. And use .nextTick() function to achieve asynchronous flow. But I am not totally sure if this is fine. And also are there any asynchronous functions for GraphicsMagick.

Please help me and an example code would be very appreciated as to how to get asynchronous functions from graphicsmagick.

Saransh Mohapatra
  • 9,430
  • 10
  • 39
  • 50

3 Answers3

6

UPDATE:
The actual answer from @Saransh Mohapatra is actually wrong. As after little investigation turned out that all methods that perform operations over images, actually do not perform anything but only append arguments to the list that will be used after when you write or call any buffer related methods executed in order to get/write actual image buffer.

Here is details over it in example of blur:

  1. We call blur: https://github.com/aheckmann/gm/blob/master/lib/args.js#L780
  2. It calls this.out which will call: https://github.com/aheckmann/gm/blob/master/lib/command.js#L49
  3. Which has method made for it when it was constructed: https://github.com/aheckmann/gm/blob/master/lib/command.js#L34
  4. Which all it does - a.push(arguments[i]); and then concats it to all list (to other arguments).
  5. Thats it.

Then when write is called:

  1. https://github.com/aheckmann/gm/blob/master/lib/command.js#L62
  2. It gets list of arguments self.args(): https://github.com/aheckmann/gm/blob/master/lib/command.js#L78
  3. Which just filters off some reserved fields: https://github.com/aheckmann/gm/blob/master/lib/command.js#L274
  4. So then those arguments will be joined in _spawn which is called from write: https://github.com/aheckmann/gm/blob/master/lib/command.js#L187
  5. Thats it.

So based on this, any method that makes operations over image, but do not save or persist buffer of it - do not need any async, as they actually do not do any work at all. So that means - you do need to worry about them.

OLD:
The best approach for any heavy processing stuff is to use separate processes.
You can create another small node.js process, that will have some communication abilities with main process (ZeroMQ is good choice here).

This separate process have to be notified about file (path) and what to do with it, you can easily send that data from main process which makes such decisions via ZeroMQ.

This approach will allow you to have independence in the way main (web?) node processes work, as well as possibility in the future to scale to separate hardware/instances.
It is very good practice as well (unix-like application logic separation).

moka
  • 22,846
  • 4
  • 51
  • 67
  • Is zeromq some kind of queuing service? If yes than I m not looking for that. I don't believe is that heavy processing. – Saransh Mohapatra Jul 19 '13 at 14:02
  • ZeroMQ is simple socket messaging system for internal use (it is not really Message Queue). The idea is not about queue the jobs, but about take them off the main process, in order to prevent heavy image processing block. Image processing, or any sort of content generation/rendering (like pdf, recompression, resizing) - is heavy process, and should be handled independently from web services. – moka Jul 19 '13 at 14:18
  • 1
    Ok. But still I don't understand what's the need of taking if off the main process using ZeroMQ while I can do it in node itself. Node.js has this built-in for I/O intensive work to be handled asynchronously. – Saransh Mohapatra Jul 19 '13 at 16:36
  • You have wrote that GraphicsMagic functions are synchronous. – moka Jul 19 '13 at 16:40
  • Yeah...thats what I want to know. How to convert a synchronous functions to asynchronous. – Saransh Mohapatra Jul 19 '13 at 18:56
  • It is not about sync vs async. Its about where actual function is getting executed. If it is executed within node thread, then you have to do C++ async wrapper. Or have separate process, like I've suggested previously. – moka Jul 19 '13 at 20:07
  • Have you even checked which module I am talking about? I doubt it coz you are asking me to write a async wrapper around C++ library. But thats exactly what this module is. So please if you have any real suggestion then say. Otherwise leave it. – Saransh Mohapatra Jul 21 '13 at 06:37
5

And here is how to promisify gm:

var Promise = require('bluebird');
var gm = require('gm').subClass({imageMagick: true});
Promise.promisifyAll(gm.prototype);

gm('1.jpg')
  .resize(240, 240)
  .noProfile()
  .writeAsync('1b.jpg')
  .then(function () {
    console.log('done');
  });
  .catch(function (err) {
    console.log(err);
  });

https://github.com/aheckmann/gm/issues/320

Wtower
  • 18,848
  • 11
  • 103
  • 80
2

Sorry my observation was mistaken, though the GraphicsMagick module seems as synchronous function but they are not. They spawn child process each time manipulation is done. And this has been confirmed here.

So anyone else looking for this problem, GraphicsMagick functions are Asynchronous. And you don't have to do anything from your part. Its a very good module and worth checking out.

Thanks.

Saransh Mohapatra
  • 9,430
  • 10
  • 39
  • 50
  • This answer is actually wrong, as after investigation in my answer, I've found that those operations do not do any work at all, but just add arguments to the list, which is after is used when `write`ing or getting buffer of image. So only at that last point - child process is spawned. – moka Jan 02 '14 at 16:03
  • Graphicsmagick uses imagemagick or whatever to actually work on the image so it has to fork a process and wait for the fork to respond a asynchronously, right? – orourkedd Feb 24 '15 at 07:03