1

As a "branch" question from my other question (see Edit 4) How to convert a png base64 string to pixel array without using canvas getImageData?

I wonder what happens when you have a set of functions like this:

function convertTileToRGB(tile,whatToDoNext){
    tile.img.onload = function() {
        //async code returns pixels
        whatToDoNext(pixels);
    }

}

function doSomethingWithTile(tile,params){
    convertTileToRGB(tile,function(pixels){
        //manipulate pixels in tile
    });
}

function doSomethingWithTile2(tile,params){
    convertTileToRGB(tile,function(pixels){
        //manipulate pixels in a different way
    });
}

Is there a way a arbitrary call sequence like this, 

var tile = initialize_tile();

doSomethingWithTile(tile,params1)
doSomethingWithTile2(tile,params2)
doSomethingWithTile(tile,params3)  
...

that is dynamic and not known in advance, in other words added asynchronously to a dispatcher while the program is running and then the dispatcher is invoking them synchronously, to be solved somehow?

If we don't have a dispatcher and we use callbacks we can see that the methods act separately on the initialized tile but their actions are not accumulated to the tile. In that example order is preserved but essentially the result of the previous async method is not used in the next one.

Thanks in advance for any help

EDIT: thanks for the answers. yeah order matters but this list of things is being changed throughout the time(more things are added) and is not known before hand. It would be nice if there was a way to have a dispatcher object that will have what is to be done in a specific order being added asynchronously but it will invoke them in that order synchronously. I changed my question accordingly because most of the people were confused.

EDIT2: Trying to change my code: The dispatcher would look like below. The tile and sample could be different for each invocation but could be the same for some. Other actions will be applied except for addSample. If the same tile is used the invocations changes should be accumulated in the tile in that order. And most importantly the dispatcher should be able to get more to do actions asynchronously.

var sample = [[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f],[0x7f,0x7f,0x7f]]
var tile = initialize_tile({width: 7,height: 7,defaultColor: [0x00,0xAA,0x00], x: 1, y: 2, zoomLevel: 1});
var dispatcher = { 
    0:{
        func: addSample, 
        tile: tile,
        sample: sample, 
        num_samples: 6, 
        which_sample: 1
    },
    1:{
        func: addSample, 
        tile: tile,
        sample: sample, 
        num_samples: 6, 
        which_sample: 2
    },
       2:{
               func: render,
               tile: tile,
         }
};
var disp = dispatcher[0];
disp.func(disp.tile,disp.sample,disp.num_samples,disp.which_sample);    
disp = dispatcher[1];
disp.func(disp.tile,disp.sample,disp.num_samples,disp.which_sample);

With that code the order is not preserved and what happens to the tile is not kept for the next invocation.

Community
  • 1
  • 1
Michail Michailidis
  • 11,792
  • 6
  • 63
  • 106

6 Answers6

2

If you want to be able to chain the doSomething calls in a defined order, the only way is to allow doSomething to signal when its complete, similarly to how convertTileToRGB does it:

function doSomethingWithTile(tile,params, onDone){
    convertTileToRGB(tile,function(pixels){
        //manipulate pixels in tile
        onDone();
    });
}

In the end your code would look like

doSomethingWithTile(tile,params1, function(){
    doSomethingWithTile2(tile,params2, function(){
        doSomethingWithTile(tile,params3, function(){
           //...
        });
    });
});

Now the code is guaranteed to run in the correct order. The "pyramid of doom" pattern is a bit ugly though so you might want to find a way around it, either by using named functions instead of anonymous functions for the callbacks or by using a "sequencer" library function:

 //There are lots of async control flow libraries available for Javascript.
 // All of them should allow you to write code sort of like this one:
sequencer([
    function(onDone){ doSomethingWithTile(tile,params1, onDone) },
    function(onDone){ doSomethingWithTile(tile,params2, onDone) },
    function(onDone){ doSomethingWithTile(tile,params3, onDone) }
], function(){
    //...
});

You can also do things more dynamically:

function render_stuff(argss, onDone){

    var i = 0;
    function loop(){
        if(i < argss.length){
            var args = argss[i];
            addSample(args.tile, args.sample, args.num_samples, args.which_sample, function(){
               i = i + 1;
               loop();
            });
        }else{
            render(tile, onDone);
        }
    }

    loop();
}

render_stuff([
   {tile:tile, sample:sample, num_samples:6, which_sample:1},
   {tile:tile, sample:sample, num_samples:6, which_sample:2}
], function(){
    console.log("done rendering");
 });

if these functions were not async this would be equivalent to some code like

function render_stuff(argss){
    i = 0;
    while(i < argss.length){
      var args = argss[i];
      addSample(/**/);
      i = i + 1;
    }
    render(tile);
}

render_stuff([
    {/*...*/}, 
    {/*...*/}
])
console.log("done rendering");
BenMorel
  • 34,448
  • 50
  • 182
  • 322
hugomg
  • 68,213
  • 24
  • 160
  • 246
  • yeah but I said that the sequence is arbitrary and dynamic - I don't have it before hand – Michail Michailidis Jun 23 '13 at 20:06
  • @MichaelMichaelidis: What do you mean by "arbitrary and dynamic"? Do you still need one operation to run at a time or do you want to run all operations in "parallel" and collect the results in the end? I got the impression that you wanted things in a fixed order because you said that order matters. – hugomg Jun 23 '13 at 20:07
  • thanks for the answer. yeah order matters but this list of things is being changed throughout the time(more things are added) and is not known before hand. As I said as a comment below. It would be nice if there is a way to have a dispatcher data structure that will have what is to be done in a specific order being added asynchronously but it will invoke them in that order synchronously. – Michail Michailidis Jun 23 '13 at 20:17
  • @MichaelMichaelidis: See if the edit I made helps clear things up. The important thing is that you need to add callbacks to your async functinos if you want to be able to know when they are "done" and to convert any code that uses `for` and `while` loops into continuation-passing code that uses explicit recursion (you can think of it a bit as using gotos for looping) – hugomg Jun 23 '13 at 20:40
  • ok. but how do you pass the tile or the output of an async method to the next one? can you make it more specific using actual code from here (EDIT 4) http://stackoverflow.com/questions/17228724/how-to-convert-a-png-base64-string-to-pixel-array-without-using-canvas-getimaged Thank you – Michail Michailidis Jun 23 '13 at 20:53
  • @MichaelMichaelidis: It would be easier for me if you created a simplified example of how things would look like if the code were in synchronous style. For passing parameters the basic trick is that any variables of the outer function can be used (and assigned in the inner function. So any parameters or local variables from `dispatcher` can be used by `loop`. For "returning" values from the async events, the rule of thumb is to pass those values as arguments to the whenThingsDone callback like you did on convertToRGB. – hugomg Jun 23 '13 at 21:25
  • please see my Edit 2 as I am trying to make a simple dispatcher – Michail Michailidis Jun 23 '13 at 21:38
  • @MichaelMichaelidis: See if its better now. As I already said, this is just a matter of taking the code written in synchronous style and converting it to CPS style. (You should notice that you will have to convert addSample, rander and any other async functinos to receive an extra calback parameter so they can signal when they are done. – hugomg Jun 23 '13 at 22:00
  • hm it is not that easy because it is not just a bunch of addSamples and then render.. It could be sample render sample render render insert sample sample ... and all of these have different signature. Let me try some things and I will respond back - thanks – Michail Michailidis Jun 23 '13 at 22:27
1

The problem is that all calls to convertTileToRGB start acting on the reference to tile, before each manipulation has taken place.

If you want manipulations to be additive, you have to make each operation work on the results of the preceding one, but to make this you will have to wait the preceding has ended.

I don't know which operations you are actually performing, but it could also be possible to do those in an "incremental" non destroying way.

If this is not an option, to control series of callbacks mantaining code readable, I would suggest the use of some flow control library like async.js

Andrea Casaccia
  • 4,802
  • 4
  • 29
  • 54
  • Thanks for the answer. I am basically changing or adding some color to some pixels of a tile - but it seems that even with async.js you should know what you are going to do before hand. – Michail Michailidis Jun 23 '13 at 20:09
1

What you can do is take the series of steps and organize that as a list:

var thingsToDo = [
  { params: p1, process: function(pixels, params) {
    // ... first function ...
  }},
  { params: p2, process: function(pixels, params) {
    // ... second function ...
  }},
  { params: p3, process: function(pixels, params) {
    // ... third function ...
  }}
];

Now you can create a function to carry out a sequence of conversions:

function sequence( tile, thingsToDo ) {
  function doOneThing( ) {
    var thing = thingsToDo[thingCount];

    convertToRgb(tile, function(pixels) {
      thing.process(pixels, thing.params);
      thingCount ++;
      if (thingCount < thingsToDo.length)
         doOneThing();
    });
  }

  var thingCount = 0;
  doOneThing();
}

When you call that, it'll iterate through your "things", doing one at a time. I've added a "params" parameter to your callbacks; it wasn't clear exactly how those parameter objects figured into the situation in your code. Thus each "thing" is a block of parameters and a callback function.

So that "sequence" function would be called like this:

sequence(tile, thingsToDo);
Pointy
  • 405,095
  • 59
  • 585
  • 614
  • Thanks for the answer.I think you also assume that I have the list of my calls before hand. Which I don't. Except if there is a way to add asynchronously to the toDoThings and them being dispatched and invoked synchronously.. – Michail Michailidis Jun 23 '13 at 20:11
  • If you don't know beforehand what you want to do, then you have to sequence things dynamically through your callbacks. In other words, each callback will have to decide what happens and then call the next "convert". – Pointy Jun 23 '13 at 20:37
  • @MichaelMichaelidis alternatively, you could have a separate mechanism that maintains the queue of work items. The callback in my simple sequencer would use that service instead of the simple counter to find out whether there are pending requests. – Pointy Jun 23 '13 at 20:38
  • I agree with your second comment. For your first one, each async function can't decide what the next one will be, because that is decided from outside. It could be nothing for a while. – Michail Michailidis Jun 23 '13 at 20:55
  • @MichaelMichaelidis ok I wasn't sure exactly what you meant at first. – Pointy Jun 23 '13 at 21:03
0

If you are having trouble with asynchronous execution try using a semaphore to synchronize. http://en.wikipedia.org/wiki/Asynchronous_semaphore

Alternately block calls until the non-threadsafe action has completed. Additionally if the methods do not seem to accumulate, insure that the result of the previous method is the argument of whatever logic you are using to manipulate the image in the next methods, rather than the original image being the argument.

The question is pretty vague so I hope this helps.

awiebe
  • 3,758
  • 4
  • 22
  • 33
  • Typically in JS, if you somehow manage to block waiting on something, that something would never get done, because you're tying up the sole interpreter thread. – cHao Jun 23 '13 at 22:17
0

You could check out JavaScript Promises. Maybe they provide some useful methods to tackle your problem.

kfis
  • 4,739
  • 22
  • 19
  • Thanks! I have tried using promises but since you have to pass to then what it will be done as a callback (something that will be unknown when I run my program), I don't see how that will fix my problem – Michail Michailidis Jun 23 '13 at 20:23
0

Ok, so I made a dispatcher that you can add asynchronously more todo methods to be called with different signatures and there is a setInterval that will actually invoke them one by one. Thanks to https://stackoverflow.com/users/90511/missingno and https://stackoverflow.com/users/182668/pointy I was given the idea of having a sequence of methods to call. The problem was that I had to change all my functions to have the same signature and have a cronjob that will check if the dispatcher has more to invoke (Pointy had that but it wasn't clear in the first glance). If they don't have the same signature you can't really go from one another (I think). That's why I didn't select missingno's answer. I had to implement the dispatcher myself, that's why I am not accepting other partial answers. Here is the example of the conversion I did for mine. It was like that:

addSample = function(tile,sample,num_samples,which_sample){}
render = function(tile){}
...

and became:

addSample = function(config,dispatcher){
     var tile = config["tile"];
     var sample = config["sample"];
     var num_samples = config["num_samples"];
     var which_sample = config["which_sample"];

     something.call(function(){
        //async part of code
        dispatcher.check(); // important! 
     });
}
render = function(config,dispatcher){
     var tile = config["tile"];
     something.call(function(){
        //async part of code
        dispatcher.check(); // important! 
     });
}
...

If you don't call dispatcher.check() as the last line, then with each interval (see below) you will have only one TODO invocation. But we need all of them that are left. Then I have a dispatcher:

var dispatcher = { };
dispatcher.current = 0;
dispatcher.total = 0 ;
dispatcher.next = function(){
    this.current++;
}
dispatcher.hasMore = function(){
    return dispatcher.current<dispatcher.total;
}
dispatcher.addNew = function(todo){
    this.total++;
    this[this.total] = todo;
    return this;
}
dispatcher.exec = function(){
    this[this.current].func(this[this.current].config,this);    
    delete this[this.current]; //deletes the TODO just invoked
}

dispatcher.check = function(){
    if (this.hasMore()){
        dispatcher.next();
        dispatcher.exec();
    }
}

and then I add in the dispatcher more TODO jobs each has the func: funcName and config with all the inputs that the asynchronous function used to have.

dispatcher.addNew({func: addSample, config: {tile: tile,sample: sample, num_samples: 6, which_sample: 1}});
dispatcher.addNew({func: render,config: {tile: tile}});
dispatcher.addNew({func: addSample, config: {tile: tile,sample: sample2, num_samples: 6, which_sample: 2}});
dispatcher.addNew({func: render,config: {tile: tile}});
dispatcher.addNew({func: addSample, config: {tile: tile,sample: sample3, num_samples: 6, which_sample: 3}});
dispatcher.addNew({func: render,config: {tile: tile}});
dispatcher.addNew({func: addSample, config: {tile: tile,sample: sample4, num_samples: 6, which_sample: 4}});
dispatcher.addNew({func: render,config: {tile: tile}});
dispatcher.addNew({func: addSample, config: {tile: tile,sample: sample5, num_samples: 6, which_sample: 5}});

The interval checks and then invokes one by one, back to back all the TODO's left.If a group of TODOs come after a while the setInterval will pick the group all at once in that order.

setInterval(function(){dispatcher.check();},1000);

Thank you everyone for your time! I hope someone will find that useful as I did.

Community
  • 1
  • 1
Michail Michailidis
  • 11,792
  • 6
  • 63
  • 106