1

I'm trying to copy method discribed here on stackoverflow. But I'm having some problems which I don't know how to solve.

I set up jsfiddle to demonstrate everything. Here is the second jsfiddle with only particles moving and being drawn.

My problem lies in drawing, profiler showed that with around 10000 particles drawImage takes 40% of overall loop time. Without drawing directly and only calculations nothing hinders code exectuion so problem lies in drawing.


Is there a way how to use this technique without these side effects? Currently I show you how I create circle areas with arc but I also use png files for some other objects and they exhibit exactle the same behaviour.

problem: black overlapping area instead of transparent area, bottim circle's edge can be seen through the circle above

(problem: black overlapping area instead of transparent area, bottim circle's edge can be seen through the circle above)

I hope I expressed myself as clearly as possible (picture abovedisplays my problem very clearly) and I would like to thank you for your help.

Draw function - final draw to visible canvas.

Game.prototype.draw2 = function(interpolation, canvas, ctx, group)
{
    var canvasData = ctx.createImageData(canvas.width, canvas.height),
        cData = canvasData.data;

    for (var i = 0; i < group.length; i++)
    {
        var obj = group[i];

        if(!obj.draw)
        {
            continue;
        }

        var imagePixelData = obj.imagePixelData;

        var x = obj.previous.x + (obj.x - obj.previous.x) * interpolation;
        var y = obj.previous.y + (obj.y - obj.previous.y) * interpolation;

        for (var w = 0; w < obj.width; w++)
        {
            for (var h = 0; h < obj.height; h++)
            {
                if (x + w < canvas.width && obj.x + w > 0 &&
                    y + h > 0 && y + h < canvas.height)
                {
                    var iData = (h * obj.width + w) * 4;
                    var pData = (~~ (x + w) + ~~ (y + h) * canvas.width) * 4;

                    cData[pData] = imagePixelData[iData];
                    cData[pData + 1] = imagePixelData[iData + 1];
                    cData[pData + 2] = imagePixelData[iData + 2];
                    if (cData[pData + 3] < 100)
                    {
                        cData[pData + 3] = imagePixelData[iData + 3];
                    }

                }
            }
        }    
    }
    ctx.putImageData(canvasData, 0, 0);
};

And here is how I prepare pinkish circular area in other invisible canvas.

Game.prototype.constructors.Attractor.prototype.getImageData = function(context)
{
    this.separateScene = new context.constructors.Graphics(this.width, this.height, false);
    this.image = this.separateScene.canvas;
    this.separateScene.ctx.beginPath();
    this.separateScene.ctx.arc(this.radius, this.radius, this.radius, 0, 2 * Math.PI, false);
    this.separateScene.ctx.fillStyle = '#ff9b9b';
    this.separateScene.ctx.fill();
    this.separateScene.ctx.beginPath();
    this.separateScene.ctx.arc(this.radius, this.radius, this.radiusCut, 0, 2 * Math.PI, false);
    this.separateScene.ctx.fillStyle = 'rgba(255, 255, 255, 0.27)';
    this.separateScene.ctx.fill();
    this.separateScene.ctx.beginPath();
    this.separateScene.ctx.arc(this.radius, this.radius, this.coreRadius, 0, 2 * Math.PI, false);
    this.separateScene.ctx.fillStyle = '#ff64b2';
    this.separateScene.ctx.fill();
    this.imageData = this.separateScene.ctx.getImageData(0, 0, this.width, this.height);
    this.imagePixelData = this.imageData.data;
};
Community
  • 1
  • 1
Vico Lemp
  • 133
  • 12
  • Your question is too large for anyone to be able to answer it. We can't go through your 500 lines of code to see what you did wrong, and it's better to ask one question per post. The post you linked is good for when you have 10000+ objects to draw. Under 5000, `drawImage` is faster on most machines I have tested, so I'm not even sure you need it. The bigger your original images will be, the slower this solution will become. The `<100` is to check if the pixel is transparent or not. Above this threshold, we can consider that it's an anti-alias artifact and that we don't want it. – Kaiido Mar 22 '16 at 09:12
  • I removed extra information and kept only the main problem. Thank you. Btw I'm working with higher number of particles (arround 10k on screen). With chrome profiler I reduced the problem to drawImage() function which takes 40% of the overall time when running my entire code. My draw funcitons don't have 500 lines the rest is not really connected to the problem itself. – Vico Lemp Mar 22 '16 at 09:32
  • I have added one more fiddle where are only particles. I added a way to stop the loop and start it. The second fiddle shows the same problem with loaded png image which is drawn into a separate canvas first and imageData from this canvas are later used for drawing onto the visible canvas. I can't provide any other lines other than what I already did in jsfiddle because I have everything else commented out I literally only load/create images when I create a new object (new Particle, new Attractor) and in case of particles move them and then only draw. And on separate canvas I draw FPS (...). – Vico Lemp Mar 22 '16 at 11:26
  • Just a minute ago I managed to remove the black effect with manually filtering the pixels during every draw but I obviously can't do that for procedurally generated graphics on fly. I hope I adjusted jsfiddle in the right way for you if there are any changes needed pls let me know I'm trying my best to adjust everything to make it more friendly. – Vico Lemp Mar 22 '16 at 11:33
  • 1
    Ok found it, it's because Loktar was only checking against black pixels, so the alpha check alone was enough. You need to wrap the whole pixel manipulation block in `if(imagePixelData[iData]+imagePixelData[iData + 1]+imagePixelData[iData + 2]+imagePixelData[iData + 3]>0)`and to remove the `if (cData[pData + 3] < 100)` line. As it is in the other answer, it still does change the r,g and b values to the pixel we're on. Which isn't a problem with black color. Updated fiddle : https://jsfiddle.net/3tLzpLyd/7/ And sorry for misleading comments earlier. – Kaiido Mar 22 '16 at 12:05
  • For the transparent border, it is because of the rounding, you may want to `ceil` when you're on right side of image instead of flooring. – Kaiido Mar 22 '16 at 12:08
  • Ps: your implementation makes me think that we could even win more by storing a 6 slots imageData kind array (`[x,y,r,g,b,a]`) which would avoid to loop through empty pixels. I don't have time tonight to write an answer or to test this thought, but I may do so in next days. – Kaiido Mar 22 '16 at 12:25
  • Oh that worked wonders way better than hand-picking. But I don't understand that part about ceiling instead of flooring. The only flooring I have is ~~ so I tried to adjust as: `(Math.ceil(x + w) + Math.ceil(y + h) * canvas.width) * 4` but that if anything else made it worse (in case of particles it added white line acros in the middle) :-). Currently the problem is that white edge which is all around the object I moved it a bit to make it a bit more visible in this [fiddle](https://jsfiddle.net/3tLzpLyd/9/). – Vico Lemp Mar 22 '16 at 13:18
  • I already thought about storing the pixel data in a similar way but currently my main problem is that I still have those pixel which make that white edge around - thanks to you it got reduced from abundance of black pixels, I'm so grateful. – Vico Lemp Mar 22 '16 at 13:19
  • I tried adjusting your condition (more similar to what I did) as: `imagePixelData[iData]+imagePixelData[iData + 1]+imagePixelData[iData + 2]+imagePixelData[iData + 3]>0 && cData[pData + 3] < 255` and that removed the outer edge perfectly. I tested it with soft edges but it still works fine except that it renders like-in reverse order (but I can loop renderarray in reverse order too). Here are fiddles: [pink circles](https://jsfiddle.net/3tLzpLyd/12/), [particles](https://jsfiddle.net/3tLzpLyd/13/) and [softedges](https://jsfiddle.net/3tLzpLyd/14/). – Vico Lemp Mar 22 '16 at 13:39
  • @Kaiido though I'm not sure why it works and doesn't remove parts of the picture too. Because hard pixels should have 255 alpha value. Cheesh I'm gonna beat my head against the wall to be productive today. – Vico Lemp Mar 22 '16 at 13:51
  • Well the `cData[x] = imagePixel[x]` block will replace the destination's pixel at the position of your object with your object's pixel value. In OP, since he was only playing with black or transparent pixels, it didn't matter and he could only check for pixel's alpha value. You in the other hand, are playing with colored objects, so if you set the destination's pixel to the rgb values of your source one, but not its alpha, you'll make it black ([0,0,0,255]). – Kaiido Mar 23 '16 at 01:00
  • But, you're right, you could also wrap it all in a `if (cData[pData + 3] < 100)` and reverse the array. This way, you will only draw on transparent pixels. That's a good improvement to original answer. – Kaiido Mar 23 '16 at 01:01
  • Can you make an answer here I will definitely pick it for all the help you gave me? – Vico Lemp Mar 23 '16 at 09:26
  • I'm afraid I still won't have time until this weekend... if you can wait I'd be glad to do it. – Kaiido Mar 23 '16 at 09:31
  • Ofc no problem I just want to select some answer :-). In the meantime I'll be wondering why performance in the example is so fast while in my case so slow and how to remove that stuttering (seems like neither rounding nor ceiling actually helps that much especially when drawing not so round object so I will try to find out if any other part of my code is the culprit. Thank you very much again. – Vico Lemp Mar 23 '16 at 10:12

1 Answers1

0

Hunting the black pixels


@Loktar's great answer was made for a particular image, only composed of black and transparent pixels.

In the imageData, these two type of pixels are very similar, since only their alpha value differ. So his code was only doing a should-draw check over the alpha value (the fourth in each loop).

cData[pData] = imagePixData[iData];
cData[pData + 1] = imagePixData[iData + 1];
cData[pData + 2] = imagePixData[iData + 2];
// only checking for the alpha value...
if(cData[pData + 3] < 100){
  cData[pData + 3] = imagePixData[iData + 3];
}

You, in the other hand, are dealing with colored images. So when this part is executed against a transparent pixel, and that you already have a colored pixel at this position, the three first lines will convert the existing pixel to the transparent one's rgb values (0,0,0) but leave the alpha value of the existing pixel (in your case 255).

You then have a black pixel instead of the colored one that were here previously.

To solve it, you can wrap the full block in a condition that checks the opacity of the current imagePixData, instead of checking the one already drawn.

if (imagePixelData[iData+3]>150){
    cData[pData] = imagePixelData[iData];
    cData[pData + 1] = imagePixelData[iData + 1];
    cData[pData + 2] = imagePixelData[iData + 2];
    cData[pData + 3] = imagePixelData[iData + 3];
}

Fighting the white ones


Those white pixels are here because of the anti-aliasing. It was already there in @Loktar's original example, simply less visible because of the size of his images.

These artifacts are crap when you do deal with imageData, since we can just modify each pixel, and that we can't set values on sub-pixels. In other words, we can't make use of anti-aliasing.

That's the purpose of the <100 in original checking, or the >150 in my solution above.

The smallest range you will take in this check against the alpha value, the less artifacts you'll get. But in the other hand, the rougher your borders will be.

You ave to find the right value by yourself, but circles are the worst since almost every border pixels will be anti-aliased.

Improving the awesome (a.k.a I can get you 10000 colored images)


Your actual implementation made me think about some improvements that could be made on @Loktar's solution.

Instead of keeping the original image's data, we could do a first loop over every pixels, and store a new imageData array, composed of six slots : [x, y, r, g, b ,a].

This way, we can avoid the storing of all the transparent pixels we don't want, which makes less iterations at each call, and we can also avoid any alpha checking in each loop. Finally, we don't even need to "get the position pixel from the image canvas" since we stored it for each pixel.

Here is an annotated code example as a proof of concept.

var parseImageData = function(ctx) {
  var pixelArr = ctx.getImageData(0, 0, ctx.canvas.width, ctx.canvas.height).data;
  // the width of our image
  var w = ctx.canvas.width;
  // first store our image's dimension
  var filtered = [];
  // loop through all our image's pixels
  for (var i = 0; i < pixelArr.length; i += 4) {
    // we don't want traparent or almost transparent pixels
    if (pixelArr[i + 3] < 250) {
      continue;
    }
    // get the actual x y position of our pixel
    var f = (i / 4) / w;
    var y = Math.floor(f);
    var x = Math.round((f - y) * w);
    // add the pixel to our array, with its x y positions
    filtered.push(x, y, pixelArr[i], pixelArr[i + 1], pixelArr[i + 2], pixelArr[i + 3]);
  }

  return filtered;
};
// here we will store all our pixel arrays
var images = [];
// here we will store our entities
var objects = [];

var draw = function() {
  // create a new empty imageData of our main canvas
  var imageData = mainCtx.createImageData(mainCanvas.width, mainCanvas.height);
  // get the array we'll write onto
  var pixels = imageData.data;

  var width = mainCanvas.width;

  var pixelArray,
    deg = Math.PI / 180; // micro-optimizaion doesn't hurt

  for (var n = 0; n < objects.length; n++) {
    var entity = objects[n],
      // HERE update your objects
      // some fancy things by OP
      velY = Math.cos(entity.angle * deg) * entity.speed,
      velX = Math.sin(entity.angle * deg) * entity.speed;

    entity.x += velX;
    entity.y -= velY;

    entity.angle++;
    // END update

    // retrieve the pixel array we created before
    pixelArray = images[entity.image];

    // loop through our pixel Array
    for (var p = 0; p < pixelArray.length; p += 6) {
      // retrieve the x and positions of our pixel, relative to its original image
      var x = pixelArray[p];
      var y = pixelArray[p + 1];
      // get the position of our ( pixel + object ) relative to the canvas size
      var pData = (~~(entity.x + x) + ~~(entity.y + y) * width) * 4
        // draw our pixel
      pixels[pData] = pixelArray[p + 2];
      pixels[pData + 1] = pixelArray[p + 3];
      pixels[pData + 2] = pixelArray[p + 4];
      pixels[pData + 3] = pixelArray[p + 5];
    }
  }
  // everything is here, put the image data
  mainCtx.putImageData(imageData, 0, 0);
};



var mainCanvas = document.createElement('canvas');
var mainCtx = mainCanvas.getContext('2d');

mainCanvas.width = 800;
mainCanvas.height = 600;

document.body.appendChild(mainCanvas);


// just for the demo
var colors = ['lightblue', 'orange', 'lightgreen', 'pink'];
// the canvas that will be used to draw all our images and get their dataImage
var imageCtx = document.createElement('canvas').getContext('2d');

// draw a random image
var randomEgg = function() {
  if (Math.random() < .8) {
    var radius = Math.random() * 25 + 1;
    var c = Math.floor(Math.random() * colors.length);
    var c1 = (c + Math.ceil(Math.random() * (colors.length - 1))) % (colors.length);
    imageCtx.canvas.width = imageCtx.canvas.height = radius * 2 + 3;
    imageCtx.beginPath();
    imageCtx.fillStyle = colors[c];
    imageCtx.arc(radius, radius, radius, 0, Math.PI * 2);
    imageCtx.fill();
    imageCtx.beginPath();
    imageCtx.fillStyle = colors[c1];
    imageCtx.arc(radius, radius, radius / 2, 0, Math.PI * 2);
    imageCtx.fill();
  } else {
    var img = Math.floor(Math.random() * loadedImage.length);
    imageCtx.canvas.width = loadedImage[img].width;
    imageCtx.canvas.height = loadedImage[img].height;
    imageCtx.drawImage(loadedImage[img], 0, 0);
  }
  return parseImageData(imageCtx);
};

// init our objects and shapes
var init = function() {
  var i;
  for (i = 0; i < 30; i++) {
    images.push(randomEgg());
  }
  for (i = 0; i < 10000; i++) {
    objects.push({
      angle: Math.random() * 360,
      x: 100 + (Math.random() * mainCanvas.width / 2),
      y: 100 + (Math.random() * mainCanvas.height / 2),
      speed: 1 + Math.random() * 20,
      image: Math.floor(Math.random() * (images.length))
    });
  }
  loop();
};

var loop = function() {
  draw();
  requestAnimationFrame(loop);
};

// were our offsite images will be stored
var loadedImage = [];
(function preloadImages() {
  var toLoad = ['https://dl.dropboxusercontent.com/s/4e90e48s5vtmfbd/aaa.png',
    'https://dl.dropboxusercontent.com/s/rumlhyme6s5f8pt/ABC.png'
  ];

  for (var i = 0; i < toLoad.length; i++) {
    var img = new Image();
    img.crossOrigin = 'anonymous';
    img.onload = function() {
      loadedImage.push(this);
      if (loadedImage.length === toLoad.length) {
        init();
      }
    };
    img.src = toLoad[i];
  }
})();

Note that the bigger your images to draw will be, the slowest the drawings will be too.

Kaiido
  • 123,334
  • 13
  • 219
  • 285
  • Thanks, I appreciate all the help. I managed to make a stress test with my code up to 30000 particles with no less than 30 fps which I consider to be a great result (I'm never getting 30000 particles in my game anyway :-)) and 20000 particles with 50 to 60 fps - while all particles are in all cases interacting with other objects. But that bumpy movement is killing me, when you use larger objects which move fast and not in similar directions it's not that bad but with smaller objects that move in similar directions with similar velocities it's a real killer of esthetics. :-) – Vico Lemp Mar 31 '16 at 18:27
  • I was considering my own kind of way to tackle this problem. I can render 20000 particles which is way past what I need. So with that in mind I thinking that maybe adding one particle more than once with an offset (or more precisely adding to render just the offset parts) could help to reduce bumpy feeling from movement in integer steps - while offset part has a lower alpah value so it's not entirely visible, so far my tests show that it greatly enhances the experience while keeping hihg particle count in game. But it will take me quite soem time before I fine tune it all the glitches :-). – Vico Lemp Mar 31 '16 at 18:31
  • For your trailing, I think you would be better to use `drawImage` : buffer ten frames of the whole canvas in an array of contexts, and draw the buffered images with a `globalCompositeOperation` set to `"destination-over"`, by decreasing the `globalAlpha` at each draw. – Kaiido Apr 01 '16 at 01:07