Slow image filling of polygons in canvas/javascript

Question

I'm making a business app with some animation in HTML, javascript and canvas.

I'm filling a screen, almost like a map, with polygons - each of which is filled with an image - the polygons are moving and changing shape constantly. I'm using createpattern(image, "repeat") with the images, translating the canvas so the image is centered at the centre of the polygon, specifying the edges then doing a fill.

When I use the same 256x256 image for every polygon, it takes ~2ms to draw. That slows down to about 13-40ms with a shared 1024x1024 image. However - when each polygon is given a different picture of 1920x1080 (there are 29 polygons) - it slows down to like 1500-2500ms to render - even though it's filling exactly the same polygons, exactly the same area, and in theory doing the exact same amount of work - (and all the images have been preloaded, and are sitting in image objects)

It seems like just having/using a number of different/larger images or maybe patterns is slowing it down - essentially by a factor of close to 3 orders of magnitude. Am I doing something wrong? Is there any other/better way to do this?

Answer 1

The GPU needs to have the resources (images) available in its memory to render them. The browser ensures that no matter how many images you have they are always available to render . The problem is that when the memory requirement for the images exceeds the RAM capacity of the GPU the browser will swap the images in as needed. The interface between the computer's RAM and the GPU's is very slow in comparison to the GPU's internal speed, also while it's waiting for the new data it can not render those images.

The result is that when you reach the GPU memory limit there is a dramatic reduction in performance. There is not much that you can do to fix this apart from using smaller images.

You also have to consider that different devices have differing amounts of RAM available to the GPU so the images should be a size that will fit the lowest common GPU RAM capacity.

There is no way to query the GPU directly and discover its set up from the browser. It is possible to discover it approximately by testing frame rates while increasing the image count. When the frame rate suddenly drops you know you have found the max memory for that device. You can then resize your images to fit that limit. Unfortunately this is not very user friendly.

But I personally believe there is always a solution. The user can never see more pixels than are available on the screen. If you are rendering with images that exceed the GPU RAM capacity you must be losing many pixels during the rendering due to scaling, clipping, z-buffering, and what not.

Look carefully at what you are rendering. If you are down scaling during the render then do that to the image when you load it.

If you are clipping lots of pixels out, then clip the image at load time. Or if you are only displaying a small part of a large image, consider making a second image with only what is needed, then render that. The larger image can stay in the computers RAM while the GPU only handles the smaller part of the image.

If many pixels are obstructed by other images that render on top then consider reducing the resolution of the obsured pixels.

A careful examination of your scene with these things in mind will bring your frame rate back up. It may increase the complexity of the code, but is a small price to pay for a smooth user experience.

Answer 2

thanks all - I sort of got around the issue by resizing the images on the browser, by drawing into a canvas and then copying a new image out of the canvas, producing a number of different image sizes, and then picking the smallest I need for any polygon. That worked, and it's now down to 3ms each render again.

Also, interestingly, I tried switching to drawimage instead of fill - and found it was very slightly slower.

Answer 3

Here's a test that generates 30 images at 1920x1080 and animates them. I'm getting around 60 frames per second. If you're getting a good frame rate then you may be able to rule out caching as a problem. Try creating the patterns before the animation loop and destroying the images. This should remove any duplicates of the image data.

function iGen(w, h, n) {
  var imgs = [];
  var step = 100 / n;
  var cx = Math.floor(w / 2);
  var cy = Math.floor(h / 2);

  for (var s = 0; s < n; s++) {

    var can = document.createElement('canvas');
    var ctx = can.getContext('2d');

    can.width = w;
    can.height = h;
    ctx.strokeStyle = "#FFFFFF";
    ctx.lineWidth = 3;

    genNoise(ctx);

    for (var i = 10 + (step * s); i < cx + 200; i += 100) {
      ctx.beginPath();
      ctx.moveTo(cx + i, cy);
      ctx.arc(cx, cy, i, 0, Math.PI * 2);
      ctx.stroke();
    }

    imgs.push(can);
  }
  return imgs;
}


function rCol() {
  var r = Math.floor(Math.random() * 255 * 255 * 255);
  var h = r.toString(16);
  return "#" + "000000".substr(h.length) + h;
}

function genNoise(ctx) {

  var imageData = ctx.getImageData(0, 0, ctx.canvas.width, ctx.canvas.height);
  var d = imageData.data;
  for (var i = 0; i < d.length; i += 4) {
    d[i] = d[i + 1] = d[i + 2] = Math.floor(Math.random() * 64);
    d[i+3] = 255;
  }
  ctx.putImageData(imageData, 0, 0);
}



function testImages() {
  var w = 1920;
  var h = 1080;
  var n = 30; // Number of images to generate
  var imgs = iGen(w, h, n);
  var can = document.getElementById('can');
  can.width = w;
  can.height = h;
  var ctx = can.getContext('2d');
  var i = 0;

  var frames = 0;
  var start = (new Date()).getTime();
  var fps = "fps: ?";

  ctx.font = "32px Sans-serif";
  ctx.fillStyle = "red";

  function loop() {
    ctx.drawImage(imgs[i++], 0, 0);
    i %= imgs.length;

    ctx.fillText(fps + ", img: " + i, 32, 32);
    frames++;
    var now = (new Date()).getTime();
    if ((now - start) >= 1000) {
      fps = "fps: " + frames;
      frames = 0;
      start = now;
    }
    requestAnimationFrame(loop);
  }
  loop();
}

testImages();

<canvas id="can"></canvas>