1

I have been working on a Discord bot to generate image attachments and send them into channels; and it all works a treat. However I have now encountered a problem in that when I use images, and in this case 32 images for an attachment the 'RSS' memory as per process.memoryUsage.rss() hits the roof; if I run this process 2-3 times, the serve dies due to lack of Memory.

So this is all in a Node Process, on Heroku.

The code in principle:

  let stage = new Konva.Stage({
    width: width,
    height: height,
  });

  let canvas1 = createCanvas(stage.width(), stage.height());
  const ctx1 = canvas1.getContext('2d'); // I draw to this

  let canvas2 = createCanvas(stage.width(), stage.height());
  const ctx2 = canvas2.getContext('2d'); // I draw to this

  let layer = new Konva.Layer();

  ...

  // It's these images that seem to cause the most Memory Bloat
  Konva.Image.fromURL(
    imagePath,
    imageNode => {
      imageNode.setAttrs(attrs); // attrs = { image, x, y, width, height }
      layer.add(imageNode);      
      // imageNode.destroy(); doesn't save memory
    }
  );

  ...

  // Add the canvas to the Konva, as an image
  layer.add(new Konva.Image({
    image: canvas1,
    x: 0,
    y: 0,
    width: stage.width(),
    height: stage.height(),
    opacity: 0.5,
  }));

  // Add the canvas to the Konva, as an image
  layer.add(new Konva.Image({
    image: canvas2,
    x: 0,
    y: 0,
    width: stage.width(),
    height: stage.height(),
    opacity: 0.5,
  }));

  ...

  stage.add(layer);
  layer.draw();
  let asCanvas = stage.toCanvas(); // I use this for the attachment

  stage.destroy();
  stage = null;
  layer = null;

  return asCanvas;

  ...

  
  let attachment = new Discord.MessageAttachment(
    fromAsCanvas.toBuffer(), // fromAsCanvas is from the asCanvas above.
    'imageName.png'
  );

  message.channel
    .send({
      files: [attachment],
      content: 'Message with the Attachment',
    });

My belief at this time, is that the images loaded from the system, and added to the layer, then to canvas, are not freeded up from memory and just sit there for a significant amount of time with no consistincy.

I have tried:

  • Running the Garbage Collector afterwards to confirm if this would help (it does not)
  • Destroying the image post-add to layer (this removed the image itself)
  • Destroying the Stage
  • Destroying the Layer
  • Nulling the Stage, Layer, and all 4 Canvas related variables

I have lots of logging, e.g:

Start: 74MB
Post ctx1: 75MB
Post ctx2: 75MB
Post Reduce: 77MB
Post forEach: 237MB // adds all 32 the images +160MB
Post Mark Destroyed Guns (spread arrays): 237MB
Post Added Some Things 1: 247MB +10MB
Post Added Some Things 2: 249MB
Post Added Some Things 3: 259MB
Post Added Some Things 4: 260MB
Post Add canvas1 Canvas to Layer: 260MB
Post Add canvas2 to Layer: 260MB
Post Add Layer to Stage: 293MB +33MB
Post Layer.draw: 294MB
Post toCanvas: 321MB +27MB
Post Destroy Stage/etc: 308MB -13MB
Sends message
5 Seconds later RSS is at: 312MB +4MB

As you can see, once I have this solved, I might still have 50MB of extra Memory usage to debug too.

Nimantha
  • 6,405
  • 6
  • 28
  • 69
Yin117
  • 31
  • 7

1 Answers1

1

I believe I have now solved this, based on this article: https://github.com/Automattic/node-canvas/issues/785 Basically, I added inside 'onload' the statement img.onload = null; as I had refactored to a promise/onload system during my attempts to solve this.

Followup 1:

In addition to setting img.src = null I then followed up with the following:

process.nextTick(() => {
  img.onload = null;
  img.onerror = null;
  img = null;
})

The setting of img = null brought back a lot more memory, but still left plenty taken up.

Followup 2:

I discovered that destroying the layer and image also helped free up even more memory, to the point that it seems I actually end up recovering memory between processes:

layer.destroyChildren();
layer.destroy();
stage.destroyChildren();
stage.destroy();
stage = null;
layer = null;

I am hoping, this has finally solved it.

Yin117
  • 31
  • 7