Heroku Slug Size too large - nodejs

Question

This is my first heroku app... and I see that my heroku slug size is 296MB... getting uncomfortably close to the fast 300MB boot time.

This is a puppeteer app with ejs, path, and express installed. I have a bunch of static files but they don't seem to be taking up majority of the space..

I would really appreciate help with this!

Edit: My package.json looks like this now (don't see how I could really trim down my node_modules - in fact, I wanted to add more dependencies as I further develop my app):

    "scripts": {
    "start": "node server.js",
    "heroku-postbuild": "curl -sf https://gobinaries.com/tj/node-prune | PREFIX=. sh&&./node-prune"
  },
  "dependencies": {
    "ejs": "^3.1.5",
    "express": "^4.17.1",
    "path": "^0.12.7",
    "puppeteer": "^5.3.1"
  }

Answer 1

Since Puppeteer takes so much Heroku slug size, there's really very little room to maneuver. For now, sticking to Node 12 and Puppeteer 2.0.0 will ensure that you can manage to keep your Heroku slug size just below 300M.

Use these in your package.json:

"dependencies": {
    "puppeteer": "2.0.0"
}

"engines": {
    "node": "12.x"
}

If you upgrade either of the version numbers of node or puppeteer, your slug size will exceed 300M even if you have very little other dependencies.

Another reason I'm using Puppeteer 2.0.0 is that I need to use the waitForFileChooser() function to upload media files to some web sites. But waitForFileChooser() is broken in Puppeteer 2.1.1. And it's still broken in 5.5.0.

Plus, I think Node 12 and Puppeteer are actually quite stable. So I can accept a little bit version lag for the benefit of helping Heroku manage the cost of its fantastic free service.

Answer 2

After all the talk in the comments, I realized one thing: you can't delete folders in Heroku dynos! And you probably wouldn't want to anyways... Those folders look important (remember: Heroku dynos are usually mini-linux OSes, so that .apt folder probably contains some of those mini-linux OS files). And of course, those .apt and .heroku folders probably don't add to your slug size. Sorry for that wild goose chase.

So? Let's reduce the sizes of stuff that actually matter.

Number one: reduce your node_modules size.

Since your node_modules folder is the biggest folder that you could control, let's start there. We'll be following this article. I'll be modifying some instructions so that they work with Heroku.

1. Reduce the number of dependencies: This may sound like a no-brainer, but it helps a lot. Think about the packages you install. Do you really need them? And if you do, is there a more lightweight package somewhere out there that you could use instead? Example:

A lot of times I see people installing Jest just for simple unit tests (about 460+ dependencies, +/-60MB) when there are smaller libs that will do exactly the same (Mocha – 115 dependencies, 12MB). 2. Remove unnecessary files: Besides the usual installed when you install a package (.js files, etc.), there's also a lot of... unneeded junk included (READMEs, CHANGELOGs, source files...). Because you can't remove those files manually (and who wants to), you need to use an automated tool and Heroku build scripts. Here's an example (remove the comments when you put this in your package.json).

"scripts": {
  // Other scripts...
  // The following script dowloads node-prune (https://github.com/tj/node-prune)
  // in the current build directory and runs it.
  // The following script is run AFTER Heroku installs dependencies, but BEFORE
  // Heroku caches them.
  // EDIT: You have to do `./node-prune` instead of `node-prune`.
  "heroku-postbuild": "curl -sf https://gobinaries.com/tj/node-prune | PREFIX=. sh&&./node-prune"
}

Annnd...

That's about all you could do. The other steps mentioned in that article won't help on Heroku. And as you already mentioned, your static files aren't really taking up much space–it's actually just node_modules. I can't think of any other ways to reduce your slug size (unless you want to move your static files to an external storage facility and do some obscure get-and-cache maneuverers to serve them to clients...).

NOTE: I have not tried any of these steps. These should work in theory, but I'm not positive if they work or not.

Answer 3

I wanted to chip in to this discussion, because I have faced to same problem, and was really frustrated that I can't get it work because of the 500MB limit. I tried every way to reduce it but I found that the performance is hugely degraded even if I get to make it just under 500MB. So what I decided to do was to have a separate dedicated server for Puppeteer, and completely remove the Puppeteer portion out of my codebase, this reduced my slug size by 200+MB. Take a look at it: https://github.com/davidjuhyung/puppeteer-api