Running Puppeteer with xfvb headless : false

Question

I am running Puppeteer in a headless Ubuntu 16.04 AWS EC2 instance and would like to run it with a virtual display through xfvb. whenever I try to run it I continue to get the error:

/home/ubuntu/node_modules/xvfb/index.js:84
throw new Error('Could not start Xvfb.');    
Error: Could not start Xvfb.
at Xvfb.startSync (/home/ubuntu/node_modules/xvfb/index.js:84:17)
at Object.<anonymous> (/home/ubuntu/puppeteer-works.js:39:6)
at Module._compile (internal/modules/cjs/loader.js:689:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
at Module.load (internal/modules/cjs/loader.js:599:32)
at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
at Function.Module._load (internal/modules/cjs/loader.js:530:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
at startup (internal/bootstrap/node.js:266:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:596:3)

My code is below:

const puppeteer = require('puppeteer');
const fs = require("fs");
const Xvfb = require('xvfb');
var xvfb = new Xvfb();

var text = fs.readFileSync("proxy.txt").toString('utf-8');
const textByLine = text.split(" ");


const preparePageForTests = async (page) => {
  const userAgent = 'Mozilla/5.0 (X11; Linux x86_64)' +
  'AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.39 
Safari/537.36';
  await page.setUserAgent(userAgent);
  await page.evaluateOnNewDocument(() => {
  Object.defineProperty(navigator, 'webdriver', {
    get: () => false,
  });
});
await page.evaluateOnNewDocument(() => {
  window.chrome = {
    runtime: {},
  };
});
await page.evaluateOnNewDocument(() => {
  const originalQuery = window.navigator.permissions.query;
  return window.navigator.permissions.query = (parameters) => (
    parameters.name === 'notifications' ?
      Promise.resolve({ state: Notification.permission }) :
      originalQuery(parameters)
  );  
});
await page.evaluateOnNewDocument(() => {
  Object.defineProperty(navigator, 'plugins', {
    get: () => [1, 2, 3, 4, 5],
  });  
});
}

xvfb.startSync();

(async () => {
  const browser = await puppeteer.launch({
    args: ['--no-sandbox', '--proxy-server='+textByLine[0]],
    headless: true, });
  const page = await browser.newPage();
  page.authenticate({
      username: textByLine[1],
      password: textByLine[2]
  });
  await preparePageForTests(page);

  const testUrl ="https://publicindex.sccourts.org/abbeville/publicindex/";
  await page.goto(testUrl);
  const html = await page.content();
  await page.screenshot({path: 'result.png'});
  await browser.close()
  console.log(html)

})();
xvfb.stopSync();

I appreciate any help, am pretty new to node.js so I apologize in advance for any format errors. I am not being allowed to post this due to it being mainly code, so I am adding this extra sentence.

see: https://github.com/GoogleChrome/puppeteer#debugging-tips — , Aug 16 '18 at 21:04

Kazami · Answer 1 · 2021-04-02T17:02:10.890

12

You seem to be trying to use the Xvfb node module. While the other answers definitely work, here's a snippet that works fully within nodejs

const puppeteer = require('puppeteer')
const Xvfb = require('xvfb');

(async () => {
    var xvfb = new Xvfb({
        silent: true,
        xvfb_args: ["-screen", "0", '1280x720x24', "-ac"],
    });
    xvfb.start((err)=>{if (err) console.error(err)})
    const browser = await puppeteer.launch({
        headless: false,
        defaultViewport: null, //otherwise it defaults to 800x600
        args: ['--no-sandbox', '--start-fullscreen', '--display='+xvfb._display]
        });
    const page = await browser.newPage();
    await page.goto(`https://wikipedia.org`,{waitUntil: 'networkidle2'});
    await page.screenshot({path: 'result.png'});
    await browser.close()
    xvfb.stop();
})()

This isn't perfect in terms of handling errors (and possible race conditions) in xvfb.start(), but it should get you started, and it works pretty consistently for me.

Edit: Remember to install Xvfb first: sudo apt-get install xvfb (Thanks, @iamfrank)

edited Apr 02 '21 at 17:02

answered Jun 12 '20 at 21:28

Kazami

221
2
4

1

Make sure to run `sudo apt-get install xvfb`, too! – Raphael Nov 27 '20 at 22:42
"--start-fullscreen" starts Chromium fullscreen but doesn't set the viewport to the same size. I needed to add `await page.setViewport({width: 1280, height: 720});` to make it do what I wanted. – user1629060 Feb 09 '21 at 18:05
1

@user1629060 True. Alternatively you can just set defaultViewport:null in launch() options. Otherwise it defaults to 800x600 until you fix it. – Kazami Feb 10 '21 at 20:24
Any idea why it does not work from a cron task ? – yarek Nov 09 '21 at 21:42

Ameur Bennaoui · Answer 2 · 2019-05-18T20:43:50.097

5

it's seem th xfvb not install properly

you should Install xvfb (the X windows virtual framebuffer) packages for ubuntu OS

$ sudo apt-get update

$ sudo apt-get install xvfb

Run Xvfb in the background and specify a display number (10 in my example)

$ Xvfb :10 -ac &

Set the DISPLAY variable to the number you chose

$ export DISPLAY=:10

edited May 18 '19 at 20:43

answered May 18 '19 at 19:59

Ameur Bennaoui

309
3
5

I'm still stuck with this error when doing the above and then trying to launch headless chromium/puppeteer: `ERROR:browser_main_loop.cc(536)] Failed to open an X11 connection.` – AlxVallejo Jan 24 '23 at 15:15

Liju Kuriakose · Answer 3 · 2019-07-26T16:58:38.283

4

If other required packages and xvfb are properly installed, then run the following command.

xvfb-run -a --server-args="-screen 0 1280x800x24 -ac -nolisten tcp -dpi 96 +extension RANDR" command-that-runs-chrome

Cheers!!!

edited Jul 26 '19 at 16:58

answered Jun 26 '19 at 14:39

Liju Kuriakose

445
3
11

Running Puppeteer with xfvb headless : false

3 Answers3

Linked