0

I'm trying to use headless Chrome (v66 on Win10, using C#) to take a series of screenshots from a 3D model in forge autodesk viewer.

The problem i'm facing is that once the model is loaded i set the camera into the first position, take a screenshot and then try to set the camera to the next position for the next screenshot. Once i try that (setting the camera position later then on the initial load), the webgl context is lost.

I have too little knowledge of WebGL / swiftshaders / etc, but what i find frustrating, is that when i position the camera directly after the load, it does work. (IE, the workaround is to spawn a seperate headless session per camera view, but since the loading of the geometry takes 20sec or more, thats not preferred)

So, this:

viewerApp.myCurrentViewer.addEventListener(Autodesk.Viewing.GEOMETRY_LOADED_EVENT,
function () {
    _viewer = viewerApp.myCurrentViewer;
    SetPerspective();
    SetCamera(cams[0].position, cams[0].target);//no probs here
    document.getElementById('MyViewerDiv').classList.add("geometry-loaded");       
 });

works (the camera is positioned), but when i execute a javascript function later (using driver.ExecuteScript($"SetCamera({JsonConvert.SerializeObject(target.Value.Position)},{JsonConvert.SerializeObject(target.Value.Target)});"); or on a timeout in the page itself, it outputs WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost.

When i use a smaller model, everything works. Thus i think i understand the reason is too much memory/processing consumption, but why is it working at all then?

Looking at resource monitor, im not convinced that the consumption is actually problematic, my laptop should be capable (i7HQ7700, gtx1050, 16gbRam) I tried fiddling around with some GPU and GL flags of Chrome, to no avail. I suspect the GPU isn't used (which i found some posts that it actually can be used in headless...) Also, the forge viewer outputs GPU mem used, but that might be just the log message:

Starting ChromeDriver 2.38.552522 (437e6fbedfa8762dec75e2c5b3ddb86763dc9dcb) on port 62676
Only local connections are allowed.
[0517/203535.902:ERROR:gpu_process_transport_factory.cc(1007)] Lost UI shared context.

DevTools listening on ws://127.0.0.1:12556/devtools/browser/5b66c120-dc64-4211-a207-ac97152ace9a
---some ssl future warnings---
[0517/203540.524:INFO:CONSOLE(2)] "THREE.WebGLRenderer", source: https://developer.api.autodesk.com/modelderivative/v2/viewers/three.min.js (2)
[0517/203543.074:INFO:CONSOLE(0)] "[.Offscreen-For-WebGL-00000237DECBB270]RENDER WARNING: there is no texture bound to the unit 0", source: http://localhost:8881/Content/Screenshot.html
[0517/203543.074:INFO:CONSOLE(0)] "[.Offscreen-For-WebGL-00000237DECBB270]RENDER WARNING: there is no texture bound to the unit 0", source: http://localhost:8881/Content/Screenshot.html
[0517/203552.280:INFO:CONSOLE(2)] "Total geometry size: 8.434013366699219 MB", source: https://developer.api.autodesk.com/modelderivative/v2/viewers/three.min.js (2)
[0517/203552.281:INFO:CONSOLE(2)] "Number of meshes: 2909", source: https://developer.api.autodesk.com/modelderivative/v2/viewers/three.min.js (2)
[0517/203552.281:INFO:CONSOLE(2)] "Num Meshes on GPU: 2908", source: https://developer.api.autodesk.com/modelderivative/v2/viewers/three.min.js (2)
[0517/203552.281:INFO:CONSOLE(2)] "Net GPU geom memory used: 7494392", source: https://developer.api.autodesk.com/modelderivative/v2/viewers/three.min.js (2)
[0517/203558.143:INFO:CONSOLE(0)] "WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost", source: http://localhost:8881/Content/Screenshot.html

To be complete, running the same program without the --headless flag, is working fine, so i guess the code itself is ok.

Is there any way to increase the allowed resources or anything?

(code for SetCamera)

        function SetCamera(newPos, newTarget) {
        nav = nav || viewerApp.myCurrentViewer.navigation;

        nav.setPosition(newPos);
        nav.setTarget(newTarget);
        nav.orientCameraUp();
    }

EDIT: Test case (currently on a test website, so this will we deleted at some point)

EDIT2: Result for running code below

NodeJS:

try {

    const URN = '';
    const Token = '';

    (async () => {
        const puppeteer = require('puppeteer');
        const browser = await puppeteer.launch();
        const page = await browser.newPage();
        console.log('browsing');

        await page.goto('https://rogerintelligentcloud.azurewebsites.net/test?urn=' + URN + '&token=' + Token);
        //replace autodeskURN and token to point to your model

        console.log("waiting");
        await page.mainFrame().waitForSelector(
            '.geometry-loaded', {
                timeout: 60000
            });

        await takescreen(page, 'nodetest1');
        await takescreen(page, 'nodetest2');
        await takescreen(page, 'nodetest3');
        await takescreen(page, 'nodetest4');
        await takescreen(page, 'nodetest5');
        await takescreen(page, 'nodetest6');    

        await page.evaluate("Test();");

        await takescreen(page, 'nodetest11');
        await takescreen(page, 'nodetest12');
        await takescreen(page, 'nodetest13');
        await takescreen(page, 'nodetest14');
        await takescreen(page, 'nodetest15');
        await takescreen(page, 'nodetest16');        

        await browser.close();
    })();
} catch (e) {
    console.log(e);
}

async function takescreen(page, name){
    await page.screenshot({
        path: 'c:\\temp\\'+name+'.png'
    });
}

Result of code

user1515791
  • 675
  • 4
  • 20
  • There seems not to be any official .NET Chrome headless API. As I know, there is only Chrome headless API for Node.js called [puppeteer](https://github.com/GoogleChrome/puppeteer). Have you checked the awesome puppeteer demo posted [here](https://forge.autodesk.com/blog/running-forge-viewer-headless-chrome-puppeteer)? – Eason Kang May 21 '18 at 01:55
  • As i understand, puppeteer handles Chrome over the "Chrome DevTools Protocol" as well as the selenium webdriver (which i use). I can try it using puppeteer, but do you really think itll make a difference?? (the WebGL message comes from Chrome, not the driver) – user1515791 May 22 '18 at 11:48
  • I didn’t see any error shows up while using the puppeteer, I think this is the only difference between us. Could you please have it a try? – Eason Kang May 22 '18 at 13:06
  • @EasonKang, same results: the first screenshot is ok, the second is empty. I don't see the errormessage right now (using puppeteer) but the result is exactly the same. I know the function is called also, because in my test method i also append a div element and that is actually printed. The canvas however is empty. – user1515791 May 23 '18 at 07:24
  • @EasonKang, what is the model 'size' you tested with? Does that compares to the output of the model I have? – user1515791 May 23 '18 at 07:27
  • My model size is similar to yours. Could you consider providing a minimum complete yet reproducible case contain your nodes app and model to forge.help@autodesk.com? – Eason Kang May 23 '18 at 07:32
  • Will try to supply it. Problem is that the model contains data that im not allowed to share. – user1515791 May 24 '18 at 07:02
  • We won't ask for confidential models. May I ask a favor to help me check if this issue could be reproduced with Revit sample models? https://knowledge.autodesk.com/support/revit-products/getting-started/caas/CloudHelp/cloudhelp/2016/ENU/Revit-GetStarted/files/GUID-61EF2F22-3A1F-4317-B925-1E85F138BE88-htm.html – Eason Kang May 24 '18 at 08:30
  • rac_basic_sample_project.rvt is fine, but rme_advanced_sample_project.rvt will trigger the same behavior. So the size does matter. I've updated my question with my code for setcamera – user1515791 May 24 '18 at 14:52
  • @EasonKang, does rme_advanced_sample_project.rvt work for you, or do you need/is it helpfull to get code from me to test? – user1515791 May 29 '18 at 07:49
  • A non-confidential reproducible case which contains a simple Node.js project with puppeteer and the `rme_advanced_sample_project.rvt` model is good to me. This will ensure that we are in the same boat! :) – Eason Kang May 29 '18 at 07:53
  • @EasonKang, see updated answer. ive uploaded a temporary site to demonstrate the behavior, in the nodejs app, you just have to replace the token and urn to point to rme_advanced_sample_project.rvt model – user1515791 May 29 '18 at 10:14
  • I put my test codes and comment below, please have a try. – Eason Kang May 30 '18 at 03:48

1 Answers1

1

I didn't see any WebGL related error messages show up with your snippet and the rme_advanced_sample_project.rvt model, the only one I can see is page.delay is not defined. Here is my test code modified from the your code snippet and Philippe's forge-viewer-headless demo. If I missed something, please kindly point out. Thanks~

import puppeteer from 'puppeteer'
import 'babel-polyfill'
import path from 'path'
import os from 'os';

try {
    const URN = 'YOUR_URN';
    const Token = 'YOUR_TOKEN';

    (async () => {
        const browser = await puppeteer.launch();
        const page = await browser.newPage();
        await page.goto(`https://rogerintelligentcloud.azurewebsites.net/test?urn=${ URN }&token=${ Token }`);
    //replace autodeskURN and token to point to your model

        await page.mainFrame().waitForSelector(
            '.geometry-loaded', {
                timeout: 60000
            });

        await delay(3000);

        await page.screenshot({
          path: path.join( os.tmpdir(), 'nodetest.png' )
        })


        await page.evaluate("Test();");
        await page.evaluate("Test();");
        await page.evaluate("Test();");
        await page.evaluate("Test();");

        await delay(3000);

        const targetTxt = await page.evaluate(() => document.querySelector('body > :last-child').innerText );
        console.log( targetTxt );

        const targetLen = await page.evaluate(() => document.querySelectorAll('body > div:not(.box)').length );
        console.log( targetLen );

        await page.screenshot({
          path: path.join( os.tmpdir(), 'nodetest2.png' )
        })

        await browser.close();
    })();

    function delay(timeout) {
        return new Promise((resolve) => {
            setTimeout(resolve, timeout);
        });
    }
}
catch (e) {
    console.log(e);
}

Snapshot from my test result:

  1. nodetest.png: enter image description here

  2. nodetest2.png: enter image description here

Edit 2:

Tested with your code, it works fine on my machine. enter image description here

But there is one change to run your code properly in my env. I modified your takescreen function definition:

function takescreen(page, name){
    return page.screenshot({
        path: 'c:\\temp\\'+name+'.png'
    });
}
Eason Kang
  • 6,155
  • 1
  • 7
  • 24
  • Thanks for your efforts, for the delay undefined; did you also copy the last part of my nodejs-script? For the WEBGL errors: the browser needs some time to render the model, you can see this because the screenshot created in your example are not fully rendered. Therefor i added the delay function, to give the browser time to render the complete model. This is also needed when camera is moved. If you run the site in browser and Test() using F12 tools, youll see what the screens should look like (1st should be turned relative to other) – user1515791 May 30 '18 at 20:05
  • Thank you for pointing out, but I'm still not able to see any WebGL error message shows up after adding your `delay` function. – Eason Kang May 31 '18 at 13:12
  • Hmm, so, in your test, you end up with 2 screenshots of the rendered model, from a different angle? (just to be precise, i do not actually see the webgl error in puppeteer. I just end up with 1 good screenshot and 1 empty) – user1515791 May 31 '18 at 14:48
  • I got two good snapshots in my test, and yes, they are in different view angles, the camera does move it position after calling your ‘Test()’. – Eason Kang May 31 '18 at 15:18
  • I attached my snapshots above~ :) – Eason Kang May 31 '18 at 15:28
  • Yeah, thats indeed what im after... Can i ask what computer setup you use? – user1515791 Jun 01 '18 at 07:03
  • node.js X64 v6.10.2, npm v5.4.2, Windows 7 with i7-6700 3.4GHz, 8GB RAM & NVIDIA Quadro K620 – Eason Kang Jun 01 '18 at 07:22
  • Weird. My PC should be capable. But i cannot get it to work. – user1515791 Jun 04 '18 at 09:15
  • Ive edited the code and supplied result for me, can you (one last time) try it? Also, did you by any chance did configuration edits that allow the GPU to be used while running puppeteer? I think for me its not running on GPU. – user1515791 Jun 04 '18 at 11:46
  • I tested with your codes on my machine, it works fine as I expected. – Eason Kang Jun 06 '18 at 02:44
  • Ok. Thanks for the help and testing. – user1515791 Jun 06 '18 at 06:17