1

I am trying to build a messenger bot which does some image processing in 3d and returns a brand new image. I use THREE.CanvasRenderer and my app is hosted on Heroku.

When a user /POSTs an attachment to my webhook, I want to take the URL of the newly created image and insert it into my 3d Scene.

This is my code (using the node-canvas library):

const addTexture = (imageUrl) => {

    request({
        uri: imageUrl,
        method: 'GET'
    }, (err, res, body) => {
        let image = new Canvas.Image();
        image.src = body;
        mesh.material.map = new THREE.Texture(image);
        mesh.material.needsUpdate = true;
    });
}

The callback gets run and I can actually console.log() the image's contents, but nothing shows up in the scene - the plane I am supposed to render to just gets black without any errors... What am I missing here?

I also tried several other ways, without any success...

I tried using THREE.TextureLoader and patch it with jsdom (mock document, window) and node-xmlhttprequest, but then there is error with my load event (event.target is not defined...) Just like in this example

How should one approach this problem? I have a url generated by Facebook, I want to download the image from it and place it in my scene?

Community
  • 1
  • 1
Georgi B. Nikolov
  • 976
  • 2
  • 13
  • 24

1 Answers1

2

Okay, I figured it out after half a day and am posting it for future generations:

Here is the code that did the trick for me:

const addTexture = (imageUrl) => {
    request.get(imageUrl, (err, res, data) => {
        if (!err && res.statusCode == 200) {
            data = "data:" + res.headers["content-type"] + ";base64," + new Buffer(data).toString('base64');

            let image = new Canvas.Image();

            image.onload = () => {
                mesh.material.map = new THREE.Texture(image);
                mesh.material.map.needsUpdate = true;
            }
            image.src = data;
        }
    });
}

This way, I still get an error: document is not defined, because it seems under the hood three.js writes the texture to a separate canvas.

So the quick and ugly way is to do what I did:

document.createElement = (el) => {
    if (el === 'canvas') {
        return new Canvas()
    }
}

I really hope this helps somebody out there, because besides all the hurdles, rendering WebGL on the server is pure awesomeness.

Georgi B. Nikolov
  • 976
  • 2
  • 13
  • 24
  • I'm glad you posted this! If only it worked in my case. Could you tell me what you're using for the actual rendering? I've gone through headless-gl, node-webgl, software renderer and I don't know how many others in the past 12 hours. No dice with texturing. When you do get around to seeing this, I'd appreciate the help! – Amit G Sep 05 '17 at 20:49
  • Hey, I am using THREE.CanvasRenderer. Since THREE is using separate 2d canvas to write the texture to, which obviously is not supported in Node.js, I had to use https://github.com/Automattic/node-canvas in order to patch it (also see my ugly patch of document.createElement, so it returns new Canvas()). I did this project a while back and not everything is fresh in my memory, but you can check out https://github.com/gnikoloff/messenger-bot-vaporwavify-me, the repo behind the project :) good luck! – Georgi B. Nikolov Sep 06 '17 at 08:58
  • Thanks Georgi! I'll give it a whirl. Isn't canvas capable of just a subset of what WebGL can do? Did you notice any big render differences between the browser and server version? – Amit G Sep 06 '17 at 10:51
  • Yeah, that's true, canvas2d is limited. It boils down to canvas using the GPU only for rendering, while with WebGL you can offload your animation logic AND render on the GPU. For my simple project I didn't have problems with rendering differences, I guess it depends on what you are after. I am not really sure if you can use advanced stuff such as ShaderMaterial using CanvasRenderer... – Georgi B. Nikolov Sep 06 '17 at 11:54
  • I'm going to see how far I get. It's a pity headless-gl is sorta dead; server side rendering on WebGL seems like an uphill battle right now. – Amit G Sep 06 '17 at 16:04
  • yo, @AmitG, how did it go? – Georgi B. Nikolov Sep 08 '17 at 20:53
  • Canvas worked, but my model had issues - shadows were off and texturing wasn't as good as I expected. I'm still experimenting with headless-gl. I'm trying to make anti-aliasing work by hacking at my forked version; fingers crossed. I'll update this question/you if I make some solid head way. Things that definitely didn't work out - SlimerJS run in headless mode to try and use screenshots for getting images; node-webgl, could never get it to render anything. – Amit G Sep 09 '17 at 12:43