1

I am trying to detect the objects using tensorflow cocossd model.

In case of png/jpg image: My script is working fine both in laptop and mobile browser.

But in case of camera video frame: My script is working fine in laptop browser, but it is not working in mobile browser.

Here is sample of my code:

 var img = new Image();
img.src = "catDog.jpg";

//I have videoframe buffer
var ubuf = new Uint8Array(videoFrame.buffer(0).buffer);
var imgData = imageCtx.createImageData(imageDiv.width, imageDiv.height)
imgData.data.set(ubuf)

cocoSsd.load().then(model => {
    // detect objects in the image.
    model.detect(imgData).then(predictions => {
      console.log('Predictions: ';
      console.log(predictions);
    });
  });

the above code is working fine in the laptop browser but not in the mobile browser. Though if I use img instead of imgData, the script is working fine for both laptop and mobile. In console no error is showing just print predictions: [] (object detected=0)

Update:

I just found some difference in image data when it produces in mobile or laptop browser. I updated the above code as follows:

var img = new Image();
//img.src = "catDog.jpg";

//I have videoframe buffer
var ubuf = new Uint8Array(videoFrame.buffer(0).buffer);
var imgData = imageCtx.createImageData(imageDiv.width, imageDiv.height)
imgData.data.set(ubuf)
imageCtx.putImageData(imgData,0,0)  // for debuging
img.src=cvImageDiv.toDataURL()

cocoSsd.load().then(model => {
    // detect objects in the image.
    model.detect(img).then(predictions => {
      console.log(img)
      console.log('Predictions: ';
      console.log(predictions);
    });
  });

And I found from console(img), img format is different in different browser. and result is same: working in laptop browser but not in mobile browser but if I put the img static way, which is produced in laptop browser, it works in both place.

i.e.

img.src = 'data:image/png;base64,iVBORw0KGgoAAAANS...AASUVORK5CYII='

// above value was produce in laptop and for hardcode input as img.src, prediction works well in both laptop and mobile.

But

img.src = 'data:image/png;base64,iVBORw0KGgoAAAANS...AAAAAElFTkSuQmCC'

// the above value was produced in mobile browser, and for hardcode input, prediction does not work in any laptop/mobile browser

Shaon Debnath
  • 95
  • 2
  • 10
  • You have to give more information about what errors you are seeing in the mobile browser. There are also online tools available to simulate mobile browsers in order to test your application/view the console errors. – jlewkovich Mar 10 '19 at 09:51
  • In mobile browser console, it does not show any error. it just shows predictions: [], which mean, number of detected object is 0 – Shaon Debnath Mar 10 '19 at 11:07

0 Answers0