How to call tensorflow_model_server from Nodejs, with an image

Question

This is the metadata of my serving model

"metadata": {"signature_def": {
 "signature_def": {
  "serving_default": {
   "inputs": {
    "vgg16_input": {
     "dtype": "DT_FLOAT",
     "tensor_shape": {
      "dim": [
       {
        "size": "-1",
        "name": ""
       },
       {
        "size": "224",
        "name": ""
       },
       {
        "size": "224",
        "name": ""
       },
       {
        "size": "3",
        "name": ""
       }
      ],
      "unknown_rank": false
     },
     "name": "serving_default_vgg16_input:0"
    }
   }...

Sadly I don't know how to talk to it from NodeJs. How to transorm a local image to a valid 224,224,3 DT_FLOAT Tensor ...

In python, i can do it with this code, but I would like the nodejs version

import numpy as np
import requests
from keras.preprocessing import image

image_path = './data/black/fhb2l97vdi8qc0rt5ow3.jpg'
img = image.img_to_array(image.load_img(image_path, target_size=(224, 224))) / 255.
img = img.astype('float16')

payload = {
    "instances": [{'vgg16_input': img.tolist()}]
}

r = requests.post('http://ip:port/v1/models/color:predict', json=payload)
print(r.content)

So far my code

var request = require('request');
var fs = require('fs');
var myImg = __dirname + '/../tensorflow2/data/black/0a13y2gtunswi8ox4bjf.jpg';



var options = {
    method: 'POST',
    url: `http://ip:port/v1/models/color:predict`,
    json:{ 
          instances: [{'vgg16_input': ??????}]
        }

};

request(options, function (err, resp, body) {
    if (err)
      cb(err);

      console.log(body);
  });

Maybe i could use some function from tensorflowjs ...

Michał Słapek · Accepted Answer · 2019-11-04T18:28:57.577

2

The image must be passed in JSON as list of lists of floats (pixel is list of 3 RGB values, row is list of pixels and image is list of rows).

We need to decode and resize the JPEG image. Install sharp package with npm install sharp.

Image preparation is as follows:

const fs = require('fs');
const sharp = require('sharp');

function imgToJson(buffer) {
  var decoded = [];

  var h;
  var w;
  var line;
  var pixel;

  var b = 0;
  for (h = 0; h < 224; h++) {
      var line = [];
      for (w = 0; w < 224; w++) {
          var pixel = [];

          pixel.push(buffer[b++] / 255.0); /* r */
          pixel.push(buffer[b++] / 255.0); /* g */
          pixel.push(buffer[b++] / 255.0); /* b */

          line.push(pixel);
      }
      decoded.push(line);
  }

  return decoded;
}

async function prepare_image(imagePath) {
  var jpegData = fs.readFileSync(imagePath); /* for example sake synchronous */
  const buffer = await sharp(jpegData).resize(224, 224).raw().toBuffer();
  return imgToJson(buffer);
}

The result of prepare_image is future returning the list of lists of floats representing image. The last step is to perform request:

var request = require('request');

async function perform_request(imagePath) {
  var decoded = await prepare_image(imagePath);
  var options = {
      method: 'POST',
      url: 'http://ip:port/v1/models/color:predict',
      json: { 
            instances: [{'vgg16_input': decoded}]
      }
  };

  request(options, function (err, resp, body) {
      if (err)
        cb(err);

        console.log(body);
    });
}

perform_request("image.jpeg");

edited Nov 04 '19 at 18:28

answered Nov 02 '19 at 19:46

Michał Słapek

1,112
7
10

Thank you. js answer : { predictions: [ [ 0.869870663, 0.109581947, 1.78813934e-7, 0.00818741322, 4.76837158e-7, 0.00215744972, 3.57627869e-7, 0.00000619888306, 8.14288654e-8, 0.00000102930289 ] ] } python answer "predictions": [[0.979003787, 2.9951334e-05, 8.94069672e-08, 0.000181376934, 7.18832e-05, 0.00502485037, 6.68466091e-05, 0.00051087141, 3.17406625e-06, 4.25011876e-07] weird that it's not the same prediction ... – Laurent Debricon Nov 04 '19 at 16:30
Second alternative : { error: 'Input to reshape is a tensor with 153600 values, but the requested shape requires a multiple of 25088\n\t [[{{node sequential/flatten/Reshape}}]]' } – Laurent Debricon Nov 04 '19 at 17:04
1

As mentioned in answer, the second alternative does *not* perform resize - therefore it assumes, that the image is *already* `(224, 224)`. – Michał Słapek Nov 04 '19 at 17:50
1

For clarity I've decided to remove the alternative solution (which does not support resizing, using jpeg-js). It is available in answer edit history. – Michał Słapek Nov 04 '19 at 17:51
1

I've fixed path passing (remove unused variable `myImg`). Please, retry this answer. Did you used output from `import request ...` Python script or directly from TensorFlow? – Michał Słapek Nov 04 '19 at 18:30
output from import request – Laurent Debricon Nov 05 '19 at 08:44
Ok i used a 224x224 image made from Gimp and compare python and js version, and it's the same, great ! – Laurent Debricon Nov 05 '19 at 09:01

How to call tensorflow_model_server from Nodejs, with an image

1 Answers1