1

Code using tfjs-node:

const model = await tf.node.loadSavedModel(modelPath);
const data = fs.readFileSync(imgPath);
const tfimage = tf.node.decodeImage(data, 3);
const expanded = tfimage.expandDims(0);
const result = model.predict(expanded);
console.log(result);
for (r of result) {
   console.log(r.dataSync());
}

Output:

(8) [Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor]
Float32Array(100) [48700, 48563, 48779, 48779, 49041, 48779, ...]
Float32Array(400) [0.10901492834091187, 0.18931034207344055, 0.9181075692176819, 0.8344497084617615, ...]
Float32Array(100) [61, 88, 65, 84, 67, 51, 62, 20, 59, 9, 18, ...]
Float32Array(9000) [0.009332209825515747, 0.003941178321838379, 0.0005068182945251465, 0.001926332712173462, 0.0020033419132232666, 0.000742495059967041, 0.022082984447479248, 0.0032682716846466064, 0.05071520805358887, 0.000018596649169921875, ...]
Float32Array(100) [0.6730095148086548, 0.1356855034828186, 0.12674063444137573, 0.12360832095146179, 0.10837388038635254, 0.10075071454048157, ...]
Float32Array(1) [100]
Float32Array(196416) [0.738592267036438, 0.4373246729373932, 0.738592267036438, 0.546840488910675, -0.010780575685203075, 0.00041256844997406006, 0.03478313609957695, 0.11279871314764023, -0.0504981130361557, -0.11237315833568573, 0.02907072752714157, 0.06638012826442719, 0.001794634386897087, 0.0009463857859373093, ...]
Float32Array(4419360) [0.0564018189907074, 0.016801774501800537, 0.025803595781326294, 0.011671125888824463, 0.014013528823852539, 0.008442580699920654, ...]

How do I read the predict() response for object detection? I was expecting a dictionary with num_detections, detection_boxes, detection_classes, etc. as described here.

I also tried using tf.execute(), but it throws me the following error: UnhandledPromiseRejectionWarning: Error: execute() of TFSavedModel is not supported yet.

I'm using efficientdet/d0 downloaded from here.

ernewston
  • 923
  • 6
  • 22

1 Answers1

1

When you download the tensor using dataSync() it just keeps the value. If you wanted the object with a description of each of the results without the tensors you would just have to console.log(result). Then you expand the result from your log in the browsers console it should return something like this:

Tensor {
  "dataId": Object {},
  "dtype": "float32",
  "id": 160213,
  "isDisposedInternal": false,
  "kept": false,
  "rankType": "2",
  "scopeId": 365032,
  "shape": Array [
    1,
    3,
  ],
  "size": 3,
  "strides": Array [
    3,
  ],
}

The output of your console.log(result) contains 8 tensors within it which shows that it is correct. You are looping over each of the results and each of the outputs should follow this format :

['num_detections', 'detection_boxes', 'detection_classes', 'detection_scores', 'raw_detection_boxes', 'raw_detection_scores, 'detection_anchor_indices', 'detection_multiclass_scores']
yudhiesh
  • 6,383
  • 3
  • 16
  • 49
  • 1
    This helped me a lot, thank you! Some considerations: the output tensors in the array were in a different order (you can find the right order using `getMetaGraphsFromSavedModel()` or looking for the matching shapes of the tensors). For example, `detection_anchor_indices` was the first one. Afterwards, I had to process the result by hand, e.g. doing `const scoreN = result[4].dataSync()[N]`, and then normalize all the data in an array, find the class names in a json, draw the boxes on the image, export the image, etc. Now I can see the results visually :D Thank you again – ernewston Mar 25 '21 at 03:00
  • can you please expand your original answer with a code snipet of how you normalized all the data in an array and found the class names? I only need to feed my model an image and get as an output the detection result in text. I followed this to train a custom object detector model https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/training.html – user 007 Aug 23 '21 at 13:32
  • @user007 oh wait so the array is actually from the official documentation I think. If you go [here](https://tfhub.dev/tensorflow/efficientdet/d0/1) under outputs it mentions it. – yudhiesh Aug 23 '21 at 13:39
  • hey again. Thank you for the link. I understood what each tensor in the output refers to, but I don't know how to convert and associcate those numbers to a response in this format: detectedObjects = [{object: 'cat', score: 70%}]. Any idea on how I can achieve that? @ernewston mentioned he normalised the data and found the class names. and I was wondering how to find the class names from those numbers in 'detection_classes' and the correct scores in 'detection_scores' – user 007 Aug 24 '21 at 09:13
  • If I remember well, I iterated using an index to get each value from each array, and then made an array of "prettier" objects having "class_id", "class_name", "score", etc. To get the class name, I had a json file that mapped a class id with a name. Something like a list that is shown here: https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/. – ernewston Aug 27 '21 at 01:48