tensorflow js model (converted from Keras) has 3 outputs. How can I extract them?

Question

I have trained a yolov3 model in Keras, and it has 3 outputs (3D tensors). No problem there. Then I converted this model into tfjs to run it in a browser. I can easily get the content of each one of the outputs if I truncate the model to this specific output. But I cannot get the full (3 outputs) at once. I am wondering if that is even possible?

Here's what I'd like to do (it does not work, it hangs):

const myTracker = await tf.loadLayersModel(v3_model); 
const prediction = tf.tidy(() => {

      // prepare inputs tensor
      const inputs = tf.browser.fromPixels(canvas, 3).expandDims(0).toFloat().div(tf.scalar(255));
      console.log("============ inputs tensor shape:" + inputs.shape);  //--> 1,416,416,3  

      // get all 3 outputs
      const outputs = myTracker.predict(inputs).arraySync(); 
      outputs.print();
      return outputs; 
});

And here's what works if I am only interested in 1 output, after identifying the 3 outputs names.

const myTracker = await tf.loadLayersModel(v3_model); 
const prediction = tf.tidy(() => {

      // prepare inputs tensor
      const inputs = tf.browser.fromPixels(canvas, 3).expandDims(0).toFloat().div(tf.scalar(255));
      console.log("============ inputs tensor shape:" + inputs.shape);  //--> 1,416,416,3  

      // get the full object of interesting layers
      const layer3 = myTracker.getLayer('conv2d_3');
      const layer8 = myTracker.getLayer('conv2d_8'); 
      const layer13= myTracker.getLayer('conv2d_13'); 

      // get 1 specific output from a new model build from the original
      const myTracker_truncated = tf.model({inputs: myTracker.inputs, outputs: layer3.output});
      const outputs = myTracker_truncated.predict(inputs).arraySync();  
      return outputs; 
});

If I inpect (console.log(myTracker)) I get this structure for my outputs:

outputNames: (3) […]
0: "conv2d_3
1: "conv2d_8"
2: "conv2d_13"
length: 3

outputs: (3) […]
0: Object { dtype: "float32", id: 662, originalName: "conv2d_3/conv2d_3", … }
1: Object { dtype: "float32", id: 665, originalName: "conv2d_8/conv2d_8", … }
2: Object { dtype: "float32", id: 668, originalName: "conv2d_13/conv2d_13", … }
length: 3

Would anybody know if what I am trying to achieve is feasible (short of concatenating the 3 outputs into 1 in the original keras model) ?

score 1 · Answer 1 · answered Feb 20 '20 at 13:14

1

The thee outputs can be obtained by using an array of outputs when creating the model with tf.model

tf.model({inputs: myTracker.inputs, outputs: [layer3.output, layer8.output, layer13.output, ...]});

prediction = myTracker_truncated.predict(inputs).arraySync()

now prediction will be an array of three values respectively to the layers defined in outputs

answered Feb 20 '20 at 13:14

edkeveked

17,989
10
55
93

1

Yes, it's a more elegant solution than concatenation. In Keras, I just have to "stack" my 3 output tensors with an additional Keras layer (tf.keras.backend.stack((a, b,c)) where a,b,c are the 3 outputs. In JavaScript, they become accessible as an array of 3 tensors. – Gill Mac Mar 05 '20 at 11:24

tensorflow js model (converted from Keras) has 3 outputs. How can I extract them?

1 Answers1