1

I'm working on a Keras neural net that does key point prediction of body parts (left foot, left knee, left hip, etc.). For each image (X), the target (Y) is a list of coordinates for the keypoints (left_foot_x, left_foot_y, left_knee_x, left_knee_y), etc.). Whenever a keypoint isn't visible the X and Y coordinate equal = 0.

I suspect that leaving the data in this state will lead to bad results because 0 really means NA (i.e. the keypoint isn't visible). Leaving the X and Y coordinates 0 would suggest the keypoint is actually in the corner which is false. What is the right way to structure data to do multi target regressions when the correct output is sometimes NA?

megashigger
  • 8,695
  • 17
  • 47
  • 79
  • What about using Python's None type? For example foo = None. foo, in this case, would be an object that has no value. – blackHoleDetector Oct 02 '17 at 03:11
  • If the training data has targets that are equal to None, would during inference the model predict None type as well? That's the behavior I'd like to create.. – megashigger Oct 02 '17 at 03:13
  • I haven't tried training/predicting with None type labels, but I would think that the model should be able to predict Nones as long as Nones were included in your train labels. I did just think about the fact that None types could end up causing issues with calculating the loss on your model. I don't think Keras will be able to calculate the loss if some of your data is labeled with None types. In this case, you may see loss = NaN. Is it possible for you to instead use a number for non-visible key points that's guaranteed not to be used in your data set elsewhere, maybe like -1? – blackHoleDetector Oct 02 '17 at 03:24
  • @blackHoleDetector you could use numbers but my guess is that would skew the results in a way you don't want them to. a value of -1 would be intepreted as being close to 0,1,2,3, etc. and far from larger numbers. but that's not what -1 represents. – megashigger Oct 02 '17 at 03:42
  • Would probably make more sense to use two separate networks, since two separate tasks are being done. That way you're not artificially altering the data to affect the Loss, also I dont think None types work in the way being described – DJK Oct 02 '17 at 18:28
  • @megashigger any progress on the issue? I am facing similar use case ... – Asterisk Oct 30 '17 at 16:57

0 Answers0