0

I'm trying to get the human pose information on low-resolution images. Particularly I've tried Keras OpenPose implementation by michalfaber, but the model seems to not perform well on low-resolution images while performing pretty well on higher resolution. I posted a question as an issue on GitHub repo as well but I thought I'd try here as well as I'm not set on that particular implementation of human pose detection.

My images are about 50-100 pixels width and height wise. This is an example of the image. I wonder if anyone knows a way to modify the program, network, or knows of a human pose network that performs well on such low-resolution images.

enter image description here

Mike Azatov
  • 402
  • 6
  • 22

1 Answers1

1

If you are looking for a different human pose estimation network , I would highly recommend the MxNet GluonCV framework (https://gluon-cv.mxnet.io/model_zoo/pose.html). It is very simple to use and also contains many different pose estimation networks that you can try and compare tradeoff between accuracy and speed. For example, to use it you can do (Taken from the tutorial page):

from matplotlib import pyplot as plt
from gluoncv import model_zoo, data, utils
from gluoncv.data.transforms.pose import detector_to_alpha_pose, heatmap_to_coord_alpha_pose

detector = model_zoo.get_model('yolo3_mobilenet1.0_coco', pretrained=True)
pose_net = model_zoo.get_model('alpha_pose_resnet101_v1b_coco', pretrained=True)

# Note that we can reset the classes of the detector to only include
# human, so that the NMS process is faster.

detector.reset_class(["person"], reuse_weights=['person'])

im_fname = utils.download('https://github.com/dmlc/web-data/blob/master/' +
                          'gluoncv/pose/soccer.png?raw=true',
                          path='soccer.png')
x, img = data.transforms.presets.yolo.load_test(im_fname, short=512)
print('Shape of pre-processed image:', x.shape)

class_IDs, scores, bounding_boxs = detector(x)

pose_input, upscale_bbox = detector_to_alpha_pose(img, class_IDs, scores, bounding_boxs)

predicted_heatmap = pose_net(pose_input)
pred_coords, confidence = heatmap_to_coord_alpha_pose(predicted_heatmap, upscale_bbox)

For accuracy comparison for example, their AlphaPose with Resnet 101 network is significantly more accurate than OpenPose (You can find more accuracy benchmarks from the link above). A caveat, is however understanding the difference between types of these networks such as implementing Bottom-Up and Top-Down approach since it can affect the inference speed at different scenarios.

For example, the runtime of the top-down approaches is proportional to the number of detected people, it can be time-consuming if your image has a crowd of people.

velociraptor11
  • 576
  • 1
  • 5
  • 10
  • Thanks, any ideas on implementations in Keras, Tensorflow? I am going to tweak the code and am only comfortable with Keras-Tensorflow enough to do it – Mike Azatov Jan 06 '20 at 19:43
  • Yeah I don't know any similar libraries that provides various implementation of pose estimation in Tensorflow and Keras. May I know why and which part of the code you are interested in tweaking? Are you trying to take chunks of the original implementation for your own novel neural network or just looking to improve pose predictions? – velociraptor11 Jan 06 '20 at 20:20
  • Sure, I'm interested in poses themselves as well as other parameters. Specifically, besides poses I'm interested in body orientation. I was planning to take the features from the last layer of pose estimator and plug them into my neural net that calculates body orientation. – Mike Azatov Jan 06 '20 at 20:47
  • I'm doing something similar for a project where I'm using the key points prediction from the poses and feeding it into another neural network. However, I treat the problem as two different neural networks. So training is separate. Are you planning to do a single-joint training or is two separate neural network fine for you? So I'm guessing you are planning to freeze the poses weights and only update weights on your "new" part of the neural network? – velociraptor11 Jan 06 '20 at 21:01
  • Yeah that was my plan. Make one big network and freeze all the layers from the pose network – Mike Azatov Jan 06 '20 at 21:23
  • Hmm I see, well I guess if you are not constrained with having two networks it's worth giving it a try. I've always been a Tensorflow person but when I found out about this library and the significantly better accuracy on benchmark datasets (also MIT license whereas OpenPose is not) I just had to use it. Goodluck – velociraptor11 Jan 06 '20 at 21:47