Pose keypoints numpy averaging

Question

I know you're supposed to give examples when you ask questions here, but I can't really think of anything that wouldn't involve pasting a massive project worth of code, so I'll just try to describe this as well as possible.

I'm working on a project that involves using keypoints generated by using OpenPose (after I've done some preprocessing on them to simplify everything, I come up with data formatted like this: [x0, y0, c0, x1, y1, c1...], where there are 18 points total, and the x's and y's represent their coordinates, while the c's represent confidence.) I want to take a nested list that has the keypoints for a single person listed in the above manner for each frame, and output a new nested list of lists, made up of the weighted average x's and y's (the weights would be the confidence values for each point) along with the average confidences by each second (instead of by frame), in the same format as above.

I have already converted the original list into a 3-dimensional list, with each second holding each of its frames, each of which holds its keypoint list. I know that I can write code myself to do all of this without using numpy.average(), but I was hoping that I wouldn't have to, because it quickly becomes confusing. Instead, I was wondering if there were a way I could iterate over each second, using said method, in a reasonably simple manner, and just append the resulting lists to a new list, like this:

out = []
for second in lst:
  out.append(average(second, axis=1, weights=?, other params?))

Again, I'm sorry for not giving an example of some sort.

Answer 1

Maybe you could get some inspiration from this code:

import numpy as np

def pose_average(sequence):
    x, y, c = sequence[0::3], sequence[1::3], sequence[2::3]
    x_avg = np.average(x, weights=c)
    y_avg = np.average(y, weights=c)
    return x_avg, y_avg

sequence = [2, 4, 1, 5, 6, 3, 5, 2, 1]
pose_average(sequence)
>>> (4.4, 4.8)

For multiple sequences of grouped poses:

data = [[1, 2, 3, 2, 3, 4, 3, 4, 5], [1, 2, 3, 4, 5, 6, 7, 8, 9], [4, 1, 2, 5, 3, 3, 4, 1, 2]]

out = [ pose_average(seq) for seq in data ]
out
>>> [(2.1666666666666665, 3.1666666666666665),
(5.0, 6.0),
(4.428571428571429, 1.8571428571428572)]

Edit

By assuming that:

• data is a list of sequence
• a sequence is a list of grouped poses (for example grouped by seconds)
• a pose is the coordinates of the joins positions: [x1, y1, c1, x2, y2, c2, ...]

the slightly modified code is now:

import numpy as np

data = [
        [[1, 2, 3, 2, 3, 4, 3, 4, 5], [9, 2, 3, 4, 5, 6, 7, 8, 9], [4, 1, 2, 5, 3, 3, 4, 1, 2], [5, 3, 4, 1, 10, 6, 5, 0, 0]],
        [[6, 9, 11, 0, 8, 6, 1, 5, 11], [3, 5, 4, 2, 0, 2, 0, 8, 8], [1, 5, 9, 5, 1, 0, 6, 6, 6]],
        [[9, 4, 7, 0, 2, 1], [9, 4, 7, 0, 2, 1], [9, 4, 7, 0, 2, 1]]
        ]


def pose_average(sequence):
    sequence = np.asarray(sequence)
    x, y, c = sequence[:, 0::3], sequence[:, 1::3], sequence[:, 2::3]
    x_avg = np.average(x, weights=c, axis=0)
    y_avg = np.average(y, weights=c, axis=0)
    return x_avg, y_avg

out = [ pose_average(seq) for seq in data ]
out
>>> [(array([4.83333333, 2.78947368, 5.375     ]),
array([2.16666667, 5.84210526, 5.875     ])),
(array([3.625, 0.5  , 1.88 ]), array([6.83333333, 6.        , 6.2   ])),
(array([9., 0.]), array([4., 2.]))]

x_avg is now the list of x position averaged over the sequence for each point and weight by c.