How to access Mediapipe Hand Landmarks for Android (Java)

Question

How can the landmarks for hands be accessed in the Android version of MediaPipe? (Java)

I'd like to access the joints' positions in space.

A similar project that someone else has done in Python is available.
https://github.com/GasbaouiMohammedAlAmin/Finger-Counter-using-Hand-Tracking-And-Open-cv
The landmarks seem to be from the "solutions" part of the library.

I have a sample of code that works for Android, but it seems that all the processing is done without ever giving the landmarks to the Java file. https://github.com/jiuqiant/mediapipe_multi_hands_tracking_aar_example Is the base code of my project.

From that file, getMultiHandLandmarksDebugString takes in NormalizedLandmarkList, but how can those landmarks be accessed normally in the program?

import com.google.mediapipe.formats.proto.LandmarkProto.NormalizedLandmark;

score 0 · Answer 1 · answered Aug 19 '22 at 03:16

Here the result is the object of HandsResults

To get the landmark position of each points i have used a list of normalizedlandmark type

List<NormalizedLandmark> ll = result.multiHandLandmarks().get(0).getLandmarkList();

The list size will be 21 to access those i have created a loop

for (int i = 0; i < ll.size(); i++)

For getting x, y and z of each of 21 landmarks

int x = (int) (ll.get(i).getX() * 1280);

Multiplied by 1280 texture width to get the pixel value

Here is the sample code u can check it out for more reference follow this link

List<NormalizedLandmark> ll = result.multiHandLandmarks().get(0).getLandmarkList();
        Log.d("check", String.valueOf(ll.get(0).getX()));
        for (int i = 0; i < ll.size(); i++) {

            int x = (int) (ll.get(i).getX() * 1280);
            int y = (int) (ll.get(i).getY() * 720);
//            Log.d("chec1", String.valueOf(i)+" "+String.valueOf(x)+" "+String.valueOf(y));
            for (int tip : finger_tips) {
                Log.d("chec1", String.valueOf(tip) + " " + String.valueOf((int) (ll.get(tip).getX() * 1280)) + " " + String.valueOf((int) (ll.get(tip).getY() * 720)));
                if ((ll.get(tip).getX() * 1280) < (ll.get(tip - 3).getX() * 1280)) {
                    finger_fold_status.add(false);
                } else {
                    finger_fold_status.add(true);
                }
            }
            int r = (int) (ll.get(thumb_tip-1).getY()*1280);
            if(((int)ll.get(thumb_tip).getY()*1280)<r && r< (ll.get(thumb_tip-2).getY()*1280)){
                if(finger_fold_status.contains(true)){
                    Log.d("res","LIke");
                    is = true;
                    runOnUiThread(new Runnable() {
                        @Override
                        public void run() {
                            tv.setText(R.string.like);
                        }
                    });
                }
            }if(((int)ll.get(thumb_tip).getY()*1280)>r && r>(ll.get(thumb_tip-2).getY()*1280)){
                if(finger_fold_status.contains(true)){
                    Log.d("res1","DisLIke");
                    is = false;
                    runOnUiThread(new Runnable() {
                        @Override
                        public void run() {
                            tv.setText(R.string.dislike);
                        }
                    });
                }
            }
            Log.d("stat", String.valueOf(finger_fold_status));
        }

How to access Mediapipe Hand Landmarks for Android (Java)

1 Answers1