Collouring Android Sceneform PointCloud by confidence value

Question

I want to collourize the PointCloud by it's confidence value.

First I create three FloatBuffer, where I put single Points based on a threshold.

private FloatBuffer makeFLoatBuffer(ArrayList<Float> conf) {
    FloatBuffer confFB = FloatBuffer.allocate(conf.size());
    for(Float value : conf){
        confFB.put(value);
    }

    confFB.rewind();
    return confFB;
}

private ArrayList<FloatBuffer> splitPointCloudOnConfidence(PointCloud cloud) {
    ArrayList<Float> conf1 = new ArrayList<>();
    ArrayList<Float> conf2 = new ArrayList<>();
    ArrayList<Float> conf3 = new ArrayList<>();
    ArrayList<FloatBuffer> floatBufferArray = new ArrayList<>();

    FloatBuffer buf_temp = cloud.getPoints();
    int buf_temp_limit = buf_temp.limit();
    for (int i = 0; i < buf_temp_limit / 4; i++) {

        float x = buf_temp.get(i * 4);
        float y = buf_temp.get(i * 4 + 1);
        float z = buf_temp.get(i * 4 + 2);
        float confidence = buf_temp.get(i * 4 + 3);

        if(confidence < 1.0 && confidence > 0.5){
            conf1.add(x);
            conf1.add(y);
            conf1.add(z);
            conf1.add(confidence);
        } else if(confidence < 0.5 && confidence > 0.2) {
            conf2.add(x);
            conf2.add(y);
            conf2.add(z);
            conf2.add(confidence);
        } else {
            conf3.add(x);
            conf3.add(y);
            conf3.add(z);
            conf3.add(confidence);
        }
    }

    floatBufferArray.add(makeFLoatBuffer(conf1));
    floatBufferArray.add(makeFLoatBuffer(conf2));
    floatBufferArray.add(makeFLoatBuffer(conf3));

    return floatBufferArray;
}

I also create three Nodes which are responsible to visualize the PointCloud. For that I used a slightly modified version of this class https://github.com/claywilkinson/arcore-android-sdk/blob/sceneform-samples/samples/cloud_anchor_java/app/src/main/java/com/google/ar/core/examples/java/cloudanchor/sceneform/PointCloudNode.java

pointCloudNodeTestRed = new PointCloudNode_v2(this, color_red);
pointCloudNodeTestRed.setEnabled(true);
pointCloudNodeTestYellow = new PointCloudNode_v2(this, color_yellow);
pointCloudNodeTestYellow.setEnabled(true);
pointCloudNodeTestGreen = new PointCloudNode_v2(this, color_green);
pointCloudNodeTestGreen.setEnabled(true);

fragment.getArSceneView().getScene().addChild(pointCloudNodeTestRed);
fragment.getArSceneView().getScene().addChild( pointCloudNodeTestYellow);
fragment.getArSceneView().getScene().addChild( pointCloudNodeTestGreen);

And in a onUpdateListener I feed the specific data to the three Nodes.

PointCloud pC = frame.acquirePointCloud();

ArrayList<FloatBuffer> bufHolder = splitPointCloudOnConfidence(pC);

pointCloudNodeTestRed.update(pC, bufHolder.get(2));
pointCloudNodeTestYellow.update(pC,  bufHolder.get(1));
pointCloudNodeTestGreen.update(pC,  bufHolder.get(0));

pC.release();

My question in general. Is there a better way to implement such a visualization?

score 1 · Answer 1 · edited Oct 01 '20 at 13:50

My understanding is that SceneForm is no longer being maintained by Google. It would be nice to have a way to do this easily using ARCore and GLES20 as a modified version of PointCloudRenderer.java example file as seen at link below, but with multiple colors instead of one color for all points

https://github.com/google-ar/arcore-android-sdk/blob/54861e38cceb406021fc8ece495f82387376af62/samples/augmented_faces_java/app/src/main/java/com/google/ar/core/examples/java/common/rendering/PointCloudRenderer.java

Collouring Android Sceneform PointCloud by confidence value

1 Answers1