1

Description of the "issue":

I want to use keypoints (a.k.a. tie points) between two successive images from an Apple smartphone using ARKit but I can't find these.

I can find 3D values in world reference frame from the ARPointCloud or rawFeaturePoints but I cannot find 2D values (i.e. in the image reference frame) for each images of the pair where they were actually detected (probably using some modified SIFT detector or whatever algorithm... in fact I'd like to know which algo is used aswell).

Question:

Do you know in which object they are stored or how I can retrieve them?
I'd like to reproject them onto the images taken by the camera in an other software (python, scikit-image, or even opencv) to do some processing.

swiss_knight
  • 5,787
  • 8
  • 50
  • 92

1 Answers1

0

Whatever algorithm ARKit uses to generate feature points is internal and private to ARKit. As such, any intermediary results are equally hidden from public API, and both the algorithm and results are subject to change between iOS releases or across different devices.

rickster
  • 124,678
  • 26
  • 272
  • 326
  • 1
    Ok. But the 2D points are drawn onto the device, so they should be accessible somwhere I think, and it's should not be a big secret to share a bunch of (X,Y) coordinates in image space for each image, is it? If yes; is it possible to retrieve these 2D coordinates in another way? If no; I wonder how people could develop personalized application...? – swiss_knight Sep 10 '18 at 06:20