4

I am getting my hands dirty with the new ARKit2 object scanning and recognition tool that was part of WWDC 2018 Demos. All details can be found here.

https://developer.apple.com/documentation/arkit/scanning_and_detecting_3d_objects

Once the scanning process is completed this app exports the scanned object file as arobject to iOS files app.

I cannot find answers on apple sites or developer forums on the following:

  1. How to display the arobject files? Clearly the files app cannot understand the arobjects file extension.
  2. How can we store the files as usdz format from scanner app to allow for the new quick look feature to be used with iOS file app (as shared in the WWCD 2018 Demos)?
Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
Ahmad Al-Baqawi
  • 189
  • 1
  • 7

1 Answers1

8

As noted in Apple's Scanning and Detecting 3D Objects doc:

An ARReferenceObject contains only the spatial feature information needed for ARKit to recognize the real-world object, and is not a displayable 3D reconstruction of that object.

An .arobject file is simply an instance of the ARReferenceObject class in serialized filesystem form, so this advice applies to both.

There's nothing displayable about a reference object — it's opaque data representing whatever magic ARKit remembers upon scanning an object that helps it recognize that object later. It's not equivalent to USDZ, SCN, OBJ, ABC, DAE, FBX, STL or other 3D asset formats — those describe polygon meshes, shader features, and other attributes relevant to rendering realistic virtual objects and/or 3D printing.

One thing you can do with an ARReferenceObject is give it a 2D preview image when saving one — see export(to:previewImage:). This gets displayed in Xcode, but it doesn't look like QuickLook in macOS Finder or iOS Files know how to display it, and it's not documented how another app is supposed to read that from the file. (Those would be good things to file bugs with Apple about.)

rickster
  • 124,678
  • 26
  • 272
  • 326
  • Thanks for explaining this concept. But do you know if there is a way to convert a DAE, OBJ, SCN etc into an .arobject format? Many thanks. – Chalikov Jan 24 '19 at 22:14
  • That’s the point — there *is no basis* for equivalence or conversion between a 3D art asset intended for visual display and the information needed to recognize a real-world object. “How to draw a picture of a ham sandwich” is not “how to make a ham sandwich”, much less “how to eat a ham sandwich”, so you can’t just ask a computer to change one of those into another. The closest thing to a conversion procedure from OBJ/DAE/etc would be: 1. Transform the art to something acceptable by a 3D printer; 2. Make a 3D print; 3. Scan it using the AR Scanner sample app linked above to get an `.arobject`. – rickster Jan 24 '19 at 23:19
  • 1
    I understand that, but there is a nuance. When scanning an object (with arkit scanner) it forms point clouds to be able to recognise it right? And basically the process of scanning is very similar to photogrammetry (a reason why I still think it might be somehow possible). I found an app called Qlone that does a fantastic job at creating those 3d models, it would be amazing if they would allow exporting .arobject format. P.S. Unity has an object scanning demo app that does the exact same thing as ARKit scanner and imo it does it better (I have tested both). – Chalikov Jan 25 '19 at 14:28
  • @Chalikov Have you found a solution to convert 3D asset to .arobject? – Kawe Jul 01 '21 at 06:58