1

I am trying to work with the Google Project Tango tablet, I currently am sending data from my tablet to Epic's Unreal Engine 4 and trying to visualize it properly. The orientation of the cloud data however is having consistent issues lining up with the orientation of the Tango's pose data.

Currently I am taking the pose from the same frame as the xyzIj data, but the resulting point clouds never line up quite right. I tried doing manual rotation and correction of the cloud data so that it matched up with the pose data, however the solution to rotating one set of xyzIj data does not correct the remaining sets either.

I am currently under the assumption that the poses I am using to match these sets are in error, but I am unsure as to how to find the correct one. I also find that using getPoseAtTime tends to produce similarly flawed results, but that may have something to do with the fact that I don't quite understand it's usage.

Is there a solution to this issue outside of hand correcting each data set that I have failed to realize, or does everyone deal with similar issues?

Jason Ryan
  • 69
  • 6

2 Answers2

1

Lets start with a few basics

1) Have you created, and subsequently loaded an ADF before capturing data ?

2) Are you capturing poses relative to the ADF or the Session sart - the former is more accurate

3) There shouldn't be too much drift in rotation - those sensors are reasonably accurate - what looks like rotational drift is either positional error or using the wrong pose

4) Make sure you have an operating mode that lets you focus on stable data, i.e. getting point sets when the camera is not in motion - that's a good way to work out whether its bad data from Tango or issues with your matrix chain

Mark Mullin
  • 1,340
  • 1
  • 9
  • 21
  • As I haven't loaded an ADF, I can assume the answer to #2 would be that I am capturing from session start. Is the accuracy loss that I take from not having one loaded large enough to get a near 45 degree misplacement on most of the xyzIj sets found? – Jason Ryan Feb 20 '15 at 17:09
  • Drift accumulates so you really want to use an ADF - the error seems excessive though - if the error is immediate and constant, I would suspect something in your matrix chain – Mark Mullin Feb 20 '15 at 17:44
  • I have spent a few hours on this issue now, I have ruled out the matrix chain, and I have tested several ADF against earlier shots without noticable improvement, I have found that occasionally the pose hiccups and can give a huge error in positional data(which is easily solved), but the orientations all seem to still be off, it also appears to afflict all pose data in my recordings, not just the ones I am using to place the cloud data. At this point I am going to have to rule it as tango error, but I would love it if there was just something I have overlooked. – Jason Ryan Feb 23 '15 at 23:31
  • Marking this answered, the information you provided me was extremely useful. Thank you. – Jason Ryan Mar 09 '15 at 17:09
  • Hello @JasonRyan, I am experiencing a similar issue like yours. My test data is a 360-degree scan, and by applying those pose transformations on a single test point (0.0, 1.0, 0.0) in camera space, transformed points would correctly fall on a circle. But when applying the same transformations on point clouds, the results are indeed some 45-degree rotations short for each of them. Have you figured out where the offsets in rotation come from in your case? – Dale Z Mar 27 '15 at 15:41
  • 1
    Hey @DaleZ The offsets were occuring because of the manner I was using the ADF, and due to the timing between pose data and pointcloud data. In all honesty I am a bit fuzzy on the details as my work has taken a bit of a turn. Mark's answer is the most solid information you wilol probably get, The matrix chain is super important though, so be sure your test point is accounting for orientation as well as position. – Jason Ryan Apr 02 '15 at 23:41
  • 1
    For our in house solution we would literally mark every key frame with a set of arrows indicating up, right, and forward. We even built a rail that pushed a virtual camera along all of the pose data just to make sure it didn't have any outliers (which happen from time to time) – Jason Ryan Apr 02 '15 at 23:41
  • Thank you @JasonRyan. Yes, using markers is very helpful in debugging. Also, my ~45-degree error comes from confusing the device camera frame with OpenGL camera frame, which are different. But correcting this is not enough - The drifting issue still needs to be addressed. I posted a few tips that work in my case in [another thread](http://stackoverflow.com/questions/29266284/project-tango-converting-between-coordinate-systems-and-merging-point-clouds/29439873#29439873) and hope this helps too. – Dale Z Apr 03 '15 at 21:26
0

Can't append to that answer with this data

You may be right about hiccups - this is a grab from my database, with data that Tango Tricorder shipped up - I think my software is OK, but you never know - it seems that Tango hiccuped on start, but then at the end things got going ok - maybe this will help you with a heuristic - the data you are looking at is effectively the raw pose records.

On another note, I do suspect your matrix chain - I do see drift and anomalies, but I can't per se claim tango is bad at math - I will be publishing a writeup of the matrix chains I use across 3 platforms and support libraries (Android, Windows,OpenGL, raw .net math classes,WebGL) on the G+ page in the next few days - in any case, have some example data of what I think was a hiccup 

Device  At  Timestamp   Quality PX  PY  PZ  AX  AY  AZ  AW  ADF
e75de7b04768f83a    2015-02-15 23:38:08.0910000 1211.9930910005 0   0   0   0   -6.72623262875912E-44   0   -2.21596756341829E-36   -2.21603644004041E-36   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:08.6520000 1212.5919690005 0   0   0   0   -6.72623262875912E-44   0   -2.35768121342709E-36   -2.35775009004921E-36   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:08.9690000 1212.9246790005 0   0   0   0   0   0   -4.46572875271279E-33   -4.46586760798299E-33   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:09.6010000 1213.5568280005 0   0   0   0   0   0   -8.56151098540779E-38   -8.5617228617356E-38    00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:09.9590000 1213.8895380005 0   0   0   0   0   0   -2.35768228962431E-36   -2.35775009004921E-36   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:10.5790000 1214.5216870005 0   -5.04611733348779E-33   -6.72623262875912E-44   -5.04611733348779E-33   0   0   0   -7.09253042359596E-30   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:10.9350000 1214.8543970005 0   0   0   0   -6.72623262875912E-44   0   -4.46572654866089E-33   -4.46586760798299E-33   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:11.2480000 1215.1871070005 0   0   0   0   0   0   -8.71705399390908E-38   -8.7172692333532E-38    00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:11.6140000 1215.5198170005 0   0   0   0   -6.72623262875912E-44   0   -4.46572654866089E-33   -4.46586760798299E-33   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:11.9170000 1215.8525270005 0   0   0   0   -6.72623262875912E-44   0   -4.46572654866089E-33   -4.46586760798299E-33   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:12.2280000 1216.1852370005 0   0   0   0   0   0   -2.35768228962431E-36   -2.35775009004921E-36   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:12.6080000 1216.5179470005 0   -7.49758949629256E-33   -6.72623262875912E-44   -7.49758949629256E-33   0   0   0   -7.09253042359596E-30   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:12.8930000 1216.8506570005 0   -7.50624113471461E-33   -6.72623262875912E-44   -7.50624113471461E-33   -7.5062367266108E-33    0   -7.50624113471461E-33   -7.09253042359596E-30   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:13.2440000 1217.1833670005 0   0   0   0   -6.72623262875912E-44   0   -4.46572654866089E-33   -4.46586760798299E-33   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:13.5540000 1217.5160770005 0   0   0   0   -2.11694608368399E-30   0   -2.35768211025811E-36   -2.35775009004921E-36   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:13.9260000 1217.8487870005 0   -7.46970677029106E-33   -6.72623262875912E-44   -7.46970677029106E-33   0   0   0   -7.09253042359596E-30   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:14.2170000 1218.1814970005 0   -6.23929332339835E-33   -6.72623262875912E-44   -6.23929332339835E-33   -6.23928891529454E-33   0   -6.23929332339835E-33   -7.09253042359596E-30   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:14.5710000 1218.5142070005 0   0   0   0   0   0   -8.72060880785337E-38   -8.72082068418118E-38   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:14.9260000 1218.8469170005 0   0   0   0   -6.72623262875912E-44   -3.30428292727447E-06   -8.56150762229148E-38   -8.5617228617356E-38    00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:15.2410000 1219.1796270005 0   0   0   0   -6.72623262875912E-44   0   -2.27243491411843E-36   -2.27250379074055E-36   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:15.5500000 1219.5123370005 0   0   0   0   -6.72623262875912E-44   0   -8.72060544473706E-38   -8.72082068418118E-38   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:15.8870000 1219.8450470005 0   0   0   0   -6.72623262875912E-44   0   -2.35768121342709E-36   -2.35775009004921E-36   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:16.2580000 1220.1777570005 0   0   0   0   -6.72623262875912E-44   0   -8.71705399390908E-38   -8.7172692333532E-38    00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:16.5740000 1220.5104670005 0   0   0   0   0   0   -2.21596863961551E-36   -2.21603644004041E-36   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:16.9130000 1220.8431770005 0   0   0   0   0   0   0   0   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:17.2580000 1221.1758870005 0   0   0   0   -6.72623262875912E-44   0   -4.46572654866089E-33   -4.46586760798299E-33   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:17.5860000 1221.5085970005 0   0   0   0   0   0   -2.21596863961551E-36   -2.21603644004041E-36   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:17.9240000 1221.8745780005 0   0   0   0   -6.72623262875912E-44   0   0   0   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:18.2900000 1222.2072880005 0   0   0   0   -6.72623262875912E-44   0   -4.46572654866089E-33   -4.46586760798299E-33   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:18.5980000 1222.5399980005 0   0   0   0   -6.72623262875912E-44   0   0   0   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:18.6860000 1222.783591 0   -0.489519119262695  -2.58179521560669   0.296925604343414   0.695962131023407   0.607802748680115   0.237487226724625   0.299687057733536   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:18.7360000 1222.816918 0   -0.509992599487305  -2.66766571998596   0.299138188362122   0.695954203605652   0.605366230010986   0.240672022104263   0.302086800336838   00000000-0000-0000-0000-000000000000
e75de7b04768f83a    2015-02-15 23:38:18.8280000 1222.916901 1   -0.242476642131805  -2.10855436325073   0.997836887836456   0.706526875495911   0.588680744171143   0.261307954788208   0.293245434761047   00000000-0000-0000-0000-000000000000
Mark Mullin
  • 1,340
  • 1
  • 9
  • 21