0

I am working on an application with an huge amount of different three-dimensional data. The 3 dimensional data are relative small (like 100 x 100 x 1000) but likely millions of these objects. Now I wonder if anyone has experience dealing such data in breeze. Although I can use nested datastructures like a matrix of vectors, it is important to address single values of that structure by indexing (x,y,z). Is it better to define a own structure like Point3d(x,y,z) - but x,y,z are vectorsa itself - or use predefined breeze classes like DenseMatrix. My question is how the performance is affected by those alternatives.

Thanks for your replies Rolf-Dieter

zero323
  • 322,348
  • 103
  • 959
  • 935
kumaro
  • 29
  • 3
  • What do you want to achieve? For simple indexing, breeze may not be what you want, even apache-sparc may may be overkill for a few million objects, depending on your requirements... – TilmannZ Jul 02 '16 at 18:21

1 Answers1

0

In my experience, for performance, the simpler the object the better. That means using only primitive type, no nested objects, etc... Simple objects are faster to serialize and are smaller so you can pack more of them into memory.

In your cases, I think using one 9-element tuple is better than 3 3-element tuple.

(x1, x2, x3, y1, y2, y3, z1, z2, z3)
is better than
((x1, x2, x3), (y1, y2, y3), (z1, z2, z3))
Kien Truong
  • 11,179
  • 2
  • 30
  • 36
  • Thanks Dikei, I did it as you proposed - a very big flat map. It needs some logic to extract the right values for processing, but has good performance I see. – kumaro Jul 08 '16 at 15:31