Efficient way of threading an octree such that the pointers contained by each octcell in an oct make it easy in the traversal through the tree at the same level.
We have to make use of fully threaded trees here so that i can use openmp to parallelize the code at the same level.
I have some experience with oct-trees and coded several myself. The basic problem is that the tree has (at least) two directions of traversal: horizontal (between daughter cells) and vertical (between mother and daughter cells), which cannot be mapped to linear memory. Thus, traversing the tree (for example for neighbour search) will inevitably result in cache misses.
For a most efficient implementations, you should have all (up to 8) daughter cells of a non-final cell to be in one contiguous block of memory, avoiding both cache misses when traversing over them and the need to link them up with pointers. Each cell then only need one pointer/index for their first daughter cell and, possibly (depending on the needs of your applications), a pointer to their mother cell.
Similarly, any particles/positions sorted by the tree should be ordered such that all contained within a cell are contiguous in memory, at all tree levels. Then each cell only has to store the first and number of particles, allowing access to all them at at every level of the tree (not just final cells).
In practice, such an ordering can be achieved by first building a fully linked tree and then mapping it to the form described above. The overhead of this mapping is minor but the gain in a applications substantial.
Finally, when re-building the tree with only slightly changed particle positions, it makes for a significant speed up (depending on your algorithm) to feed the particles in the previous tree order to the tree building algorithm.
