Chunks loading and sort

Question

I working on clone of minecraft and i have 2 problem with chunk loading.

First: Determinate chunks to be loaded.

i found one way it's ugly but works fast for me

1. Define 3d array (array) (size : MAX_CHUNKS_X,MAX_CHUNKS_Y,MAX_CHUNKS_Z)
2. Fill 3d array with FALSE
3. While passing from list of chunks checking if chunk inside a vision range
4. if inside set array[chunk_x][chunk_y][chunk_z] = true;
5. After passing list begin bassing array
6. For all array[chunk_x][chunk_y][chunk_z] == false add to LoadingList chunk at chunk_x chunk_y chunk_z

Another ways to less ugly and still fast ?

Code:

     ChunksRenderList.clear();
    CChunk* Chunk = NULL;

    s32 RootChunk_X_Location = (floor(RenderCenter.x) / CHUNK_SIZE);
    s32 RootChunk_Y_Location = (floor(RenderCenter.y) / CHUNK_SIZE);
    s32 RootChunk_Z_Location = (floor(RenderCenter.z) / CHUNK_SIZE);

    if(RenderCenter.x < 0)
        RootChunk_X_Location--;

    if(RenderCenter.y < 0)
        RootChunk_Y_Location--;

    if(RenderCenter.z < 0)
        RootChunk_Z_Location--;

    core::vector3s RootChunkLocation(RootChunk_X_Location,RootChunk_Y_Location,RootChunk_Z_Location);

    u32 XZ_ArraySide = (RenderDistance_XZ*2)+1;
    u32 Y_ArraySide  = (RenderDistance_Y*2)+1;
    char array[XZ_ArraySide][Y_ArraySide][XZ_ArraySide];

    memset(array,0,(XZ_ArraySide*XZ_ArraySide*Y_ArraySide));

    for(auto it = Chunks.begin(); it != Chunks.end(); it++)
    {
        Chunk = (it->second);

        if(Chunk->Locked)
            continue;

        if(Chunk->KeepAliveCounter <= 0)
        {
            ChunksUnloadList.push_back(Chunk);
            continue;
        }
        else
        {
            Chunk->KeepAliveCounter -= WORLD_UPDATE_PERIOD;
            Chunk->DistanceToCamera = RenderCenter.distance_to(Chunk->ChunkAbsolutePosition);
        }

        if(Chunk->ChunkPosition.x >= (RootChunk_X_Location - (s32)RenderDistance_XZ) && Chunk->ChunkPosition.x <= (RootChunk_X_Location + (s32)RenderDistance_XZ))
            if(Chunk->ChunkPosition.y >= (RootChunk_Y_Location - (s32)RenderDistance_Y) && Chunk->ChunkPosition.y <= (RootChunk_Y_Location + (s32)RenderDistance_Y))
                if(Chunk->ChunkPosition.z >= (RootChunk_Z_Location - (s32)RenderDistance_XZ) && Chunk->ChunkPosition.z <= (RootChunk_Z_Location + (s32)RenderDistance_XZ))
                {
                    s32 PositionInMatrix_X = Chunk->ChunkPosition.x - (RootChunk_X_Location - (s32)RenderDistance_XZ);
                    s32 PositionInMatrix_Y = Chunk->ChunkPosition.y - (RootChunk_Y_Location - (s32)RenderDistance_Y);
                    s32 PositionInMatrix_Z = Chunk->ChunkPosition.z - (RootChunk_Z_Location - (s32)RenderDistance_XZ);

                    array[PositionInMatrix_X][PositionInMatrix_Y][PositionInMatrix_Z] = true;

                    Chunk->KeepAliveCounter = CHUNK_LIVE_TIME;
                }


        if(not Chunk->NeightboarsUpdated)
        {
            ChunksNeightboarUpdateList.push_back(Chunk);
        }

        if(not Chunk->ChunkUpdated)
        {
            ChunksRebuildList.push_back(Chunk);
        }
        if(not Chunk->Locked and Chunk->VisibleBlocks > 0)
        {
            ChunksRenderList.push_back(Chunk);
        }

    }

    for(u32 y = 0; y < Y_ArraySide; y++)
        for(u32 x = 0; x < XZ_ArraySide; x++)
            for(u32 z = 0; z < XZ_ArraySide; z++)
            {
                s32 ChunkPosition_X = (s32)x + (RootChunk_X_Location - (s32)RenderDistance_XZ);
                s32 ChunkPosition_Y = (s32)y + (RootChunk_Y_Location - (s32)RenderDistance_Y);
                s32 ChunkPosition_Z = (s32)z + (RootChunk_Z_Location - (s32)RenderDistance_XZ);

                if(array[x][y][z] == 0)
                {

   SPendingToLoad ToLoad;
                    ToLoad.Position.set(ChunkPosition_X,ChunkPosition_Y,ChunkPosition_Z);
                    ToLoad.DistanceToCamera = ToLoad.Position.distance_to_sqr(RootChunkLocation);
                    ChunksLoadList.push_back(ToLoad);
                }
            }

Second: how to sort ChunksLoadList to take effect like left on this pic https://www.dropbox.com/s/owjfaaekcj2m23w/58f2e4c8.png?dl=0 Red = nearest to ChunksLoadList.begin() Blue = farest to ChunksLoadList.begin()

im try to use

    ChunksLoadList.sort([&RootChunkLocation](SPendingToLoad& i,SPendingToLoad& j)
    {

        return i.DistanceToCamera < j.DistanceToCamera;
    }
    );

But it method to slow for big vision ranges... How i must rewrite code to take fast wave-loading effect ?

Sorry me horrible english, i hope you understand me...

Answer 1

Lets first look at the distance sorting problem, if your ChunksLoadList is a std::list and not a std::vector or std::array(C++11) you have lost the performance race already! Bjarne Stroustrup: Why you should avoid Linked Lists Pay close attention to the graph!!!

If its still too slow after you've changed it into a std::vector you can try "this method I just invented(TM)"!

The best sorting algorithms are something like

O(C+K*N log log N) fastest?

With a horrible C constant prep time, horrible K per element and a very nice N log log N

For N -> infinity this gets to be O(N log log N)

BUT for this problem there is an even better algorithm!
Flood fill followed by an insertion sort, the flood fill produces a nearly sorted list in O(N) and the insertion sort secures the totally ordered list from the partially ordered in O(N) for a total of O(N) ...

O(C+K*N)

with a horrible constant prep time, and an awful per element but only N times

variant of wikipedia

 Flood-fill (node, target-color, replacement-color): 

 If target-color is equal to replacement-color, return.
 Set Q to the empty queue. [must be std::vector or std::array or this will fail]
 Add camera node to the end of Q.
 While Q is not empty: 
     Set n equal to the *first* element of Q.
     Remove *first* element from Q.
     If the color of n is equal to target-color:
         Add n to the distance list as the next closed (this will be nearly correct)
         Set the color of n to replacement-color and mark "n" as processed.
         Add adjacent nodes to end of Q if they has not been processed yet. (x/y/z +1/-1)
 Return.

Queue elements are x,y,z
use std::dequeue

The distance list must also be a random access contain, that is fully allocated from start of size (viewdistance*2+1)^3, that is potentially big.
If view distance 100 is 201^3 = ~80000000 voxels, did you really want this? if you need some info from it you must have some pointer or index, at least 4 bytes, this blows the cache away on most systems.

As a flood fill its not effective but as a approximation to distance it is.
You could stop here if your requirements are fulfilled.
IF you need total ordered then run an insertion-sort on the nearly sorted list O(N), but then you need to calculate the camera distance also.

Potential further optimization:

• opaque voxels doesn't add neighbours that also are opaque.
• air(totally transparent) doesn't add to the camera list but need to be there for the fill, in case a flying island is present.