Fastest path on a flat map

Question

I'm trying to make units farm wood and then bring it back to the camp. I'm using a randomly generated map contained in a 2-dimensional array. See here : Randomly generate blocks on a flat map

I do some kind of BFS I wrote and I keep a 2-dimensional array of predecessors and I also use a 2-dimensional array for marking the positions I already checked. When I found the block (wood) I'm searching for, I create the fastest path using the predecessor array.

This code works well until the number of units reaches about 100 ( it also gets slower when wood is far away ).

With 50 units :

With 100 units I get small freezes :

Here is the "BFS" :

void findPathTo<T>() {
    Queue q = new Queue();
    // Set position
    int x = (int)transform.position.x;
    int y = (int)transform.position.z;
    // Set marked array to false
    for (int i = 0; i < gm.mapSizex; i++) {
        for (int j = 0; j < gm.mapSizey; j++)
            marked[i][j] = false;
    }
    // Set predecessor array to -1
    for (int i = 0; i < gm.mapSizex; i++) {
        for (int j = 0; j < gm.mapSizey; j++) {
            for (int k = 0; k < 2; k++)
                predecessor[i][j][k] = -1;
        }
    }
    // Add element adjacent to current pos
    marked[x][y] = true;
    int[,] pos = new int[4,2] { { 0, 1 }, { 0, -1 }, { -1, 0 }, { 1, 0 } };
    for (int i = 0; i < 4; i++) {
        int[] l1 = { x - pos[i,0], y - pos[i,1] };
        if (l1[0] < gm.mapSizex && l1[1] < gm.mapSizey && l1[0] >= 0 && l1[1] >= 0 && !marked[l1[0]][l1[1]]) {
            q.Enqueue(l1);
            predecessor[l1[0]][l1[1]][0] = x;
            predecessor[l1[0]][l1[1]][1] = y;
        }
    }
    // For each added element 
    while (q.ToArray().Length != 0) {
        int[] l = (int[])q.Dequeue();
        if (!marked[l[0]][l[1]]) {
            marked[l[0]][l[1]] = true;
            // If the block we're searching is found create the path and exit
            if (gm.map[l[0]][l[1]] != null && gm.map[l[0]][l[1]].GetType() == typeof(T)) {
                dest = (ObjectInterface)gm.map[l[0]][l[1]];
                if(gm.map[l[0]][l[1]].isFarmable())
                    gm.map[l[0]][l[1]] = null;
                int i = l[0];
                int j = l[1];
                // Set the path
                while (i >= 0 && j >= 0 && predecessor[i][j][0] != -1 && predecessor[i][j][1] != -1) {
                    path.Push(predecessor[i][j]);
                    i = predecessor[i][j][0];
                    j = predecessor[i][j][1];
                }
                int[] next = (int[])path.Pop();
                destX = next[0];
                destY = next[1];
                break;
            }
            // If we can move on this surface add unmarked adjacent blocks
            if (gm.map[l[0]][l[1]] == null) {
                for (int i = 0; i < 4; i++) {
                    int[] l1 = { l[0] - pos[i, 0], l[1] - pos[i, 1] };
                    if (l1[0] < gm.mapSizex && l1[1] < gm.mapSizey && l1[0] >= 0 && l1[1] >= 0 && !marked[l1[0]][l1[1]]) {
                        q.Enqueue(l1);
                        predecessor[l1[0]][l1[1]][0] = l[0];
                        predecessor[l1[0]][l1[1]][1] = l[1];
                    }
                }
            }
        }
    }
}

The BFS idea can't support a large amount of units so I probably should use A* (I didn't read much about it yet). But I also thought about making one 2-dimensional array (for wood, I'll have to add one more for gold...) which contains the successor to go to the closest wood block, but it will have to be updated very often so it's probably not a good idea.

What do you think ?