How to avoid DirectoryOperationException: The Server Is Busy when USNChange Poll-Synchronizing an AD LDS directory

Question

We are running a .NET 4.5 console application that performs USNChanged polling on a remote LDAP server and then synchronizes the records into a local AD LDS on Windows Server 2008R2. The DirSync control was not an option on the remote server but getting the records isn't the problem.

The directory is quite large, containing millions of user records. The console app successfully pulls down the records and builds a local cache. It then streams through the cache and does lookup/update/insert as required for each record on the local directory. The various network constraints in the environment had performance running between 8 and 80 records per second. As a result, we used the Task Parallel Library to improve performance:

var totalThreads = Environment.ProcessorCount *2;
var options = new ParallelOptions { MaxDegreeOfParallelism = totalThreads };
Parallel.ForEach(Data.ActiveUsersForSync.Batch(250), options, (batch, loopstate) =>
    {
        if (!loopstate.IsExceptional
            && !loopstate.IsStopped
            && !loopstate.ShouldExitCurrentIteration)
            {
                ProcessBatchSync(batch);
            }
     });

After introducing this block, performance increased to between 1000 and 1500 records per second. Some important notes:

1. This is running on an eight core machine so it allows up to 16 operations simultaneously Environment.ProcessorCount * 2;
2. The MoreLinq library batching mechanism is used so each task in the parallel set is processing 250 records on a given connection (from pool) before returning
3. Each batch is processed synchronously (no additional parallelism)
4. The implementation relies on System.DirectoryServices.Protocols (Win32), NOT System.DirectoryServices (ADSI)

Whenever a periodic full synchronization is executed, the system will get through about 1.1 million records and then AD LDS returns "The Server Is Busy" and the system throws a DirectoryOperationException. The number it completes before erroring is not constant but it is always near 1.1 million.

According to Microsoft (http://support.microsoft.com/kb/315071) the MaxActiveQueries value in AD LDS is no longer enforced in Windows Server 2008+. I can't change the value anyway, it doesn't show. They also show the "Server is Busy" error coming back only from a violation of that value or from having too many open notification requests per connection. This code only sends simple lookup/update/insert LDAP commands and requests no notifications from the server when something is changed.

As I understand it, I've got at most 16 threads working in tandem to query the LDS. While they are doing it very quickly, that's the max number of queries coming in in a given tick since each of these are processed single-threaded.

Is the Microsoft document incorrect? Am I misunderstanding another component here? Any assistance is appreciated.