Retrieve All Members of Large AD Groups

Question

Working with an Microsoft Active Directory and Unboundid SDK and there is a group with >29k members.

I am trying to utilize the range values to get all the groups, but can not determine when the end has been reached.

I am using this method: (Updated to working code)

  public static List<String> getAttributeRangeBasedSearch(LDAPConnection ldc, String basedn, String filter, int step, String return_attribute) throws LDAPException
{
List<String> allValues = new ArrayList<String>();
// initialize counter to total the group members and range values
int allvalues = 0;
int start = 0;
// int step = 1000;
int finish = step - 1;
boolean finallyFinished = false;
String range;
// loop through the query until we have all the results
while (!finallyFinished)
{
    range = start + "-" + finish;
    String currentRange = return_attribute + ";Range=" + range;
    String range_returnedAtts[] = { currentRange };
    SearchRequest searchRequest = new SearchRequest(basedn, SearchScope.BASE, filter, range_returnedAtts);
    List<SearchResultEntry> rangedEntries = ldc.search(searchRequest).getSearchEntries();
    for (Iterator<SearchResultEntry> iterator = rangedEntries.iterator(); iterator.hasNext();)
    {
    SearchResultEntry searchResultEntry = iterator.next();
    Collection<Attribute> allAttribute = searchResultEntry.getAttributes();
    for (Iterator<Attribute> attributeIterator = allAttribute.iterator(); attributeIterator.hasNext();)
    {
        Attribute attribute = attributeIterator.next();
        log.debug("---> " + allvalues + ": " + attribute.getName());
        if (attribute.getName().endsWith("*"))
        {
        currentRange = attribute.getName();
        finallyFinished = true;
        }
        String[] attributeBatch = searchResultEntry.getAttributeValues(currentRange);
        for (int i = 0; i < attributeBatch.length; i++)
        {
        allValues.add(attributeBatch[i]);
        log.debug("-- " + allvalues++ + " " + attribute.getName() + ":" + attributeBatch[i]);
        }
    }

    }// for SearchResultEntry
    start = start + step;
    finish = finish + step;
}// finallyFinished
return allValues;
}

Any ideas?

Thanks -jim

Answer 1

I got things working, but the process is very difficult and currently I am using a hard coded value for the step as this could be dynamically changed formt he default of 1,500 to a hard coded limit of 5,000.

I have not been able to determine the value dynamically. Appears, maybe, that if it is not defined at: CN=Query-Policies,CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,forest root then is must be at defaults, which the default, also varies based on which version of Microsoft Active Directory is being used.

There is also described in MSDN about some sort of control that might help, but no information on how it could be used. Anyone ever use this?

LDAP policies are specified using the lDAPAdminLimits attribute.

The lDAPAdminLimits attribute of a queryPolicy object is a multivalued string where each string value encodes a name-value pair. In the encoding, the name and value are separated by an "=". For example, the encoding of the name "MaxActiveQueries" with value "0" is "MaxActiveQueries=0". Each name is the name of an LDAP policy, and the value is a value of that policy. There can be multiple queryPolicy objects in a AD Forest. A DC determines the queryPolicy object that contains its policies according to the following logic:

• If the queryPolicyObject attribute is present on the DC's nTDSDSA object, the DC uses the queryPolicy object referenced by it.
• Otherwise, if the queryPolicyObject attribute is present on the nTDSSiteSettings object for the Active Directory Site to which the DC belongs, the DC uses the queryPolicy object referenced by the Active Directory Site.
• Otherwise, the DC uses the queryPolicy object whose DN is "CN=Default Query Policy,CN=Query-Policies" relative to the nTDSService object (for example, "CN=Default Query Policy, CN=Query-Policies, CN=Directory Service, CN=Windows NT, CN=Services" relative to the root of the config NC).

Answer 2

Here is a very good code example where you can get all members of a group by ranges. It handles the case when you're on the last range too. You can also transform this method in a paginated request. Have a look. It helped me.

try
{
    DirectoryEntry entry = new DirectoryEntry("LDAP://CN=My Distribution List,OU=Distribution Lists,DC=Fabrikam,DC=com");
    DirectorySearcher searcher = new DirectorySearcher(entry);
    searcher.Filter = "(objectClass=*)";

    uint rangeStep = 1000;
    uint rangeLow = 0;
    uint rangeHigh = rangeLow + (rangeStep - 1);
    bool lastQuery = false;
    bool quitLoop = false;

    do
    {
        string attributeWithRange;
        if(!lastQuery)
        {
            attributeWithRange = String.Format("member;range={0}-{1}", rangeLow, rangeHigh);
    }
        else
        {
            attributeWithRange = String.Format("member;range={0}-*", rangeLow);
    }        
        searcher.PropertiesToLoad.Clear();
        searcher.PropertiesToLoad.Add(attributeWithRange);
        SearchResult results = searcher.FindOne();
        foreach(string res in results.Properties.PropertyNames)
        {
            System.Diagnostics.Debug.WriteLine(res.ToString());
    }
        if(results.Properties.Contains(attributeWithRange))
        {
            foreach(object obj in results.Properties[attributeWithRange])
            {
                Console.WriteLine(obj.GetType());
                if(obj.GetType().Equals(typeof(System.String)))
                {
            }
                else if (obj.GetType().Equals(typeof(System.Int32)))
                {
            }
                Console.WriteLine(obj.ToString());
        }
            if(lastQuery)
            {
                quitLoop = true;
        }
    }
        else
        {
            lastQuery = true;
    }
        if(!lastQuery)
        {
            rangeLow = rangeHigh + 1;
            rangeHigh = rangeLow + (rangeStep - 1);
    }
}
    while(!quitLoop);
}
catch(Exception ex)
{
    // Handle exception ex.
}

Source: http://systemmanager.ru/adam-sdk.en/netds/enumerating_members_in_a_large_group.htm

Answer 3

This one can retrieve and store in textfile any number of users. Moreover it will not finish in infinite loop if group is empty

$myGroup = [string]$args[0];
$myGroup = $myGroup.replace(" ",",");
$group = [adsi]("LDAP://$($myGroup)");
$from = 0 
$all = $false 

$members = @() 


while (! $all) { 
   trap{$script:all = $True;continue} 
   $to = $from + 999 
   $DS = New-Object DirectoryServices.DirectorySearcher($Group,"(objectClass=*)","member;range=$from-$to",'Base') 
   $members += $ds.findall() | foreach {$_.properties | foreach {$_.item($_.PropertyNames -like 'member;*')}} 
   if($from -gt $members.count){
      break;
   }
   $from += 1000 
} 

$currentExecuting = (Get-Item $MyInvocation.MyCommand.Path)
$group.sAMAccountName
$members | measure-object 

$members > "$($currentExecuting.Directory)\$($group.sAMAccountName).txt"

usage:

getADGroupMembers.ps1 CN=groupName,OU=myOrgUnit,DC=contoso,DC=com