Paging using ldapsearch

Question

I am searching an LDAP directory that has a much larger number of results than the sizelimit currently set,500, by slapd.conf that for all intents and purposes cannot be changed)

My idea was to keep running ldapsearch but from a different offset each time (501, 1001, etc.) until all results have been obtained.

I have seen the man pages for ldapsearch, and it appears that this is handled for you using the -E options:

-E [!]<ext>[=<extparam>] search extensions (! indicates criticality)
         [!]domainScope              (domain scope)
         [!]mv=<filter>              (matched values filter)
         [!]pr=<size>[/prompt|noprompt]   (paged results/prompt)
         [!]subentries[=true|false]  (subentries)
         [!]sync=ro[/<cookie>]            (LDAP Sync refreshOnly)
                 rp[/<cookie>][/<slimit>] (LDAP Sync refreshAndPersist)

So i tried: ldapsearch -h $HOST -p $PORT -x -L -b "$BASE" '*' '+' -E pr=$SIZE

However when the results (even paged results with size = 50 for example) hit 500, I get the same error as if the results were not paged:

Size limit exceeded (4)

I have seen in the man pages there is another option for virtuallistview but have been unable to find examples for it, and as well I dont think my version of ldapsearch has this option.

The goal here is to create a backup using ldapsearch and the -L option to create an ldif file that is suitable for restoring the database.

A few google searches show up with the same problem I'm having, but none have an applicable solution..

I would suggest you setting `loglevel` to 256 to see what it say. — quanta, Nov 08 '11 at 05:30
I do not have direct access to the slapd.conf file, and I know what the problem is so a more verbose log will not be of much use. — steve-gregory, Nov 08 '11 at 19:04
In response to Ethan, 2147483647 is 2^32, or 0b1111111111111111111111111111111. This is the maximim for a 32-bit signed int. This is the maximum allowable size limit. — exigent, May 31 '13 at 13:27

score 15 · Answer 1 · answered Apr 16 '12 at 13:37

15

ldapsearch -LLL  -x -h $LDAPHOST -b"dc=whatever" -D${LDAPUSER} -w"${LDAPPASW}" objectclass=* -E pr=2147483647/noprompt

The important part is at the end: -E pr=2147483647/noprompt. I implemented this today, so I know it works, at least with Active Directory's LDAP backend. For me, this was able to bypass server limits.

From your example, it looks like you may be missing a /noprompt or /prompt. The difference is that with /prompt, it stops between each page.

I am not sure why the number 2147483647 works, but it does.

My source: http://www.commandlinefu.com/commands/view/2779/bypass-1000-entry-limit-of-active-directory-with-ldapsearch

answered Apr 16 '12 at 13:37

Ethan

341
3
9

5

The number `2147483647` is the page size. If you change the `-E` parameter to `pr=100/prompt` you will see that `ldapsearch` aks you to press a key every 100 results. – rpet Jun 24 '14 at 16:23
That works !!!! – Saheb Oct 26 '15 at 10:34
1

Active Directory is unusual in my experience in that it lets paged searches exceed the server configured size limit. It will, however, enforce the paged size limit on the searches. Because you've told ldapsearch to not prompt you, you're not seeing that the page size it's actually using for that search is much smaller. This won't work against Sun LDAP as of 10 years ago, OpenLDAP as of 8 years ago, or 389-ds as of 10 minutes ago (that's what I run at home these days.) – Ed Grimm Dec 02 '19 at 15:05

score 10 · Accepted Answer · answered Nov 08 '11 at 19:55

The directory server administrator is free to impose a limit on the number of entries that can be returned in the response to a search request. The LDAP client can request a size limit, but this client-requested limit cannot override the server-imposed limit. The paging is working correctly: paging simply sends multiple search responses, each the size requested by the client, but still cannot exceed the size limit imposed by the server. The Virtual list View is similar to the simple paging, except that the LDAP client can start and resume anywhere, whereas in simple paged results, the LDAP client must read through results sequentially.

score 0 · Answer 3 · answered Nov 08 '11 at 22:59

0

ApacheDS can do the paged searches you are looking for. At least against Active Directory.

answered Nov 08 '11 at 22:59

geoffc

2,165
5
25
40

Paging using ldapsearch

3 Answers3

Linked