2

I'm learning ldap3 for Python (as well as learning Python...)

I'm trying to pull some records using ldap3, iterate through them and perform some work on them before putting the results into another database, simple stuff.

The problem I'm having is I'm getting significantly different performance when using two different methods of fetching the data...

Using the basic search http://ldap3.readthedocs.io/tutorial_searches.html

from ldap3 import Server, Connection, ObjectDef, AttrDef, Reader, ALL

server = Server('ldap.xxxx.xxx.xxx', use_ssl=True, get_info=ALL)
conn = Connection(server, 'cn=username,cn=applications,dc=xxx,dc=xxx', 'password',
                  auto_bind=True, check_names=True)

conn.search('cn=people, dc=xxx, dc=xxx','(HomeCity=nowhere)')

#end

This performs very quickly with minimal memory usage, less than 200MB for about 43000 results

However using the reader cursor http://ldap3.readthedocs.io/tutorial_abstraction_reader.html

from ldap3 import Server, Connection, ObjectDef, AttrDef, Reader, ALL

server = Server('ldap.xxxx.xxx.xxx', use_ssl=True, get_info=ALL)
conn = Connection(server, 'cn=username,cn=applications,dc=xxx,dc=xxx', 'password',
                  auto_bind=True, check_names=True)

obj_person = ObjectDef('xxxPerson',conn)

r = Reader(conn,obj_person,'cn=people, dc=xxx, dc=xxx','HomeCity:=nowhere',sub_tree=False)
r.search()

#end

takes longer and uses about 2 GB

both are retrieving the exact same # of results. using

for entry in conn.entries
    print(sys.getsizeof(entry))

or

for entry in r
    print(sys.getsizeof(entry))

return the same size objects

I'm at a loss of what is causing one method to use so much more resources than the other, or how to free up resources during processing

EDIT 1: Ok I think with the first method, I'm actually not pulling in any attributes, which is why it's not really using any memory.. (more testing soon)

martineau
  • 119,623
  • 25
  • 170
  • 301
dangel
  • 7,238
  • 7
  • 48
  • 74

0 Answers0