I'm learning ldap3
for Python (as well as learning Python...)
I'm trying to pull some records using ldap3, iterate through them and perform some work on them before putting the results into another database, simple stuff.
The problem I'm having is I'm getting significantly different performance when using two different methods of fetching the data...
Using the basic search http://ldap3.readthedocs.io/tutorial_searches.html
from ldap3 import Server, Connection, ObjectDef, AttrDef, Reader, ALL
server = Server('ldap.xxxx.xxx.xxx', use_ssl=True, get_info=ALL)
conn = Connection(server, 'cn=username,cn=applications,dc=xxx,dc=xxx', 'password',
auto_bind=True, check_names=True)
conn.search('cn=people, dc=xxx, dc=xxx','(HomeCity=nowhere)')
#end
This performs very quickly with minimal memory usage, less than 200MB for about 43000 results
However using the reader cursor http://ldap3.readthedocs.io/tutorial_abstraction_reader.html
from ldap3 import Server, Connection, ObjectDef, AttrDef, Reader, ALL
server = Server('ldap.xxxx.xxx.xxx', use_ssl=True, get_info=ALL)
conn = Connection(server, 'cn=username,cn=applications,dc=xxx,dc=xxx', 'password',
auto_bind=True, check_names=True)
obj_person = ObjectDef('xxxPerson',conn)
r = Reader(conn,obj_person,'cn=people, dc=xxx, dc=xxx','HomeCity:=nowhere',sub_tree=False)
r.search()
#end
takes longer and uses about 2 GB
both are retrieving the exact same # of results. using
for entry in conn.entries
print(sys.getsizeof(entry))
or
for entry in r
print(sys.getsizeof(entry))
return the same size objects
I'm at a loss of what is causing one method to use so much more resources than the other, or how to free up resources during processing
EDIT 1: Ok I think with the first method, I'm actually not pulling in any attributes, which is why it's not really using any memory.. (more testing soon)