Increasing Batch size in SOQL

Question

I am using simple_salesforce package in python to extract data from SalesForce.

I have a table that has around 2.8 million records and I am using query_more to extract all data.

SOQL extracts 1000 rows at a time. How can I increase the batchsize in python to extract maximum number of rows at a time. [I hope maximum number of rows is 2000 at a time]?

Thanks

Salesforce may, at their discretion, adjust the batch size based on the fields you are selecting. As a quick test. Try selecting just the ID of the records and see if the batch size increases to 2000. If it doesn't then you won't be able to adjust it. If it doesn't then you will need to find how simple_salesforce exposes the setting. — Daniel Ballinger, Apr 23 '15 at 22:01

score 0 · Answer 1 · edited Dec 04 '15 at 22:40

If you truly wish to extract everything, you can use the query_all function.

query_all calls the helper function get_all_results which recursively calls query_more until query_more returns "done". The returned result is the full dictionary of all your results.

The plus, you get all of your data in a single dictionary. The rub, you get all 2.8 million records at once. They may take a while to pull back and, depending on the size of the record, that may be a significant amount of ram.

Increasing Batch size in SOQL

1 Answers1