1

I'm using Google's Distance Matrix API and wanted to get up to the rate limit of 1000 requests in 10 seconds without necessarily using the matrix component but just higher limit capability (directions API has 10 a second -> this in effect has 100 a second).

EDIT:

My main goal is to be able to exactly control how many requests I am filling a second (e.g do 800 in 10 seconds, or do 1000 in seconds) and ideally I would use this kind of pseudo code but I'm not sure:

pool = Pool()  # Initialize a pool of max processes
start_time = time.time()
counter = 0
for one_url in URL_LIST:
    counter += 1
    # Once sent 1000 requests
    if counter = 1000:
        # If elapsed less than 10 seconds
        if time.time() - start_time < 10 seconds:
            # Wait until we hit 10 seconds
            time.sleep(time.time() - start_time)  
        else:
            #Taking more than 10 seconds
            pass
        #Reset counter and timer
        start_time = time.time()  
        counter = 0   
    RESULTS_OUT = pool.map(GeocodeHandler, one_url) 
pool.close()  # This means that no more tasks will be added to the pool
pool.join()  # This blocks the program till function is run on all the items
Ilia Karmanov
  • 215
  • 5
  • 13
  • Why not search for a provider without that API limits? – Karussell Oct 07 '15 at 08:04
  • @Karussell I guess that's the sensible long-term approach; however for now I felt that going from 10 requests to 100 will still be good. To be honest with you I'm more curious with learning about how to get it done than getting it done – Ilia Karmanov Oct 07 '15 at 09:17
  • Would this be better suited for code review? As you've said this works. – Noelkd Oct 07 '15 at 11:54
  • @Noelkd thanks for the response, however I have updated the question to focus on the more pressing matter for me which is trying to hit exactly X number of requests every Y seconds – Ilia Karmanov Oct 07 '15 at 13:39

0 Answers0