python- How to properly send http GET requests to different Host-IPs but same host

Question

I'm trying to make a small script to send GET requests to a Host that has multiple IP addresses.

So far I was able to make the script send the GET requests with those different IPs all at once(same time), is there a simple way to make it so I provide the list of IPs and it would send the requests with IP#1 then next request with IP#2, etc..

Ps: here is the full code so you can understand a bit what I'm asking, and if you feel like improving it you can let me know too :)

Thanks for your help ! :D

#!/usr/bin/env python

import requests
import ujson
import time
import random


url = 'site'
headeR = {'Host': 'site.com'} 

while 1:


    hosts = ['X.X.X.235', 'X.X.X.94', 'X.X.X.191', 'X.X.X.247'] 
    cnt = 0
    while cnt<len(hosts):
      currentUrl = url.replace("site.com", hosts[cnt])
    cnt += 1
    r = requests.get(currentUrl , headers=headeR)
    listingInfoStr = r.content
    result= ujson.loads(listingInfoStr)
    listingInfoJson= result['listinginfo']  
    if listingInfoJson:
        for key, value in listingInfoJson.iteritems():
            #print("key %s, value %s" % (key, value))
            listingId = key
            try:
                subTotal = value["converted_price_per_unit"]
                feeAmount = value["converted_fee_per_unit"]
            except KeyError:
                continue
            totalPrice = subTotal + feeAmount
            totalPriceFloat = float(totalPrice) / 100
            print("listingId %s = [ %s + %s = %s ]" % (listingId, subTotal, feeAmount, totalPrice))
    else:
            print "Still Looking"
    time.sleep(25)

Instead of `for` loop, you can use a `while` loop with a counter. Increase the counter every time, then you can do that properly. — GLHF, Feb 16 '15 at 05:19

score 1 · Accepted Answer · answered Feb 16 '15 at 07:00

If you can make a request to a single host then it easy to make multiple requests to different hosts in sequence one after another:

import time

def query_listing_info(hosts):
    for host in hosts:
        info = get_listing_info(host)
        if not info:
            print "Still looking"
        else:
            for listing_id, sub_total, fee_amount in parse_listing_info(info):
                total_price = sub_total + fee_amount
                print("listingId {listing_id} = [ {sub_total} + {fee_amount} = "
                      " {total_price} ]".format(**vars()))

hosts = ['x.x.x.x', 'y.y.y.y']
while 1:
    query_listing_info(hosts)
    time.sleep(25) # pause requests

where get_listing_info(host) makes a single request to host:

import urlparse
import requests

parsed_url = urlparse.urlsplit('http://example.com/render/')
params = {'start': '0', 'count': '1', 'currency': '20', 'country': 'CDN'}

def get_listing_info(host): 
    url = urlparse.urlunsplit(parsed_url[:1] + (host,) + parsed_url[2:])
    r = requests.get(url, headers={'Host': parsed_url.netloc}, params=params)
    return r.json().get('listinginfo')

and parse_listing_info(info) extracts the necessary info:

def parse_listing_info(listing_info):
    for id, info in listinginfo.iteritems():
        try:
           yield id, info["converted_price_per_unit"], info["converted_fee_per_unit"]
        except KeyError:
           pass

python- How to properly send http GET requests to different Host-IPs but same host

1 Answers1