2

I am trying to simulate Network Address Translation for some test code. I am mapping virtual users to high port numbers, then, when I want to test a request from user 1, I send it from port 6000 (user 2, from 6001, etc).

However, I can't see the port number in the response. 

        connection = httplib.HTTPConnection("the.company.lan", port=80, strict=False,   
                                          timeout=10, source_address=("10.129.38.51", 6000))
        connection.request("GET", "/portal/index.html")
        httpResponse = connection.getresponse()
        connection.close()

httpResponse.status is 200, but I don't see the port number anywhere in the response headers.

Maybe I should be using some lower level socket functionality? If so, which is simplest and supports both HTTP and FTP? Btw, I only want to use built-in modules, nothing which I have to install.

[Update] I should have made it clearer; I really do need to get the actual port number received in the response, not just remember it.

JumpAlways
  • 306
  • 2
  • 14
Mawg says reinstate Monica
  • 38,334
  • 103
  • 306
  • 551

3 Answers3

2

HTTP messages do not contain anything about ports so the httpResponse will not have that information.

However, you will need a different connection object (which will map to a different underlying socket) for each request anyway so you can get that information from the HTTPconnection object.

_, port = connection.source_address

Does that help?

Tim Spence
  • 67
  • 7
  • Sorry, I don't quite understand. How do I get the port number from the response? – Mawg says reinstate Monica Mar 21 '16 at 12:59
  • Oh, I think I see what you mean. I should code `connection_1 = httplib.HTTPConnection`, `connection_2 = httplib.HTTPConnection`, etc? Alas, that won't work, for reasons which I will explain when I update my question. Sorry that I wan't so clear at first, and plus one for your answer. – Mawg says reinstate Monica Mar 21 '16 at 13:15
2

To complete @TimSpence answer, you can use a socket object as an interface for your connection and then treat with some API your data as an HTTP object.

host = 'xxx.xxx.xxx.xxx'
port = 80
address = (host, port)

## socket object interface for a TCP connection
listener_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM,
                                socket.IPPROTO_TCP)
listener_socket.bind(address)
listener_socker.listen(MAX_CONNECTIONS)

## new_connection is the connection object you use to handle the data echange
## source_address (source_host, source_port) is the address object
##    source_port is what you're looking for
new_connection, source_address = listener_socket.accept()

data = new_connection.recv(65536)

## handle data as an HTTP object(for example as an HTTP request)

new_connection.close()
JumpAlways
  • 306
  • 2
  • 14
  • And this will help me to get the port number in the response (after I figure out how to send an HTTP request)? – Mawg says reinstate Monica Mar 21 '16 at 13:34
  • 1
    accept() return the object connection and the address connection and the address object is a tuple that contains the host `xxx.xxx.xxx.xxx` and the port number, as you can see from the code where I declare and initialize `address`: `address = (host, port)` – JumpAlways Mar 21 '16 at 13:39
  • Not yet. I am trying to see how to send an HTTP request and get the response, using the socket which you showed me how to open – Mawg says reinstate Monica Mar 21 '16 at 14:02
  • I've showed you the code for the listener, which should send the response... `request()` use itself a socket object to create a TCP connection and send a request. – JumpAlways Mar 21 '16 at 14:09
  • 1
    In your code, you send a request to a specif address (host, port): `("the.company.lan", port=80)`, so why do you expect a response from a different address when you use the HTTPConnection object with these settings to send the request? – JumpAlways Mar 21 '16 at 14:57
  • The `HTTPConnection` object has `sock`, `host` and `port` object as its attributes and they're linked to the parameters you've passed to the `HTTPConnection` initializer, I don't think the `response` object has a `socket` object too. You can use a socket object as interface for the client too to access to these infos, but since the `request()` method use a `socket` object by itself I think you have to find the way to bypass this and provide your own `socket` object as interface for the `HTTPConnection` object to access all the infos the IP and TCP headers can provide. – JumpAlways Mar 21 '16 at 15:15
  • To your comment before the previous one I want to send to port 80 on the server, from port 6000 on the client (and get the response back to port 6000 on the client) – Mawg says reinstate Monica Mar 21 '16 at 15:16
  • 1
    Ok, and why do you have to check the address? The point is the `HTTPConnection` object is not related to a server, you don't have to handle multiple connections with your `HTTPConnection` object, you initialize that providing a specific address, the server one, so the port associated with the response can only be that one, because your object describe a single connection. Can you be more clear about what you need and why? – JumpAlways Mar 21 '16 at 18:07
  • I am testing a router. I need to simulate users of that router. The router has to be sure to return responses to the correct user. Thus, I need to be able to associate responses with requests (as in Network Address Translation). – Mawg says reinstate Monica Mar 21 '16 at 19:21
  • I don't think you have to, because each `HTTPConnection` object is associated to a unique address, so a unique host address and a unique port. I can't see how you can return a return a response to a wrong user, without doing it on purpose. – JumpAlways Mar 21 '16 at 19:26
  • Well, the code that should do that is being written from scratch, an it communicates with some new third party code which communicates with the internet, so they want me to test that there is no chance of responses getting confused. I tend to agree with you, but no matter what you and I think, I have to find a way to code this. – Mawg says reinstate Monica Mar 22 '16 at 08:30
  • 1
    Ok, I gave a look to `httplib.py`, the`HTTPResponse` object take the sock as parameter, but don't store any infos about the connection, becuse of course they would be redoundant, since the `HTTPConnection` has a `sock` object itself, but we can make a subclass of `HTTPResponse` with an `__init__()` that store infos about the address. You have an high reputation, anyhow if you need some help we can chat and do it togheter and then post the final solution. – JumpAlways Mar 22 '16 at 09:20
  • Or I can code it myself, using your suggestion of sub-classing ;-) Thanks for your help; the requirements just shifted, so this is now a moot point. – Mawg says reinstate Monica Mar 22 '16 at 10:40
  • 1
    I've read your update... If you want to receive the port number, you should look at my second answer, because the `HTTPResponse` object is initialized with the same `socket` object of the `HTTPConnection` object which you can access without storing redoundant data. So, I though you could include the infos you need in the response headers. – JumpAlways Mar 22 '16 at 11:01
  • Alas, the solution would need to work with all servers, and there is nothing that we can write to the header that all servers will return to us - apart for the source IP & port number, but I need to be able to prove that those made the round trip. I may not store them locally. – Mawg says reinstate Monica Mar 22 '16 at 11:10
  • If you can't control the server program or you can't write to the headers, i think you can't get the infos you need; if you can't write anything except the source IP and the port number it's enough, in fact, I proposed to write a `host` header: `'Host: IP:port'`, which is a standard request header, but appending to the response wouldn't harm even if that response was received from a different client. – JumpAlways Mar 22 '16 at 11:25
0

Considring your comments, I had to provide a new answer.

I though you can also put a non standard header host in your HTTPRespose,
'Host: domain/IP:port', so that your client can read it when it receives a response.


Server Response:

HTTP/1.1 200 OK
Date: Day, DD Month YYYY HH:MM:SS GMT
Content-Type: text/html; charset=UTF-8
Content-Encoding: UTF-8
Content-Length: LENGTH
Last-Modified: Day, DD Month YYYY HH:MM:SS GMT
Server: Name/Version (Platform)
Accept-Ranges: bytes
Connection: close
Host: domain/IP:port                                  #exapmple: the.company.lan:80

<html>
<head>
    <title>Example Response</title>
</head>
<body>
    Hello World!
</body>
</html>


Client:

connection = httplib.HTTPConnection("the.company.lan", port=80, 
                                    strict=False, timeout=10,
                                    source_address=("10.129.38.51", 6000))

connection.request("GET", "/portal/index.html")
httpResponse = connection.getresponse()

## store a dict with the response headers
## extract your custom header 'host'
res_headers = dict(httpResponse.getheaders());
server_address = tuple(headers['host'].split(':'))

## read the response content
HTMLData = httpResponse.read(CONTENT_LENGTH_HEADER)

connection.close()


This way you got server_address as a tuple (domain, port).

JumpAlways
  • 306
  • 2
  • 14