Raw sockets and sendto in python

Question

I am working on integrating scapy with twisted, but I ran into this very weird bug on OSX that I can't seem to figure out.

Basically I am unable to send a valid TCP packet (inclusive of IP headers) via a raw socket. This is what I am doing:

import socket
from scapy.all import IP, TCP
pkt = IP(src='0.0.0.0', dst='127.0.0.1')/TCP()
spkt1 = str(pkt)
outs = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
outs.setsockopt(socket.SOL_IP, socket.IP_HDRINCL, 1)
outs.sendto(spkt1, ('127.0.0.1', 0))

When I run this I get the following error:

outs.sendto(spkt1, ('127.0.0.1', 0)) socket.error: [Errno 22] Invalid argument

In case you don't have scapy on don't want to use it this is the packet base64 encoded:

import base64
spkt1 = base64.b64decode("RQAAKAABAABABvvOAAAAAH8AAAEAFABQAAAAAAAAAABQAiAAEH4AAA==")

The very strange thing is that a packet that is nearly identical appears to be sent properly:

spkt2 = base64.b64decode("RQBAAAWwAAACBgAAAAAAAH8AAAEAyAOEAAAAAAAAAACwAgDIAHsAAAIEBbQBAwMBAQEICk3PUjMAAAAABAIAAA==")

This is how the two packets look like:

SPKT1
0000   45 00 00 28 00 01 00 00  40 06 FB CE 00 00 00 00   E..(....@.......
0010   7F 00 00 01 00 14 00 50  00 00 00 00 00 00 00 00   .......P........
0020   50 02 20 00 10 7E 00 00                            P. ..~..
SPKT2
0000   45 00 40 00 05 B0 00 00  02 06 00 00 00 00 00 00   E.@.............
0010   7F 00 00 01 00 C8 03 84  00 00 00 00 00 00 00 00   ................
0020   B0 02 00 C8 00 7B 00 00  02 04 05 B4 01 03 03 01   .....{..........
0030   01 01 08 0A 4D CF 52 33  00 00 00 00 04 02 00 00   ....M.R3........

By checking them out in wireshark they only differ in the TCP part.

I have done a lot of different experiments and I was able in the end by setting certain specific TCP options to get the packet sent, but it does not make sense that such a packet should not work.

Does anybody have an idea why this may be happening?

EDIT:

This packet does appear to work:

pkt = IP(len=16384, src='0.0.0.0', dst='127.0.0.1',
     id=RandShort(), ttl=2)/TCP(sport=255,
      dport=900, flags="S", window=200,
      options=[('MSS', 1460), ('WScale', 2)])
spkt = bytes(pkt)
spkt += '\x00'*20

If you don't add the zeros it does not work.

Could you fix the `import` in your first code snippet? (also, fun fact while I read through the request of your question: you can use `"…".decode("base64")` and `"…".encode("base64")` instead of `import base64`). Ok, sorry, can't help with this one. But you have my upvote. — David Wolever, Jun 12 '12 at 20:27

score 3 · Accepted Answer · answered Jun 13 '12 at 14:34

I ended up deciding that Raw Sockets are just to bugged to be usable. Especially since this software needs to be cross platform, quirks for OSX may not be applicable to other OSs.

For the time being I simply wrapped the "sockets" that are provided by scapy. In the future I will write something that only depends on libdnet (as that is what scapy does to write raw frames).

You can find this implemented here:

https://github.com/hellais/txscapy

score 1 · Answer 2 · edited Sep 16 '15 at 23:06

1

An IP header is required to have a multiple of 32 bits to be valid. And there is also a padding area on the end.

So depending on the IP options set in the header — which occupies a variable amount of bits — one needs to count bits and padding.

Looks like different OSes handle this differently. One might consider that a clever OS would do this padding for you.

edited Sep 16 '15 at 23:06

Paul Roub

36,322
27
84
93

answered Sep 16 '15 at 22:46

cox

11
1

score 0 · Answer 3 · answered Jun 12 '12 at 21:38

0

0.0.0.0 doesn't look like a valid IP source address to me. Does changing this to any other value make any difference?

answered Jun 12 '12 at 21:38

Jean-Paul Calderone

47,755
6
94
122

Looks like it makes no difference. – Yuval Adam Jun 12 '12 at 21:44
no, it does not make any difference. Curiously enough this packet, for example, does appear to work: pkt = IP(len=16384, src='0.0.0.0', dst='127.0.0.1', id=RandShort(), ttl=2)/TCP(sport=255, dport=900, flags="S", window=200, options=[('MSS', 1460), ('WScale', 2)]) spkt = bytes(pkt) spkt += '\x00'*20 If you don't add the 20 zeros it fails like the other one. (adding the packet to the post properly formatted) – Arturo Filastò - hellais Jun 12 '12 at 21:48
Weird. Dunno. My next thought is that OS X kernel is full of obscure bugs like this one. ;) – Jean-Paul Calderone Jun 12 '12 at 22:05
Yeah, this probably is an OSX bug. I am just going to give up using raw sockets as they are terrible for cross platform support and just use libdnet. – Arturo Filastò - hellais Jun 13 '12 at 10:23

score 0 · Answer 4 · answered Jun 12 '12 at 21:49

Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> from scapy.all import IP, TCP
WARNING: No route found for IPv6 destination :: (no default route?)
>>> pkt = IP(src='0.0.0.0', dst='127.0.0.1')/TCP()
>>> spkt1 = str(pkt)
>>> 
>>> outs = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
>>> outs.setsockopt(socket.SOL_IP, socket.IP_HDRINCL, 1)
>>> outs.sendto(spkt1, ('127.0.0.1', 0))
40

I don't seem to have any error writing that specific set of packets - I'm using x86_64 with a 2.6.38-* Gnu/Linux kernel.

Perhaps your issue is related to some Mac OS X brain damage with raw sockets?

score 0 · Answer 5 · answered Aug 21 '12 at 20:17

Another related issue. Python impacket module has ping.py script to ping hosts. On Mac OS X Lion I got an error while using this script:

Traceback (most recent call last):
  File "/private/var/www/env/bin/ping.py", line 73, in <module>
    s.sendto(ip.get_packet(), (dst, 0))
socket.error: [Errno 22] Invalid argument

But on Ubuntu everything works fine and I am getting replies from hosts.

score 0 · Answer 6 · answered Aug 21 '12 at 22:04

I have no hard evidence, but I think this might be related to the ethernet minimum payload size.

From wikipedia:

The minimum payload is 42 octets when 802.1Q tag is present and 46 octets when absent.

Your first example packet was only 40 bytes, so it'd be below the limit in either case. You could experiment with changing the padding from 20 bytes to those values to verify that it stops working at one of the limits.

If so, the behavior makes perfect sense; the OS is rejecting the packet because you aren't giving it enough data to construct a valid packet.

Raw sockets and sendto in python

6 Answers6

Linked