5

In an application I am developing I use splice on Linux for socket-to-socket data transfer.

  1. Do other operating systems (specifically at least Windows, OS X and FreeBSD) implement splice or an equivalent solution?
  2. Is it possible to imitate socket-to-socket data spliceing on Windows with sendfile¹ + memmap¹?

¹ Both exist on Windows under different names which I do not remember.

Update

You can see the performance improvements of splice vs user space buffers on Linux.

go

  • DF, DR, F, MF, MR are my application in its different tunneling modes, NX is NGINX web server
  • -p+t uses the Linux system call splice
  • +p-t uses a portable implementation with user space buffers
  • +p+t uses a portable implementation with user space buffers and multiple OS threads
  • bars represent throughput in gigaBYTE s per second
  • a single 1 gigabyte (1048576000 bytes) file was requsted 4 times by 4 concurrent clients
Cetin Sert
  • 4,497
  • 5
  • 38
  • 76
  • 1
    I have some thoughts on this. But before I answer, let me ask, "what are you really trying do?" Are you trying to echo a data stream or packets from one socket onto another? Do you need to inspect the data before sending it on? – selbie Sep 01 '12 at 21:03
  • Echo only echo. I am working on a simple tunneling solution called [PortFusion](http://fusion.corsis.eu) which has no need to inspect data streams (but if it ever does, I can switch to a portable implementation which brings data to user space). – Cetin Sert Sep 02 '12 at 00:36
  • @selbie I just added a chart to demonstrate why I am interested in `splice`-equivalents in other operating systems. – Cetin Sert Sep 02 '12 at 00:55

2 Answers2

1

OpenBSD has sosplice and somove: http://www.openbsd.org/cgi-bin/man.cgi?query=sosplice

Cetin Sert
  • 4,497
  • 5
  • 38
  • 76
0

TransmitFile comes to mind as one such way in winsock to stream from one file handle to a socket, but I an not certain if that will help you as it's for streaming from file->disk. You might be able to use it with a memory mapped file handle.

I've heard a few people in the past speak of "kernel mode" as the way to speed up some echo and simple server operations, but I've never done it.

Some links on kernel mode winsock:

http://blogs.msdn.com/b/wndp/archive/2006/02/24/introduction-to-winsock-kernel-wsk.aspx

http://msdn.microsoft.com/en-us/library/windows/hardware/ff571084(v=vs.85).aspx

selbie
  • 100,020
  • 15
  • 103
  • 173
  • TransmitFile seems rather `File -> Socket` to me :) and I am all for trying memory mapped files but the part for receiving data from a socket directly to a memory mapped file without ever touching it in user space is missing, a zero-copy function of type `Socket -> File`. – Cetin Sert Sep 02 '12 at 01:46