5

In several languages we have the capabilities of using an input/output stream for something such as reading to or writing from a socket, but, what exactly is that physical stream and how is it connected to a socket or file if using a file stream? Most of us take a stream as a magical cord connected to a file or socket but what exactly is under the hood and how does a language "connect to such structure to input and output data? Thanks all!

polymorph
  • 79
  • 9
  • It's not a 'physical stream'. It's a logical stream, realized in software, in various ways depending on the stream, the language, the underlying connection, etc. The only thing that's physical is the disk or the network. – user207421 Mar 12 '14 at 03:16
  • Thanks, I still don't understand what the "stream" is... everyone explains it as if it were a mythical being, how the connection is done nobody explains... – polymorph Mar 12 '14 at 04:06
  • Exactly. It *is* a mythical being. It's a conspiracy of pretence that there is nothing there but a byte stream, which is implemented by various layers of software. At a low layer there are physical disk blocks, network packets, etc, and at a really low layer there are electrons, gamma waves, etc. THe connection is done by software. There's nothing more to explain without going into the gory details of some specific implementation of some specific stream, and your question isn't specific about either. – user207421 Mar 12 '14 at 05:17
  • No sarcasm expressed or implied. I really don't know what you're talking about here. – user207421 Mar 12 '14 at 07:06

1 Answers1

8

The stream itself is really just a fixed-size FIFO queue of bytes, likely implemented as a circular buffer. At one side of this queue is your program, which places bytes into the buffer using fwrite() (or similar), or for an input stream, removes bytes from the buffer using fread(), etc.

On the other side of the queue is some part of the operating system's software; for a file that might be a filesystem layer; for a socket it would be the TCP stack; for stdout it might be a tty or pseudo-tty driver. After your program writes some bytes to the stream, the piece of OS code at the other end is woken up, and it grabs those bytes from the other end of the FIFO and does the appropriate thing with them (e.g. writes them to the disk, or places them into a TCP packet and sends that packet to the Ethernet card, or etc)

The internal machinations of the stream are deliberately kept as hidden from you as possible, so that your code won't rely on any of the particulars of how the stream was implemented. That way if the operating system changes, or you recompile for a different OS, your code will (hopefully) not break and need to be rewritten, because your code never made any assumptions that might no longer be true.

As far as how a language connects to these operating-system mechanisms, it's really not very different from any other code library you might use -- just as someone defined the fwrite()/fread() API that your code uses, someone else defined an operating system API that the implementers of fread() and fwrite() can call functions on (well, really they are system calls, but for our purposes they are similar to functions). And that operating system API can in turn call a device driver API that the kernel uses to talk to the hardware, and so on for as many layers as were deemed necessary to make things work.

It's turtles all the way down :)

Jeremy Friesner
  • 70,199
  • 15
  • 131
  • 234