2

I want to be able to save a data stream which i am returning using the curl command. I have tried using the cat command, and piping it the curl command, however i'm doing it wrong. The code im currently using is:

cat > file.txt | curl http://datastream.com/data

Any help would be appreciated.

jonhurlock
  • 1,798
  • 1
  • 18
  • 28
  • 2
    For the record, the syntax you were using would send the output of `cat` to `file.txt`, and then it would also try to send that output to the standard input of `curl` (but it couldn't, because that output was already going to the file). Problem 1: `cat` doesn't produce any output other than what it reads from its own standard input, so it would just sit there waiting for you to type something. Problem 2: `curl` doesn't read what comes into its standard input. And problem 3: you're not redirecting the output of `curl` anywhere, so it just gets printed to the screen. – David Z Jun 10 '10 at 23:56
  • Answers below are good. Still, I suggest you read a little on I/O redirection. That may help you a lot next time: http://en.wikipedia.org/wiki/Redirection_%28computing%29 – skinp Jun 11 '10 at 00:12
  • @David: Good explanation +1. I think that what would happen is that the shell would parse to the pipe, and set up the pipe with the 'cat' process's output connected to it. The 'cat' shell process would then do the output redirection (closing the pipe in the process), so 'curl' would get no data on its input. The 'cat' would sit there waiting for EOF on its input; meanwhile, 'curl' would generate its output to standard output, not the file. Very minor difference from what you describe - it is the sequence of 'open pipe; fork; close pipe; open file' that is not quite what I read in yours. – Jonathan Leffler Jun 11 '10 at 03:27
  • @Jonathan: I could believe that... I'm not exactly an expert on the internals of how I/O redirection works. – David Z Jun 11 '10 at 04:59

2 Answers2

5

Try:

curl http://datastream.com/data > output_file.txt
Jim Lewis
  • 43,505
  • 7
  • 82
  • 96
  • Brilliant that worked! Silly me over complicating things, should have read the man page for curl :p Thanks :D – jonhurlock Jun 10 '10 at 23:46
3

So, you are using 'curl' to collect data, and want to capture its output?

curl http://example.com/data > file.txt

Or, indeed (from the man page):

curl -o file.txt http://example.com/data
Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278