What could make `>` silently fail in Linux?

Question

I ran this command:

python ./manage.py dumpdata partyapp.InvitationTemplate > partyapp_dump.json

To dump data into the partyapp_dump.json file. But all the data just gets printed on the screen and an empty partyapp_dump.json file is created.

Why could this happen? I tested ls > partyapp_dump.json and that worked perfectly.

score 40 · Accepted Answer · answered Apr 14 '11 at 11:20

40

With > you only redirect the standard output. Try 2> instead to redirect the error output. Use &> to redirect both.

answered Apr 14 '11 at 11:20

fab

2,368
2
16
14

1

FYI, `&>` will only work in Bash 4.0 and iirc recent versions of zsh. For a more portable solution, `foo > bar 2&>1`. Reference: http://mywiki.wooledge.org/BashFAQ/014 – Rein Henrichs Apr 14 '11 at 20:40
6

@Rein Henrichs: That's 2>&1, not 2&>1 – camh Apr 14 '11 at 21:12
I remember it with a pneumonic(?) from programming: '2', to('>') the Location('&') Of '1' – hometoast Apr 15 '11 at 11:07
1

@hometoast: You mean mnemonic? :) Pneumonic means lung... – carlpett Apr 16 '11 at 16:14

Caleb · Answer 2 · 2011-04-14T11:44:47.223

Your python app must be writing it's output to the STDERR output channel instead of the normal STDOUT. Using the shell construct > only catches and redirects data written to the output channel, but there are actually several other channels that can be printed to, the most common being the second one, usually used for errors.

You can try trapping STDERR (2nd channel) as well like this:

python ./manage.py dumpdata partyapp.InvitationTemplate > partyapp_dump.json 2>&1

The 2>&1 construct connects the output stream for errors to normal output channel. It is unusual for a program to generate output that you would want to capture on the error channel; usually that would be reserved for debug information not application data. Please use this script with some caution since it is behaving in a non-standard way.

You could also dump the output and error channels to different files like this:

python ./manage.py dumpdata partyapp.InvitationTemplate > partyapp_dump.json 2> error_output.txt

score 5 · Answer 3 · answered Apr 14 '11 at 11:53

5

In addition to the already suggested stderr vs stdout output explanation, your application might simply ignore both of these streams and explicitly open "/dev/tty" for its output.

answered Apr 14 '11 at 11:53

jlliagre

8,861
18
36

score 1 · Answer 4 · answered Apr 15 '11 at 06:12

1

If the noclobber bash option is set, then > redirection will fail (albeit not silently) if the destination file already exists.

For better portability, use cmd >| file to force overwriting any existing file.

answered Apr 15 '11 at 06:12

tylerl

15,055
7
51
72

score 0 · Answer 5 · answered Apr 14 '11 at 12:57

0

If you are lost you can always try to run it with strace to see what processes are doing:

strace -f command

answered Apr 14 '11 at 12:57

bluszcz

409
1
6
16

1

True answer, but not particularly relevant. If the guy didn't now about managing the error stream I don't think he's going to know what to do with the output of an `strace`. – Caleb Apr 15 '11 at 07:35

What could make `>` silently fail in Linux?

5 Answers5