115

I'm well aware of the source (aka .) utility, which will take the contents from a file and execute them within the current shell.

Now, I'm transforming some text into shell commands, and then running them, as follows:

$ ls | sed ... | sh

ls is just a random example, the original text can be anything. sed too, just an example for transforming text. The interesting bit is sh. I pipe whatever I got to sh and it runs it.

My problem is, that means starting a new sub shell. I'd rather have the commands run within my current shell. Like I would be able to do with source some-file, if I had the commands in a text file.

I don't want to create a temp file because feels dirty.

Alternatively, I'd like to start my sub shell with the exact same characteristics as my current shell.

update

Ok, the solutions using backtick certainly work, but I often need to do this while I'm checking and changing the output, so I'd much prefer if there was a way to pipe the result into something in the end.

sad update

Ah, the /dev/stdin thing looked so pretty, but, in a more complex case, it didn't work.

So, I have this:

find . -type f -iname '*.doc' | ack -v '\.doc$' | perl -pe 's/^((.*)\.doc)$/git mv -f $1 $2.doc/i' | source /dev/stdin

Which ensures all .doc files have their extension lowercased.

And which incidentally, can be handled with xargs, but that's besides the point.

find . -type f -iname '*.doc' | ack -v '\.doc$' | perl -pe 's/^((.*)\.doc)$/$1 $2.doc/i' | xargs -L1 git mv

So, when I run the former, it'll exit right away, nothing happens.

Mogsdad
  • 44,709
  • 21
  • 151
  • 275
kch
  • 77,385
  • 46
  • 136
  • 148
  • Does your complex command work when you pipe to a temp file first and then source it? If not, what's the problem with the generated output? The output of your command won't work if your filenames have spaces in them or if certain sequences aren't escaped properly. I'd want to add quotes around $1 and $2.doc at a minimum. – Kaleb Pederson Aug 14 '09 at 23:22
  • Is there any good reason for having to run this in the original shell ? - these examples doesn't manipulate the current shell so you gain nothing by doing so. The quick solution is you redirect output to a file and source that file though – nos Aug 14 '09 at 23:43
  • @kaleb the output runs fine. in this particular case, even if i pipe to sh. the file names are space-safe, but thanks for noting. @nos git environment variables on the original shell. and again, these are just examples. the question is for life. – kch Aug 15 '09 at 09:30
  • source /dev/stdin didn't work for me when needing assigned variables to stick around. geirha on freenode bash pointed me to http://mywiki.wooledge.org/BashFAQ/024 and suggested I try a process substitution source <(command) which worked for me – srcerer Jan 30 '15 at 23:44

9 Answers9

159

The eval command exists for this very purpose.

eval "$( ls | sed... )"

More from the bash manual:

eval

          eval [arguments]

The arguments are concatenated together into a single command, which is then read and executed, and its exit status returned as the exit status of eval. If there are no arguments or only empty arguments, the return status is zero.

that other guy
  • 116,971
  • 11
  • 170
  • 194
Juliano
  • 39,173
  • 13
  • 67
  • 73
  • 11
    The only issue here is that you may need to insert ;'s to separate your commands. I use this method myself to work on AIX, Sun, HP, and Linux. – Tanktalus Aug 17 '09 at 22:25
  • 2
    Tanktalus, thanks for that comment, it just made my script work. On my machine eval does not separate commands on newlines and using source does not work even with semi-colons. Eval with semi-colons is the solution. I wish I could give you some points. – Milan Babuškov Apr 21 '11 at 20:48
  • 3
    @MilanBabuškov: I ranked him up for you ;-) – Phil Dec 13 '12 at 11:38
  • 3
    This is excellent. However, for my use case, I ended up having to put quotes around my $(), like so: `eval "$( ssh remote-host 'cat ~/.bash_profile' )"` Note that I am indeed using bash 3.2. – Zachary Murray Nov 07 '13 at 08:47
  • 3
    This works for me where the accepted answer did not. – Dan Tenenbaum Nov 13 '14 at 21:58
103
$ ls | sed ... | source /dev/stdin

UPDATE: This works in bash 4.0, as well as tcsh, and dash (if you change source to .). Apparently this was buggy in bash 3.2. From the bash 4.0 release notes:

Fixed a bug that caused `.' to fail to read and execute commands from non-regular files such as devices or named pipes.

mark4o
  • 58,919
  • 18
  • 87
  • 102
  • Oh, and since you're listing the shells, it works in zsh too. – kch Aug 15 '09 at 09:35
  • 2
    I thought this was ingenious until I tried it in msys/mingw (where there is no /dev folder (or even devices)! I tried many combinations of eval, $(), and backticks ``. I couldn't make anything work, so I finally just redirected output into a temp file, sourced the temp file, and the removed it. Basically "sed ... > /tmp/$$.tmp && . /tmp/$$.tmp && rm /tmp/$$.tmp". Anybody got a msys/mingw solution without temp files??? – chriv Sep 18 '12 at 22:58
  • 17
    Just a remark: This one does not seem to handle export statements as expected. If the piped string has an export statement, the exportet variable is not available in your terminal environment afterwards, whereas with eval export also works perfectly. – Phil Dec 13 '12 at 12:07
  • 2
    In bash without the lastpipe option set, this creates a subshell just like `| bash` would – that other guy Jan 30 '15 at 23:37
  • 1
    Given that MacOS X usually has bash 3.2 (maybe Yosemite has 4.0?), and the need for lastpipe trickery in my use case (exporting all the variables in a file by pre-processing each line with sed to add 'export ') I chose to go with the `eval "$(sed ...)"` approach - but thanks for the 3.2 bug heads-up! – Alex Dupuy May 20 '15 at 16:07
  • @AlexDupuy nope, Yosemite still provides the old `3.2.57(1)-release (x86_64-apple-darwin14)`. – shrx Jun 24 '15 at 13:42
  • The problem with this answer is that the source is executed in a subshell and thus the values are lost at the invoking shell this variation tho worked for me `source <(sed ...)` ... but now I see that it was already given here http://stackoverflow.com/a/22233248/652904 by @rishta – nhed May 22 '16 at 19:09
49

Try using process substitution, which replaces output of a command with a temporary file which can then be sourced:

source <(echo id)
Mark Stosberg
  • 12,961
  • 6
  • 44
  • 49
Jacek Krysztofik
  • 1,266
  • 1
  • 13
  • 29
  • 11
    Which kind of sorcery is this? – paulotorrens Sep 09 '15 at 09:03
  • 2
    This was my first thought as well. Though I like the eval solution better. @PauloTorrens <(xyz) simply executes xyz and replaces <(xyz) with the name of a file that will write the output of xyz. It's really easy to understand how this works by doing for example: `echo <(echo id)`, giving the output `/dev/fd/12` (12 is an example), `cat <(echo id)`, giving the output`id`, and then `source <(echo id)` giving the same output as simply writing `id` – netdigger Feb 10 '16 at 10:04
  • 4
    @PauloTorrens, this is called [process substitution](http://tldp.org/LDP/abs/html/process-sub.html). See linked docs for official explanation, but the short answer is that "<()" is a special syntax that was designed for cases like this in mind. – Mark Stosberg Mar 25 '16 at 17:43
  • 1
    @MarkStosberg, do you know if this is a special syntax, or just a subshell `(...)` redirected? – paulotorrens Mar 25 '16 at 20:36
  • 2
    @PauloTorrens, It is a special syntax just for process substitution. – Mark Stosberg Mar 26 '16 at 01:02
  • I'm from the future and this is still the best solution. Clear, short, simple and free of side effects. Specially now that backticks are considered obsolete and anti-pattern. – JP de la Torre Mar 05 '22 at 16:13
  • I prefer `. <(echo id)`, which does the same thing with less typing. – Drew Nutter Jun 30 '23 at 12:52
  • This can set variables in the current environment, and it executes multiple lines of code. For my use case, this makes it better than the top or accepted answer. But in some situations, this may not be ideal. – Drew Nutter Jun 30 '23 at 12:53
43

Wow, I know this is an old question, but I've found myself with the same exact problem recently (that's how I got here).

Anyway - I don't like the source /dev/stdin answer, but I think I found a better one. It's deceptively simple actually:

echo ls -la | xargs xargs

Nice, right? Actually, this still doesn't do what you want, because if you have multiple lines it will concat them into a single command instead of running each command separately. So the solution I found is:

ls | ... | xargs -L 1 xargs

the -L 1 option means you use (at most) 1 line per command execution. Note: if your line ends with a trailing space, it will be concatenated with the next line! So make sure each line ends with a non-space.

Finally, you can do 

ls | ... | xargs -L 1 xargs -t

to see what commands are executed (-t is verbose).

Hope someone reads this!

Steve P.
  • 14,489
  • 8
  • 42
  • 72
rabensky
  • 2,864
  • 13
  • 18
8
`ls | sed ...`

I sort of feel like ls | sed ... | source - would be prettier, but unfortunately source doesn't understand - to mean stdin.

chaos
  • 122,029
  • 33
  • 303
  • 309
8

I believe this is "the right answer" to the question:

ls | sed ... | while read line; do $line; done

That is, one can pipe into a while loop; the read command command takes one line from its stdin and assigns it to the variable $line. $line then becomes the command executed within the loop; and it continues until there are no further lines in its input.

This still won't work with some control structures (like another loop), but it fits the bill in this case.

Geoff Nixon
  • 4,697
  • 2
  • 28
  • 34
2

To use the mark4o's solution on bash 3.2 (macos) a here string can be used instead of pipelines like in this example:

. /dev/stdin <<< "$(grep '^alias' ~/.profile)"
ish-west
  • 81
  • 2
1

I think your solution is command substitution with backticks: http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_03_04.html

See section 3.4.5

Eric Wendelin
  • 43,147
  • 9
  • 68
  • 92
  • 3
    If you're using bash the `$( )` syntax might be preferred. 'Course backticks work in *more* shells... – dmckee --- ex-moderator kitten Aug 14 '09 at 21:53
  • 6
    $(command) and $((1+1)) work in all posix shells. I think many are put off by vim marking them as syntax errors, but that's just because vim is highlighting for the original Bourne shell which very few use. To get vim to highlight correctly put this in your .vimrc: let g:is_posix = 1 – pixelbeat Aug 15 '09 at 00:58
0

Why not use source then?

$ ls | sed ... > out.sh ; source out.sh
Kaleb Pederson
  • 45,767
  • 19
  • 102
  • 147