15

I want to know how I can send the command(s) spawned by xargs to background. For example, consider 

find . -type f  -mtime +7 | tee compressedP.list | xargs compress

I tried

find . -type f  -mtime +7 | tee compressedP.list | xargs -i{} compress {} &

.. and as unexpected, it seems to send xargs to the background instead?

How do I make each instance of the compress command go to the background?

PoorLuzer
  • 24,466
  • 7
  • 31
  • 35

5 Answers5

20

Use the --max-procs / -P option to run xargs targets in the background. From the man page of GNU xargs version 4.2.27:

--max-procs=max-procs, -P max-procs

Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xargs will run as many processes as possible at a time. Use the -n option with -P; otherwise chances are that only one exec will be done.

(You may want to combine this with -n 1 to makes sure that there is a new process for each file you want to compress)

RobM
  • 8,373
  • 3
  • 45
  • 37
  • (Hmm, just realised my answer is a duplicate of the second-half of gnarf's answer, which I didn't read after the opening comment about writing a shell script. I'll leave my answer here in case others also skim over gnarf's answer as I did) – RobM May 26 '11 at 11:44
8

You could probably make a very quick shellscript to call compress.

#!/bin/sh 
# call it 'compbg' and chmod a+x
compress $* &

then

find . -type f  -mtime +7 | tee compressedP.list | xargs -I{} compbg {}

Although I think you might be happier using this xargs argument:

 -P maxprocs
         Parallel mode: run at most maxprocs invocations of utility at once.

This command should find / tee / compress 10 files at a time until its done, as well as returning control immediately to the calling script/shell.

find . -type f  -mtime +7 | tee compressedP.list | xargs -I{} -P10 compress {} &
gnarf
  • 105,192
  • 25
  • 127
  • 161
  • The UNIX I have ( $ uname -a SunOS xxx 5.10 Generic_xx7xx1-06 sun4u sparc SUNW,Sun-Fire ) does not have -P ( $ xargs -P xargs: illegal option -- P xargs: Usage: xargs: [-t] [-p] [-e[eofstr]] [-E eofstr] [-I replstr] [-i[replstr]] [-L #] [-l[#]] [-n # [-x]] [-s size] [cmd [args ...]] ) Any other alternatives than the shell script wrapper? – PoorLuzer Dec 23 '09 at 12:19
  • 1
    Sorry, I can't find much for SunOS - although I did find this package "xjobs" which you might be able to compile... http://www.maier-komor.de/xjobs.html – gnarf Dec 24 '09 at 01:13
5

For SunOS you may have a look at GNU Parallel http://www.gnu.org/software/parallel/

find . -type f  -mtime +7 | tee compressedP.list | parallel compress

It has the added benefit of not terminating incorrectly if the filename contains ' " or space. Adding -j+0 will make it run one compress per CPU core.

Roel Van de Paar
  • 2,111
  • 1
  • 24
  • 38
Ole Tange
  • 31,768
  • 5
  • 86
  • 104
1

This works for me when I want to edit all files that contain foo

grep -rl foo ./ | `xargs emacs` &

Scott Mattocks
  • 19
  • 2
0

On SunOS/Solaris, wrap the command in a call to sh (or other shell):

find . -type f  -mtime +7 | tee compressedP.list | xargs -Ifname sh -c 'compress fname &'

This should run the compresses in parallel without additional tools.

minderbin
  • 1
  • On Linux anyway, this backgrounding will cause xargs to immediately spawn the next process called for by the list being piped in. If there's a lot in compressedP.list, you'll run out of memory. – Martin Burch Apr 12 '21 at 19:55