change filenames in lists of files: slow script

Question

I want to change the name of the files stored in a series of filelists. This is the code I am using:

suffix=[event]

for i in {0..9} a b; do
    for file in $(cat gsc"${i}".list); do
        echo $file$suffix
    done
done

The output is fine, but the process is much slower than if I just output $file. That is to say, the scripts works fine, but why so slow?

EDIT: probably not clear: I need to change the filename WITHIN the filelist (text file).

What makes you think that `echo`ing an additional variable would make the script _significantly slower_? — devnull, Jan 30 '14 at 05:34
I do not think, it is a fact. I don't know the reason, this is what I am asking. — Py-ser, Jan 30 '14 at 05:40

BMW · Answer 1 · 2014-01-30T05:58:38.137

2

How about this:

suffix=[event]
cat gsc[0-9ab].list|sed "s/$/$suffix/"

If you need change the filename which list in gsc*.list file, try this:

suffix=[event]
cat gsc[0-9ab].list |while read file
do
  mv "${file}" ${file}$suffix
done

edited Jan 30 '14 at 05:58

answered Jan 30 '14 at 05:37

BMW

42,880
12
99
116

3

Or you could `sed "s|.*|mv '&' '&[event]'|" gsc[0-9ab].list | sh` – tripleee Jan 30 '14 at 06:07
@BMW: I actually do NOT know how to change the filename WITHIN the list – Py-ser Jan 30 '14 at 06:47

score 1 · Answer 2 · answered Jan 30 '14 at 05:40

1

If the lists are large, needlessly expanding them in the backticks could be slowing you down, and have additional problems. Try this instead:

suffix='[event]'

for i in {0..9} a b; do
    while read file; do
        echo $file$suffix
    done <gsc"${i}".list
done

answered Jan 30 '14 at 05:40

tripleee

175,061
34
275
318

Still slow, but thanks a lot for the explanation and the link. – Py-ser Jan 30 '14 at 05:51

chepner · Answer 3 · 2014-01-30T18:42:37.277

1

[event] is a shell pattern. When you expand $file$suffix, your shell attempts to match any files in the current directory whose names match the value of $file followed by e, v, n, or t. Quoting the expansion should speed things up, by avoiding the attempt to match the pattern. (Note that you would get very different output if there were any matching files.)

suffix=[event]

for i in {0..9} a b; do
    for file in $(cat gsc"${i}".list); do
        echo "$file$suffix"
    done
done

As others have mentioned, the loop is better written

for i in {0..9} a b; do
    while read file; do
        echo "$file$suffix"
    done < gsc"$i".list
done

or replaced altogether with tripleee's sed command

sed "s|.*|mv '&' '&[event]'|" gsc[0-9ab].list | sh

edited Jan 30 '14 at 18:42

answered Jan 30 '14 at 18:37

chepner

497,756
71
530
681

Thanks chepner. Please can you spend a couple of words on what the sed does in this case? So I can manipulate it as I want. – Py-ser Jan 31 '14 at 03:15
The pipe character is used to separate the search text (`.*`, essentially everything) from the replacement text (`mv '&' '&[event]', with `&` being replaced by the matched search text). It creates a valid `mv` command for each line in one of the input files. For example, if `sed` reads "xyz", it outputs a line `mv 'xyz' 'xyz[event]`. The single quotes ensure that the arguments are properly quoted when the commands are parsed by `sh`. – chepner Jan 31 '14 at 13:07

score 0 · Accepted Answer · answered Jan 31 '14 at 08:47

Superfast and complete:

for i in {0..9} a b; do
    sed -i.bak "s|evt|evt[events]|g" gsc"${i}".list
done

It can be improved (generalized) if the specific string in the sed can be substituted by a general string. I can't do it without avoiding slowness and confusion in the task.

change filenames in lists of files: slow script

4 Answers4