Change Filename in Pipe

Question

I have a for loop that accepts a file with one type of extension, but the final command in the pipe requires the STDIN file to have an extension of another type. In my pipe I use awk to change the file type so that it is properly formatted for the final command, but the extension is still linked to the initial input file.

For example:

for file in *.AAA
do
commandA $file | awk ' { print } ' | commandB

A typical use of commandB:

commandB -i myfile.BBB

Is there a way for me to change the extension of a file in the middle of a for loop & pipe?

What part of this involves "the middle of the pipe" deciding what the final name will be, and why do you need to do that then vs deciding before the output file is created? — Charles Duffy, Aug 25 '23 at 21:01
Which shell do you use? Please a tag for your shell to your question. — Cyrus, Aug 25 '23 at 21:03
This might help: https://en.wikipedia.org/wiki/Shell_(computing) I assume you are using `bash`. — Cyrus, Aug 25 '23 at 21:31
"linux" is not a shell -- it's a kernel; one can run any shell one chooses on that kernel, and the answer will depend on the shell that's in use, not the kernel. "bash", or "zsh", or "sh", or "fish" would be examples of shells. — Charles Duffy, Aug 25 '23 at 23:11
(and to be clear, changing a filename in the middle of a _loop_ is very different from, and much easier than, changing a filename in the middle of a _pipe_; here, it's only the loop and not the pipe that matters, because you're making up your mind on the final name when an iteration of the loop starts _before_ invoking the pipeline). — Charles Duffy, Aug 26 '23 at 00:25
(...the thing that makes that distinction important is that all even though data passes through a pipeline left-to-right, all the parts of that pipeline are started in parallel, which means that the output file is typically opened _before_ an executable in the middle of the pipeline has started processing any data at all; so there are real difficulties involved if the middle of a pipeline needs to set an output file name -- but you don't actually have that problem at all in the question described here). — Charles Duffy, Aug 26 '23 at 00:27

Freeman · Accepted Answer · 2023-08-25T21:05:37.530

I think you can change the extension of a file in the middle of a for loop and pipe, As you see ${file%.*} is a cool trick that basically removes the file extension (*.AAA) from the original file name, and then, I add the .BBB extension to create the new file name, which I call new_file,and to top it off,I pass this modified file to commandB with the -i option!

for file in *.AAA
do
    new_file="${file%.*}.BBB"  #change the extension to .BBB
    commandA "$file" | awk '{ print }' | commandB -i "$new_file"

or you can use process substitution, so I use sed to change the file extension from .AAA to .BBB in the file name,and then, I pass this fancy modified file name to commandB using something called process substitution, which treats the command's output like a temporary file :

for file in *.AAA; do
    commandA "$file" | awk '{ print }' | commandB -i <(commandA "$file" | awk '{ print }' | sed 's/\.AAA$/.BBB/')
done

Assuming that `commandB -i myfile.BBB` tells `commandB` to read `myfile.BBB` as it's input, both of those scripts have `commandB` discarding the output of the `commandA "$file" | awk '{ print }'` at the start of the line, the first one then has `commandB` trying to read a file that doesn't exist, and the second one then calls `commandA "$file" | awk '{ print }'` a second time and then has `commandB` reading a temp file name that doesn't have the desired extension and contains **content** where every `.AAA` at the end of a line has been changed to `.BBB`. So neither does what's wanted. — Ed Morton, Aug 26 '23 at 12:06

Change Filename in Pipe

1 Answers1