Extract information from fits files in a list of directories but having trouble cd-ing via scripting

Question

I'm writing a script and need for cd to iterate through a bunch of subdirectories but I can't get the shell to commit to the cd, much less execute the rest of the script properly. I've pored over similar questions but none of them have answered me properly -- making a function and sourcing the script hasn't worked. I'm still relatively new to the terminal and I'm very lost right now.

#!/bin/bash
. ./exptime.sh #without a #, this yields a segmentation fault

function exptime() {
   #make an array of directories
   filedir=( $(find ~/Documents/Images -maxdepth 1 -type d) ) 
   alias cdall 'cd ${filedir[*]}' #terminal has trouble recognizing the alias

   for filedirs in ${filedir[*]}
   do
       cdall
       ftlist "fuv.fits[1]" T column=3 rows=1 | grep "[0-9]" |
         awk '{print $2}' > fuv_exptime #recognizes this command but
                   # can't execute properly because it's in the wrong directory
   done

`${filedir[*]}` expands to the whole array. If your array is `filedir=(dir1 dir2 dir3)`, your alias becomes `cd dir1 dir2 dir3`. What do you expect that command to do? — Benjamin W., May 15 '17 at 04:08
There is a bunch of other problems, too. The `alias` lacks the `=`; you're relying on word splitting and risk filepath expansion due to lack of quoting; what are the contents of `exptime.sh`? — Benjamin W., May 15 '17 at 04:10
All in all, you should describe what you're trying to achieve rather than asking to salvage this attempt. — Benjamin W., May 15 '17 at 04:12
I was trying to cd through subdirectories using a for loop but I find terminal syntax very opaque. I didn't include an = in the alias because I didn't need it when I set aliases in ~/.bash_profile. There are a lot of directories to change, how can I expand the alias to include all of them without manually typing everything out? And the line break is unintentional, just an artifact of pasting the code to here — tashton, May 15 '17 at 15:27
exptime.sh is the name of the script. I saw a suggestion to source the script in itself, but I wasn't entirely sure what that meant or how to include it and it hasn't worked anyway — tashton, May 15 '17 at 15:28
The objective is to cd into each subdirectory and execute the ftlist command for the fuv.fits file in each directory, but I can't figure out how to use cd in shell scripting with a for loop (if a for loop is even the right way to go) — tashton, May 15 '17 at 15:33

score 0 · Accepted Answer · answered May 15 '17 at 06:42

0

As already stated in the comments of the question, it is a bit hard to guess, what the script is supposed to do. Anyway, assuming that the directory change part is the only problem, I'd try to fix the script in this way:

#!/bin/bash
. ./exptime.sh #without a #, this yields a segmentation fault

function exptime() {
   #make an array of directories
   filedirs=( $(find ~/Documents/Images -maxdepth 1 -type d) )
   scriptdir=$(pwd)

   for filedir in ${filedirs[*]}
   do
       cd $filedir
       ftlist "fuv.fits[1]" T column=3 rows=1 | grep "[0-9]" | \
         awk '{print $2}' > fuv_exptime
       cd $scriptdir 
   done

In other words, get rid of the 'cdall', and cd into each directory during the for-loop. After the ftlist call, cd back into the directory from where you call the script, which you save before the for-loop in the variable 'scriptdir'. Hope this helps.

answered May 15 '17 at 06:42

Thomas Kühn

9,412
3
47
63

This helped! It's recognizing the cd, but only if I remove the 'function exptime() { ... }' wrap. But the terminal now doesn't recognize the 'ftlist' command. It's from the heatools tasks list (https://heasarc.gsfc.nasa.gov/ftools/caldb/help/heatools.html). Is this something I'll have to alias? – tashton May 15 '17 at 15:37
Did you try the ftlist line at all before writing the script? Does that critical line actually work in any setting? Try 'which ftlist' in the folder where that command works (which is most likely where it is located) or 'locate ftlist' otherwise. If all else fails, ask the person who installed it, where to find it. Once the location is clear, I can edit my solution accordingly. – Thomas Kühn May 15 '17 at 17:33
ftlist works after declaring 'heasoft'. The command itself is located in a /usr/local/ directory (fullpath /usr/local/heasoft-6.17/x86_64-apple-darwin15.3.0/bin/) – tashton May 15 '17 at 20:59
This worked! Eventually, after declaring the proper environment. Thank you – tashton May 15 '17 at 21:04
No problem. You figured out quite a bit yourself :) – Thomas Kühn May 15 '17 at 21:26

score 0 · Answer 2 · answered May 15 '17 at 16:04

0

If I understand correctly what you're trying to do, this should work:

for dname in "$HOME"/Documents/Images/*/; do
    ftlist "$dname/fuv.fits[1]" T column=3 rows=1
done | awk '/[[:digit:]]/ { print $2 }' > fuv_exptime

This loops over all subdirectories of ~/Documents/Images and runs the ftlist command with the full path to the input file.

The output will go into a single file fuv_exptime. Notice that the grep and awk steps can be combined into a single awk command.

If you want an individual output file fuv_exptime in every subdirectory instead, change to the following:

for dname in "$HOME"/Documents/Images/*/; do
    ftlist "$dname/fuv.fits[1]" T column=3 rows=1 |
        awk '/[[:digit:]]/ { print $2 }' > "$dname"/fuv_exptime
done

answered May 15 '17 at 16:04

Benjamin W.

46,058
19
106
116

This works! It recognizes the cd loop. Thanks a lot. However my only beef is that the terminal doesn't recognize the ftlist command- it just gives me a "command not found" error. Is that something I should alias? (ftlist is a command from the heatools archive heasarc.gsfc.nasa.gov/ftools/caldb/help/heatools.html) – tashton May 15 '17 at 16:31
@tashton Is it in your `PATH`? It depends on how you installed it. If it's not in your `PATH`, you can either add it, or use the full path instead of just `ftlist`. – Benjamin W. May 15 '17 at 16:33
Someone else installed it so I'm unsure where they stored it. How can I check if it's there, and how do I work around it if it's not. Seriously, again, thanks so much for your help – tashton May 15 '17 at 16:35
@tashton If it's in your path, `which ftlist` would find it, but then you wouldn't have the "command not found" problem in the first place. You can try `locate ftlist`. – Benjamin W. May 15 '17 at 16:48
@tashton So you have to either add `/usr/local/heasoft-6.17/x86_64-apple-darwin15.3.0/bin` to your `PATH` or call `ftlist` with `/usr/local/heasoft-6.17/x86_64-apple-darwin15.3.0/bin/ftlist`. – Benjamin W. May 15 '17 at 20:44
'ftlist' is located in a /usr/local/ directory (fullpath /usr/local/heasoft-6.17/x86_64-apple-darwin15.3.0/bin/) – tashton May 15 '17 at 20:56
It's working now though. Thank you very much for your help – tashton May 15 '17 at 21:04

Extract information from fits files in a list of directories but having trouble cd-ing via scripting

2 Answers2