2

So at the beginning of my script, I am defining "threshold," which has a number that isn't going to exist in the next part, (2) and a number that will exist in the next part (6). I'm having the result of running df -h to a file called dffile. My question, is how do I get grep in line 7 to search all of the variable "threshold" for the number that will exist in the file? It works if I have the 6 before the 2 in the variable, so it seems as if it's only searching the first number in it. Thanks!

#!/bin/bash

threshold=("2%" "6%")

df -h > dffile

grep $threshold dffile >> thresh

cat thresh | awk '{print $6}' >> finding1

LINES=()
while IFS= read -r finding1
do
find $finding1 -xdev -size +40M -exec ls -lah {} \; | head -n 10
done < "finding1"

The output of df -h on my test server is: 

root@tstd0001:~/scripts# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            481M     0  481M   0% /dev
tmpfs            99M  616K   98M   1% /run
/dev/vda1        25G  1.3G   23G   6% /
tmpfs           493M     0  493M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           493M     0  493M   0% /sys/fs/cgroup
/dev/vda15      105M  3.4M  102M   4% /boot/efi
tmpfs            99M     0   99M   0% /run/user/0

As you can see above, "2" from my variable, does not exist, whereas "6" does. My goal is to make grep find any number that matches a number inside the variable.

Allan
  • 12,117
  • 3
  • 27
  • 51
Blake Smreker
  • 117
  • 1
  • 5

2 Answers2

1

Let's consider that cat output_df is the output of your df -h command:

$ cat output_df
Filesystem      Size  Used Avail Use% Mounted on
udev            481M     0  481M   2% /dev
tmpfs            99M  616K   98M   1% /run
/dev/vda1        25G  1.3G   23G   6% /
tmpfs           493M     0  493M   6% /dev/shm
tmpfs           5.0M     0  5.0M   2% /run/lock
tmpfs           493M     0  493M   0% /sys/fs/cgroup
/dev/vda15      105M  3.4M  102M   4% /boot/efi
tmpfs            99M     0   99M   0% /run/user/0

Then you change the first part of your script in the following way:

thresold="2%|6%"
cat output_df | awk -v VAR=$thresold '{if($5~VAR)print $6}'

of course you will have to replace cat output_df by df -h in the final script.

This will give the output:

/dev
/
/dev/shm
/run/lock

Explanations:

  • thresold="2%|6%" is a regex that matches 2% or 6% you can generalize it to "2%|6%|X%|Y%|...|Z%"
  • You pass it as a variable to awk via -v VAR=$thresold
  • Then you command awk to print the 6th field when the 5th field does match the regex that you have passed to it via '{if($5~VAR)print $6}'

Then you can regroup everything without using intermediate files:

thresold="2%|6%"
for f in `df -h | awk -v VAR=$thresold '{if($5~VAR)print $6}'`
do 
   find $f -xdev -size +40M -exec ls -lah {} \; 2>/dev/null | head -n10
done

On my box it gives the following output:

-rw-r--r-- 1 root root 47M  6月  1 06:33 /var/cache/apt/srcpkgcache.bin
-rw-r--r-- 1 root root 47M  6月  1 06:33 /var/cache/apt/pkgcache.bin
-rw-r--r-- 1 root root 62M  4月 28 02:27 /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar
-rw-r--r-- 1 root root 56M  2月 20 06:10 /usr/lib/libreoffice/program/libmergedlo.so
-rw-r--r-- 1 root root 73M  5月 22 19:31 /usr/lib/thunderbird/libxul.so
-rwxr-xr-x 1 root root 142M  5月 22 01:35 /usr/lib/chromium-browser/chromium-browser
-rw-r--r-- 1 root root 41M  5月  8 09:57 /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37.28.2
-rw-r--r-- 1 root root 54M 11月 17  2017 /usr/lib/x86_64-linux-gnu/libLLVM-5.0.so.1
-rw-r--r-- 1 root root 99M  3月 18  2017 /usr/lib/x86_64-linux-gnu/libOxideQtCore.so.0
-rwxr-xr-x 1 root root 42M  5月  8 09:57 /usr/lib/x86_64-linux-gnu/webkit2gtk-4.0/WebKitPluginProcess2
 ...

Notes: 2>/dev/null this redirection is to remove all the permission errors by redirecting stderr to /dev/null (muting the stderr)

Allan
  • 12,117
  • 3
  • 27
  • 51
  • @allen, I have changed the code to the following as per your example: #!/bin/bash threshold="2%|6%" df -h > dffile grep $threshold dffile >> thresh cat thresh | awk -v VAR=$threshold '{if($5~VAR)print $6}' >> finding1 LINES=() while IFS= read -r finding1 do find $finding1 -xdev -size +40M -exec ls -lah {} \; | head -n 10 done < "finding1" However, as was intended, it is putting the output of df -h to dffile, but when it's supposed to grep for threshold and send it to the "thresh" file, it does not. – Blake Smreker Jun 01 '18 at 02:49
  • Oops. I'll have to put it up in the post. – Blake Smreker Jun 01 '18 at 02:50
  • @Blake Smreker: You can replace the 3 lines `df -h > dffile grep $threshold dffile >> thresh cat thresh | awk '{print $6}' >> finding1` directly by `thresold="2%|6%" cat output_df | awk -v VAR=$thresold '{if($5~VAR)print $6}' > finding1` – Allan Jun 01 '18 at 02:52
  • 1
    @Allan Also note that `thresold="2%|6%"` must be `thresold="2%\|6%"` to use with `grep` which uses BRE. (basic regular expressions) You can use `egrep` (`grep -E`) to use extended regular expressions. (good effort and answer) – David C. Rankin Jun 01 '18 at 03:03
  • @DavidC.Rankin: thanks David! I have proposed an `awk` solution because it was an overkill to use that many intermediate files and since he was also using `awk` all of those steps could be regrouped in one call, anyway nice answer+1 – Allan Jun 01 '18 at 03:05
  • 1
    Yes, I saw that in your answer, but didn't want to leave the Questioner thinking he could plug that into `grep` and make it work too `:)` – David C. Rankin Jun 01 '18 at 03:06
  • 1
    @Allan Thank you both for your help! I have edited my script, and I got it to work leaning towards Allan's answer. Thank you both for your time, and thank you Allan for the fast replies. Have a great evening! – Blake Smreker Jun 01 '18 at 03:52
1

You initialize threshold as an indexed array,

threshold=("2%" "6%")

You then call grep with:

grep $threshold dffile

Since threshold is an array, to dereference all values in the array, you use the form:

${threshold[@]}   ## which can be quoted to preserve whitespace in elements

When you deference the array as a normal variable, e.g. $threshold you return only the 1st element, e.g.

echo $threshold  ##  output '2%'

So before going further, you need to determine what you want to pass to grep, if you want to search for either 2% or 6%, then Allan has a nice explanation above. You can also construct the grep expression using printf -v, e.g. 

printf -v gexp "%s\|%s" ${threshold[@]}

or to properly limit to the first 2-elements, 

printf -v gexp "%s\|%s" "${threshold[0]}" "${threshold[1]}"

and then call grep with

grep "$gexp" dffile >> thresh

Let me know if you have further questions.

David C. Rankin
  • 81,885
  • 6
  • 58
  • 85