Redirect cat output to bash script

Question

I need to write a bash script, which will get the subdomains from "subdomains.txt", which are separated by line breaks, and show me their HTTP response code. I want it to look this way:
cat subdomains.txt | ./httpResponse

The problem is, that I dont know, how to make the bash script get the subdomain names. Obviously, I need to use a loop, something like this:

for subdomains in list
do
    echo curl --write-out "%{http_code}\n" --silent --output /dev/null "subdomain"
done

But how can I populate the list in loop, using the cat and pipeline? Thanks a lot in advance!

No, you don't want it to look that way. Rather than the UUOC, you want it to look like `./httpResponse < subdomains.txt` — William Pursell, Apr 05 '21 at 12:29

score 0 · Answer 1 · answered Apr 05 '21 at 12:34

It would help if you provided actual input and expect output, so I'll have to guess that the URL you are passing to curl is in someway derived from the input in the text file. If the exact URL is in the input stream, perhaps you merely want to replace $URL with $subdomain. In any case, to read the input stream, you can simply do:

while read subdomain; do
        URL=$( # : derive full URL from $subdomain ) 
        curl --write-out "%{http_code}\n" --silent --output /dev/null "$URL"
done

score 0 · Answer 2 · answered Apr 05 '21 at 12:56

Playing around with your example let me decide for wget and here is another way...

#cat subdomains.txt 
osmc
microknoppix
#cat httpResponse 
for subdomain in "$(cat subdomains.txt)"
do
 wget -nv -O/dev/null --spider ${subdomain}
done
#sh httpResponse 2>response.txt && cat response.txt 
2021-04-05 13:49:25 URL: http://osmc/ 200 OK
2021-04-05 13:49:25 URL: http://microknoppix/ 200 OK

Since wget puts out on stderr 2>response.txt leads to right output.
The && is like the then and is executed only if httpResponse succeeded.

Cole Tierney · Answer 3 · 2021-04-05T14:56:56.363

0

You can do this without cat and a pipeline. Use netcat and parse the first line with sed:

while read -r subdomain; do
    echo -n "$subdomain: "
    printf "GET / HTTP/1.1\nHost: %s\n\n" "$subdomain" | \
        nc "$subdomain" 80 | sed -n 's/[^ ]* //p;q'
done < 'subdomains.txt'

subdomains.txt:

www.stackoverflow.com
www.google.com

output:

www.stackoverflow.com: 301 Moved Permanently
www.google.com: 200 OK

edited Apr 05 '21 at 14:56

answered Apr 05 '21 at 13:28

Cole Tierney

9,571
1
27
35

Redirect cat output to bash script

3 Answers3