Whois script bash

Question

I have made whois script in bash:

#!/bin/bash

# Store the list of domains in an array
while read domain
do
  # Output the header
  echo "Domain,Registrar,Creation Date,Expiration Date"

  # Perform a WHOIS lookup for each domain
  whois_output=$(whois "$domain")

  # Extract the relevant information from the WHOIS output
  registrar=$(echo "$whois_output" | grep "Registrar:" | awk '{print $2}')
  creation_date=$(echo "$whois_output" | grep "Creation Date:" | awk '{print $3}')
  expiration_date=$(echo "$whois_output" | grep "Expiration Date:" | awk '{print $3}')

  # Output the information in a row
  echo "$domain,$registrar,$creation_date,$expiration_date" >> domain_info.csv
done < domains.csv

The output is coming in vertical way:

Facebook.com
Creation date
Update date


And I want data in horizontal way like: 


Facebook.com,creation date , update date

this is the output I want & this the output I am getting

As a general note, `echo | grep | awk` is needless, because everything `grep` can do `awk` can do too: You can use `registrar=$(echo "$whois_output" | awk '/Registrar:/ {print $2}')` to do both the filtering and the column selection at once. — Charles Duffy, Feb 10 '23 at 22:44
Anyhow -- what does `bash -x yourscript` say it's assigning to the `domain`, `registrar`, &c variables? There _shouldn't_ be newlines in their contents, but if there were it would explain the bug. — Charles Duffy, Feb 10 '23 at 22:46
Hi thanks, this is an example i have made many scripts of whois if i talk about python-whois representation of output is what i needed but sadly python -whois unable to retrieve all the information and if i talk about bash output information is coming in vertical way like Facebook.com under its creation date i want in horizontal way . — ROHAN, Feb 10 '23 at 23:00
Please answer the question about the output of running your script with trace logging (as done with `bash -x`). — Charles Duffy, Feb 10 '23 at 23:35
`whois stackoverflow.com | grep Registrar: | awk '{print $2}'`; `whois facebook.com | grep Registrar: | awk '{print $2}'`. Hmmm... — jhnc, Feb 11 '23 at 02:32
`...| awk '-F:' '{print $2}'` may work better. Also msg to O.P. Make an `awk` script that captures all variables required and uses `printf("%s,%s,%s\n", Reg, CreDt, UpDt)`. Good luck. — shellter, Feb 11 '23 at 03:32
@CharlesDuffy : ++ grep 'Expiration Date:' https://www.icann.org/resources/pages/epp-status-codes-2014-06-16-en.' ++ awk '{print $3}' + expiration_date=Expiration + echo 'facebook.com,RegistrarSafe, RegistrarSafe,,1997-03-29T05:00:00Z 1997-03-29T05:00:00Z,Expiration' + read domain — ROHAN, Feb 11 '23 at 10:05
@jhnc hi , I have tried this but the output which is saving in csv is in vertical way .I need in sequence i .e in horizontal way — ROHAN, Feb 11 '23 at 10:22
@konsolebox hi nothing change data coming in vertical way only — ROHAN, Feb 11 '23 at 10:26
@shellter hi its just an example I want output in csv in horizontal way . — ROHAN, Feb 11 '23 at 10:48
`echo 'facebook.com,RegistrarSafe, RegistrarSafe,,1997-03-29T05:00:00Z 1997-03-29T05:00:00Z,Expiration'` _does_ write exactly one line "in a horizontal way" like you ask for. — Charles Duffy, Feb 11 '23 at 14:03
@CharlesDuffy sir when i save output in csv it will shows as vertical . apart from this i cannot share CSV results here other wise it will clearly shows my problem — ROHAN, Feb 11 '23 at 14:17
"Shows as vertical" in what software? Are you viewing your CSV with a text editor or with something like a spreadsheet? — Charles Duffy, Feb 11 '23 at 14:27
(And why do you mean you can't share the result here? You can insert the text of the CSV file into the question the same way your code is inserted) — Charles Duffy, Feb 11 '23 at 14:28
You have misunderstood my comment. Your variables like `registrar` can be set to multiple newline-delimited values. I showed the examples of stackoverflow and facebook which both give multiple values for `registrar`. When you later do `echo "...$registratar..."` you will find newlines inserted for such values. — jhnc, Feb 11 '23 at 14:52
@CharlesDuffy hi I have attached the csv you can check above I hope you'll get it. — ROHAN, Feb 11 '23 at 18:52
That's not a CSV, that's an Excel screenshot. Please provide **the actual CSV** -- and, ideally, the `bash -x yourscript` trace of the execution that generated it. I _suspect_ that the problem is what jhnc's answer suggests, which is also what I suggested in an earlier comment. — Charles Duffy, Feb 12 '23 at 16:07
(Note that if in `echo 'facebook.com,RegistrarSafe, RegistrarSafe,,1997-03-29T05:00:00Z 1997-03-29T05:00:00Z,Expiration'` what _looks_ like a space is actually a newline, that would explain your problem; spaces and newlines looking the same in a comment is part of why it's bad practice to paste code into comments -- it should instead be edited into the question itself to not corrupt the formatting) — Charles Duffy, Feb 12 '23 at 16:09
The other thing you can do is `echo "${registrar%%$'\n'*}"` to emit only the first line of the registrar variable. — Charles Duffy, Feb 12 '23 at 16:13
Or use `registrar=$(echo "$whois_output" | awk '/Registrar/ {print $2; exit}')` -- the `exit` will make `awk` print only one line. — Charles Duffy, Feb 12 '23 at 16:14
@CharlesDuffy thanks there one more question if I run command whois facebook.com | grep -i "Admin Organization: "| cut -d ":" -f2 . output is like : Meta Platforms, Inc. you can see comma after platforms this comma makes another column in csv is there any way to combine so that Meta Platform Inc comes under single column? — ROHAN, Feb 12 '23 at 22:13
Ideally, use a real CSV generation library instead of hacking something up with bash in the first place. Something like the Python CSV module will add quotes around the whole string so the comma isn't treated as a separator. You can of course add quotes in your bash too, but then you need to double up any quotes in the data so they aren't treated as syntax on the parsing end. — Charles Duffy, Feb 13 '23 at 17:52

jhnc · Answer 1 · 2023-02-12T16:35:43.003

You have assumed that whois_output will contain something after the whois call and that registrar, creation_date and expiration_date will all end up with precisely one value each.

These assumptions are incorrect.

For example, the output of whois facebook.com | grep Registrar: (when I run it here) is:

   Registrar: RegistrarSafe, LLC
Registrar: RegistrarSafe, LLC

so when you run your awk '{print $2}' command, the variable registrar contains:

RegistrarSafe,
RegistrarSafe,

(There is an embedded newline.).

Later when you do echo "$domain,$registrar,[...]" you will get:

facebook.com,RegistrarSafe,
RegistrarSafe,,[...]

The fix to your problem is to ensure precisely one value is assigned to each of your variables, and that they do not contain embedded newlines.

For valid csv output, you should probably also quote any value that contains commas.

score 0 · Accepted Answer · answered Feb 11 '23 at 16:36

0

For the vertical to horizontal conversion, I think the simplest fix is to simply remove the newlines which can be done by piping to tr command:

# Output the information in a row
echo "$domain,$registrar,$creation_date,$expiration_date" | tr -d '\n' >> domain_info.csv

answered Feb 11 '23 at 16:36

Jardel Lucca

1,115
3
10
19

yeah it works good for one domain present in csv if mulitples domains present in csv all domains are in single line – ROHAN Feb 11 '23 at 18:38
1

@ROHAN, so add a `printf '\n'` on a separate line after the echo that _doesn't_ go through the `tr -d` – Charles Duffy Feb 12 '23 at 16:12

Whois script bash

2 Answers2