How to fetch IP address from a log file?

Question

I am trying to split the IP address into columns, I am new to this and have no idea where to start, hope you can give me a bit of an insight.

My log file

crawl-66-249-64-13.googlebot.com - - [17/Oct/2004:04:40:15 +0100] "GET /robots.txt HTTP/1.0" 200 25 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
66-194-6-72.gen.twtelecom.net - - [17/Oct/2004:04:50:06 +0100] "GET / HTTP/1.1" 200 1727 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312460)"
dup-200-66-220-217.prodigy.net.mx - - [17/Oct/2004:05:36:43 +0100] "GET /midi/main_p.htm HTTP/1.1" 200 1061 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
dup-200-66-220-217.prodigy.net.mx - - [17/Oct/2004:05:37:08 +0100] "GET /favicon.ico HTTP/1.1" 404 1154 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
dup-200-66-220-217.prodigy.net.mx - - [17/Oct/2004:05:37:17 +0100] "GET /midi/mt_pcmid.htm HTTP/1.1" 200 1839 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
dup-200-66-220-217.prodigy.net.mx - - [17/Oct/2004:05:37:24 +0100] "GET /midi/mt_midcp.htm HTTP/1.1" 200 884 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
dup-200-66-220-217.prodigy.net.mx - - [17/Oct/2004:05:37:32 +0100] "GET /midi/mt_mpc.htm HTTP/1.1" 200 3321 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

How to I display only the IP address?

Gilles Quénot · Accepted Answer · 2013-03-13T00:53:49.237

3

Try this (using substitution and capturing groups) :

gawk '{
    print gensub(/[^0-9]*([0-9]{1,3})-([0-9]{1,3})-([0-9]{1,3})-([0-9]{1,3}).*/,
        "\\1.\\2.\\3.\\4",
        "g",
        $0)
}' file.txt

Another approach by DNS resolution :

cut -d' ' -f1 file.txt | xargs dig +short

or with awk :

awk '{print $1}' file.txt | xargs dig +short

edited Mar 13 '13 at 00:53

answered Mar 12 '13 at 12:40

Gilles Quénot

173,512
41
224
223

Hey thank you, your first example using 'gawk' just displayed the entire log file, but the second one work well but unsure how to change it to awk – user2160949 Mar 12 '13 at 20:33
Added `awk | dig` solution; The first one works for me as a charm. – Gilles Quénot Mar 13 '13 at 00:54

Thor · Answer 2 · 2014-09-29T06:29:44.647

1

You could also use grep and tr:

grep -Eo '([0-9]+-){3}[0-9]+' infile | tr - .

Output:

66.249.64.13
66.194.6.72
200.66.220.217
200.66.220.217
200.66.220.217
200.66.220.217
200.66.220.217

edited Sep 29 '14 at 06:29

answered Mar 12 '13 at 12:42

Thor

45,082
11
119
130

score 0 · Answer 3 · answered Mar 12 '13 at 12:47

0

perl -lne 'm/(\d+-\d+-\d+-\d+)\./;$a=$1;$a=~s/-/\./g;print $a' your_file

tested:

> perl -lne 'm/(\d+-\d+-\d+-\d+)\./;$a=$1;$a=~s/-/\./g;print $a' temp
66.249.64.13
66.194.6.72
200.66.220.217
200.66.220.217
200.66.220.217
200.66.220.217
200.66.220.217

answered Mar 12 '13 at 12:47

Vijay

65,327
90
227
319

How to fetch IP address from a log file?

3 Answers3