Python 2.6 - passing | char into os.system in for loop

Question

I have a list of IP's that I want to run a whois (using the linux tool whois) against and only see the Country option.

Here is my script:

import os
import time

iplist = open('ips.txt').readlines()

for i in iplist:
        time.sleep(2)
        print "Country: IP {0}".format(i)
        print os.system("whois -h whois.arin.net + {0} | grep Country ".format(i))

So I want to display what IP is being ran, then I just want to see the Country info using grep. I see this error when I run it and the grep is not ran:

sh: -c: line 1: syntax error near unexpected token `|'
sh: -c: line 1: ` | grep Country '

this code below works so it must be an issue with my for loop:

 print os.system("whois -h whois.arin.net + {0} | grep Country ".format('8.8.8.8'))

What am I doing wrong? Thank you!!!!

whats it print in the print statement before your error? (Im guessing you are getting an empty string or something as i) — Joran Beasley, Sep 12 '12 at 17:21
I see output as if I ran this: 'whois -h whois.arin.net + {0}'. it's essentially ignoring the '| grep Country' — D3l_Gato, Sep 12 '12 at 17:25
I found a cheap work-around... I can remove "| grep Country" and add to the print above "print os.system("echo Country: IP {0}".format(i))" and then run the script like "python whois.py | grep Country" — D3l_Gato, Sep 12 '12 at 17:30

score 6 · Accepted Answer · answered Sep 12 '12 at 17:31

6

You're not stripping trailing newlines from the lines you read from the file. As a result, you are passing to os.system a string like "whois -h whois.arin.net + a.b.c.d\n | grep Country". The shell parses the string as two commands and complains of "unexpected token |" at the beginning of the second one. This explains why there is no error when you use a hand-made string such as "8.8.8.8".

Add i = i.strip() after the sleep, and the problem will go away.

answered Sep 12 '12 at 17:31

user4815162342

141,790
18
296
355

Thank you! Does the strip() method, without passing any args to it, take away all special chars from the beginning/end of the string? or only the end? thank you thank you thank you – D3l_Gato Sep 12 '12 at 17:42
`str.strip` returns a new string with all the whitespace (space, tab, \n, \r and such) removed from the beginning and the end of the string. This is a good idea not only for removing newlines, but also for general robustness - you don't want a trailing space ruining your script. – user4815162342 Sep 12 '12 at 18:02
For more information on string methods such as `strip` and `rstrip`, see the documentation at http://docs.python.org/library/stdtypes.html#str.strip . – user4815162342 Sep 12 '12 at 18:11

tMC · Answer 2 · 2012-09-12T17:50:54.007

user4815162342 is correct about the issue you are having, but might I suggest you replace os.system with subprocess.Popen? Capturing the output from the system call is not intuitive.. should you want to result to go anywhere but your screen, you'll likely going to have issues

from subprocess import Popen, PIPE

server = 'whois.arin.net'

def find_country(ip):
    proc = Popen(['whois', '-h', server, ip], stdout = PIPE, stderr = PIPE)
    stdout, stderr = proc.communicate()

    if stderr:
        raise Exception("Error with `whois` subprocess: " + stderr)

    for line in stdout.split('\n'):
        if line.startswith('Country:'):
            return line.split(':')[1].strip() # Good place for regex


for ip in [i.strip() for i in open('ips.txt').readlines()]:
    print find_country(ip)

Python is awesome at string handling- there should be no reason to create a grep subprocess to pattern match the output of a separate subprocess.

Thank you for your comment. Luckily I only need to read the output on screen but I will most likely need to capture the output for reporting down the road. This is great, thank you! — D3l_Gato, Sep 12 '12 at 18:41

score 0 · Answer 3 · answered Sep 14 '12 at 21:11

0

Try sh:

import os
import time
import re
import sh

iplist = open('ips.txt').readlines()

for i in iplist:
    time.sleep(2)
    print "Country: IP {0}".format(i)
    print sh.grep(sh.whois(i, h="whois.arin.net"), "Country")

answered Sep 14 '12 at 21:11

amoffat

696
4
12

Python 2.6 - passing | char into os.system in for loop

3 Answers3