0

pretty new to python. I'm trying to integrate this github project (https://github.com/elceef/dnstwist) into a more dynamic / automated tool that will periodically execute, look for new domains that have popped up, add them to a mysql database and shoot off an email to notify that the new domain needs to be investigated. I'm still in very early stages of getting this to work. here is the code so far: 

import os

bleh = []


def writefile():
  sites = ['hi.com', 'ho.com']     
  for site in sites:
    bleh.append(os.system("python dnstwist.py " + site + " | sed -e '1,8d' | grep -v -"))
  print bleh

writefile()

Right now what i'm trying to figure out is how to take the output from 

os.system("python dnstwist.py " + site + " | sed -e '1,8d' | grep -v -")

which outputs something like: 

Original*       hi.com    205.178.189.129
Bitsquatting    ii.com    NS:queens1.tink.com MX:mxin.mxes.net
Bitsquatting    ji.com    198.148.92.53
Bitsquatting    li.com    168.63.6.189 MX:mail.li.com
Bitsquatting    xi.com    94.56.83.199 MX:mailstore1.secureserver.net
Bitsquatting    hh.com    38.75.9.218 MX:mailserv.hh.com
Bitsquatting    hk.com    NS:f.udrtld.net
Bitsquatting    ha.com    206.123.52.10 MX:ironport.heritagecoin.com
Bitsquatting    hy.com    121.40.153.241 MX:mxbiz1.qq.com
Homoglyph       h1.com    NS:ns1.sgi.net
Homoglyph       hl.com    64.106.211.22 MX:mail1.hl.com
Repetition      hii.com   104.209.45.42 MX:joy1.efwmx.net
Repetition      hhi.com   64.71.33.90 MX:mail.hhi.com
Replacement     ui.com    38.102.228.152 MX:smtp.secureserver.net
Replacement     ni.com    96.45.82.97 MX:skprod2.natinst.com
Replacement     bi.com    216.46.183.219
Replacement     gi.com    144.188.20.66 MX:motorola.com.s5a1.psmtp.com
Replacement     yi.com    184.168.221.104
Replacement     h9.com    99.192.229.56 MX:h9.com
Replacement     ho.com    69.172.201.208
Replacement     hj.com    198.148.92.53
Replacement     hu.com    83.222.226.95 MX:
Replacement     h8.com    107.20.188.160 MX:westgate.dejan.net
Transposition   ih.com    173.244.177.114
Original*       123.com    69.58.188.49 MX:mail.entelchile.net
Bitsquatting    023.com    75.126.6.168 MX:mail.023.com
Bitsquatting    323.com    220.181.57.217
Bitsquatting    523.com    198.148.92.48
Bitsquatting    923.com    121.199.16.140
Bitsquatting    q23.com    198.148.92.62
Bitsquatting    103.com    NS:dns6.iidns.com
Bitsquatting    163.com    123.58.180.8 MX:163mx03.mxmail.netease.com
Bitsquatting    1r3.com    NS:f1g1ns2.dnspod.net
Bitsquatting    122.com    219.232.254.201 MX:mxbiz1.qq.com
Bitsquatting    121.com    124.16.31.159
Bitsquatting    127.com    123.58.180.8
Bitsquatting    12s.com    5.22.149.135 MX:mx2.emailsrvr.com
Insertion       1323.com   NS:vip2.360dns.net
Insertion       1233.com   198.148.92.49
Insertion       1w23.com   NS:ns4.51dns.com
Insertion       12w3.com   85.13.215.50
Insertion       1q23.com   121.199.54.199
Insertion       12q3.com   184.168.221.53 MX:smtp.secureserver.net
Insertion       1123.com   116.211.121.191 MX:mxbiz2.qq.com
Insertion       1213.com   NS:ns0.netergy.com MX:mx0.forwarding.anynames.com
Replacement     223.com    198.44.249.20
Replacement     1w3.com    50.22.11.28 MX:1w3.com
Replacement     1q3.com    184.168.221.96 MX:mailstore1.secureserver.net

and take each line individually to pull fields out and insert them into a database. I know how to do this with awk, but i really would like to learn how to do it with python instead. What method/function can i use on the output to extract fields on each line? Right now this line: 

bleh.append(os.system("python dnstwist.py " + site + " | sed -e '1,8d' | grep -v -"))

is returning [0, 0] in the list.

dobbs
  • 1,089
  • 6
  • 22
  • 45
  • Since you are using an external python file into your `system` call. Isn't it easier to just `import dnstwist` and "rewrite" the `main` according to your needs? Especially changing the `p_out` calls to `yield`ing values or build-in up a list to `return`? – 301_Moved_Permanently Sep 30 '15 at 16:10
  • would be easier if i were better at py ;) – dobbs Sep 30 '15 at 16:20

2 Answers2

1

Output of os.system connected with stdout. You should use

os.popen("python dnstwist.py " + site + " | sed -e '1,8d' | grep -v -").read()

instead of

os.system("python dnstwist.py " + site + " | sed -e '1,8d' | grep -v -")

Or you could use subprocess module. Example code

import subprocess
output = subprocess.check_output("python dnstwist.py " + site + " | sed -e '1,8d' | grep -v -", shell=True)
kvorobiev
  • 5,012
  • 4
  • 29
  • 35
1

os.system only returns the exit value

instead use

os.popen(cmd).read() as this will return the program output, but even this is not really recommended, instead you should use subprocess.Popen like you are supposed to ...

Joran Beasley
  • 110,522
  • 12
  • 160
  • 179