Python Parse file with Regex

Question

I am hoping someone might be able to get me on the right track here. I need to parse nmap output with Python. I think using regex is going to be the right method for this, but I have never worked with regex. Here is a sample of the text I need to parse --> http://pastebin.com/QhG86D7D

What I need to do with this file is:

I need the IP addresses.
I need the open ports.

The trick is I need to obviously keep the IP and ports tied together as I will be importing them into a database that will show what ports are open for each IP.

I am also thinking I should write the logic into a function so I can parse different files with larger numbers of hosts.

I do know how to read the file into my script and put the data into my database. Where I am stuck is with the regex to parse out the data I need. Can someone help me out?

Thanks in advance!

Martin Evans · Accepted Answer · 2015-11-09T14:32:58.667

The following approach should get you started:

import re

nmap = """Host: 127.0.0.1 ()      Status: Up
Host: 127.0.0.1 ()      Ports: 22/open/tcp//ssh///, 80/open/tcp//http///, 443/open/tcp//https///        Ignored State: closed (65532)
Host: 127.0.0.2 ()      Status: Up
Host: 127.0.0.2 ()      Ports: 21/open/tcp//ftp///, 22/open/tcp//ssh///, 25/open/tcp//smtp///, 53/filtered/tcp//domain///, 80/open/tcp//http///, 110/open/tcp//pop3///, 143/open/tcp//imap///, 443/open/tcp//https///, 465/filtered/tcp//smtps///, 993/open/tcp//imaps///, 995/open/tcp//pop3s///, 5222/filtered/tcp//xmpp-client///    Ignored State: closed (65523)
# Nmap done at Sat Nov  7 10:40:36 2015 -- 2 IP addresses (2 hosts up) scanned in 32.07 seconds"""

entries = []

for line in nmap.split('\n'):
    re_host = re.match(r'Host\: ([0-9.]+?)\s+', line)
    if re_host:
        host = re_host.group(1)
        ports = re.findall('(\d+)\/open', line)

        if len(ports):
            entries.append((host,  ports))

for host, ports in entries:
    print '{:16} {}'.format(host, ports)

This would display the following output:

127.0.0.1        ['22', '80', '443']
127.0.0.2        ['21', '22', '25', '80', '110', '143', '443', '993', '995']

You would want to replace the split with reading from your file.

Thank you this is exactly what I was looking for! – Justin Nov 09 '15 at 14:34 — Justin, Nov 09 '15 at 14:34

score 0 · Answer 2 · answered Nov 09 '15 at 22:11

There's already a Python module for parsing Nmap scan output. For Python 3, there's python-nmap. For Python 2, Nmap ships with a tool called Ndiff, which is a fully-featured Nmap output parser library with a small driver to produce diffs.

These modules all parse Nmap's XML output (produced with the -oX option). This is the best option, since normal (human-readable) output is subject to change from version to version and Grepable output (your example) is deprecated, meaning that it hasn't provided output from any new Nmap version for a long time: traceroute, NSE, port state reasons, CPE for software and OS versions, etc. are not represented.

Python Parse file with Regex

2 Answers2