Obtain the difference between two files

Question

First of all, i searched on the web and stackoverflow for around 3 days and haven't found anything i've been looking for.

I am doing a weekly security audit where i get back a .csv file with the IPs and the open ports. They look like this:

20160929.csv

10.4.0.23;22
10.12.7.8;23
10.18.3.192;23

20161006.csv

10.4.0.23;22
10.18.3.192;23
10.24.0.2;22
10.75.1.0;23

The difference is: 10.12.7.8:23 got closed. 10.24.0.2:22 and 10.75.1.0:23 got opened.

I want a script which prints me out:

[-] 10.12.7.8:23
[+] 10.24.0.2:22
[+] 10.75.1.0:23

How can i make a script like this? I tried my difflib but that isn't what i need. I need to be able to also write that to files later or send that output as a mail which i have a script for already.

I can't use Unix, because in our company we have a Windows environment and are not allowed to use another OS. So i can't use diff or some other great tools.

This is my first attempt:

old = set((line.strip() for line in open('1.txt', 'r+')))
new = open('2.txt', 'r+')
diff = open('diff.txt', 'w')

for line in new:
    if line.strip() not in old:
        diff.write(line)
new.close()
diff.close()

This is my second attempt

old = set((line.strip() for line in open('1.txt', 'r+')))
new = open('2.txt', 'r+')
diff = open('diff.txt', 'w')

for line in new:
    if line.strip() not in old:
        diff.write(line)
new.close()
diff.close()

If the lines can be in any order, creating a sets for each file would be an option. What have you tried already though? — Caramiriel, Oct 06 '16 at 09:35
@Caramiriel This both: https://paste.ee/p/7KNxv and https://paste.ee/p/kPOle I hope paste.ee is ok. — Daniel S., Oct 06 '16 at 09:40
@e4c5 I know, i am relatively new in scripting with python and i posted this two sources above. For everything i find a explanation and then i use that to code it my way. But i haven't found anything on this thing. - I have found out how i can get an output if there is some difference between the same line in both files, but not if they are a random order like in my example. — Daniel S., Oct 06 '16 at 09:45
are the files sorted? if so @Chris_Rands says, you don't need anything more than `diff` — e4c5, Oct 06 '16 at 09:50
@Chris_Rands I am in an Windows environment. I can't use diff but that would be the best solution. Thanks anyways! — Daniel S., Oct 06 '16 at 09:51

score 3 · Answer 1 · answered Oct 06 '16 at 10:14

In the following solution I've used sets, so the order doesn't matter and we can do direct subtraction with the old and new to see what has changed.

I've also used the with context manager pattern for opening files, which is a neat way of ensuring they are closed again.

def read_items(filename):
    with open(filename) as fh:
        return {line.strip() for line in fh}

def diff_string(old, new):
    return "\n".join(
        ['[-] %s' % gone for gone in old - new] +
        ['[+] %s' % added for added in new - old]
    )

with open('diff.txt', 'w') as fh:
    fh.write(diff_string(read_items('1.txt'), read_items('2.txt')))

Obviously you could print out the diff string if you wanted to.

score 1 · Accepted Answer · edited Oct 06 '16 at 12:10

1

Using your code as a base, you could do the following:

old = set((line.strip() for line in open('1.txt')))
new = set((line.strip() for line in open('2.txt')))

with open('diff.txt', 'w') as diff:
    for line in new:
        if line not in old:
            diff.write('[-] {}\n'.format(line))

    for line in old:
        if line not in new:
            diff.write('[+] {}\n'.format(line))

There's a couple of tweaks in here:

We want to read the individual lines of both the old and new files to compare.
We don't have to strip each individual line as we have done that while reading the file.
We use {} and .format() to build text strings.
Using \n ensures we put each entry on a new line of our output file.
Using with for the file we are writing to lets us open it without having to call close and (if my knowledge is correct) allows for better handling of any program crashes once the file has been opened.

edited Oct 06 '16 at 12:10

Daniel S.

53
1
6

answered Oct 06 '16 at 10:12

asongtoruin

9,794
3
36
47

Shortcut can be `new.difference(old)` and the other way around, but thats entirely optional – Caramiriel Oct 06 '16 at 10:14
@asongtoruin That's exactly what i need. Thank you! This aha moment when you see the solution. - I found out, you only need to read the file so could i just write `open('1.txt')` or are you doing that for a reason? – Daniel S. Oct 06 '16 at 10:52
@JoshE. `open('1.txt')` works here because the default mode of `open` is "read only", but (personally at least) I think it makes for more readable code if you specify how you are opening the file. – asongtoruin Oct 06 '16 at 11:55

score 0 · Answer 3 · answered Oct 06 '16 at 10:12

You can try this one:

old_f = open('1.txt')
new_f = open('2.txt')
diff = open('diff.txt', 'w')

old = [line.strip() for line in old_f]
new = [line.strip() for line in new_f]

for line in old:
    if line not in new:
        print '[-] ' + str(line)
        diff.write('[-] ' + str(line) + '\n'


for line in new:
    if line not in old:
        print '[+]' + str(line)
        diff.write('[+] ' + str(line) + '\n'

old_f.close()
new_f.close()
diff.close()

Obtain the difference between two files

3 Answers3