0

I am currently trying to automatically append unique IP's that attack my Artillery Honeypot to a text file.

I've gotten to a point in this script where I am monitoring syslog for changes (where artillery puts new attack logs), and running the grep command to find all unique IP's in syslog each time it is modified.

What I need to do now is pipe the grep command output to 'something' that will only append unique IP's that aren't already in the text file they are to be appended to.

#!/bin/bash
import inotify-tools

inotifywait -r -m -e modify /var/log/syslog | 
while read path _ file; do
      grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}" /var/log/syslog | sort | uniq | ??????

done

I'm just looking for the command I need to pipe to in order to append the unique IP's to a text file, but only if they don't exist in the text file already. Thank you

Dennis Williamson
  • 346,391
  • 90
  • 374
  • 439
KHaight19
  • 1

1 Answers1

0
inotifywait -r -m -e modify /var/log/syslog | while read path _ file; do
        grep -E -o "([0-9]{1,3}[.]){3}[0-9]{1,3}" /var/log/syslog |
        sort | uniq | ??????

You're rescanning the entirety of /var/log/syslog every time it changes. The O(N²) is going to bite you, hard.

Fire off tail -f /var/log/syslog at startup and on inotifywait creation attempts, or if you're using systemd you can just do a single journalctl -f. pipe the output through e.g. this lex:

%option main nodefault
IPBYTE  [0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5]
%%
{IPBYTE}("."{IPBYTE}){3} puts(yytext);
[0-9.]+
[^0-9]{0,8192}

to extract all the ipv4 addresses, pipe the results through a single

awk 'ARGIND==1 { seen[$0]++ } !seen[$0]++' known.ips - >> known.ips

to append just the previously-unseen addresses.

So with systemd to catch up since boot and stay current it's

(journalctl -b; journalctl -f) | extractip4s | that.awk known.ips - >>known.ips

otherwise replace the journalctl with a tail -f and an inotifywait -me create loop firing off more, in a subshell.

jthill
  • 55,082
  • 5
  • 77
  • 137
  • On subsequent runs you might want to populate the `seen` array at startup by reading in the `known.ips` file. I _think_ that's what your last snippet is doing but you don't say what `extractip4s` and `that.awk` are. – Dennis Williamson Jun 04 '19 at 00:23
  • @DennisWilliamson hmmm, yes, this already does exactly what you suggest. It seemed to me "this lex [...] to extract all the ipv4 addresses" was pretty clearly what `extractip4s` was referring to, and since there's only one awk in the answer, it likewise seemed to me there was no chance of confusion about what `that.awk` could mean. – jthill Jun 04 '19 at 03:51
  • So after more fiddling with this, it seems like reading /var/log/syslog for the IP's is unreliable. I think it would be much better to append the screen output of the Artillery Honeypot window into a file, and read my IP's out from that. Does this seem feasible? – KHaight19 Jun 07 '19 at 17:05