Script to search bots from log file

Question

I have a little log-file, where i should find the bots.

File:

Mon, 22 Aug 2016 13:15:39 +0200|178.57.66.225|fxsciaqulmlk| - |user logged in| -
Mon, 22 Aug 2016 13:15:39 +0200|178.57.66.225|fxsciaqulmlk| - |user changed password| -
Mon, 22 Aug 2016 13:15:39 +0200|178.57.66.225|fxsciaqulmlk| - |user logged off| -
Mon, 22 Aug 2016 13:15:42 +0200|178.57.66.225|faaaaaa11111| - |user logged in| -
Mon, 22 Aug 2016 13:15:49 +0200|178.57.66.215|terdsfsdfsdf| - |user logged in| -
Mon, 22 Aug 2016 13:15:49 +0200|178.57.66.215|terdsfsdfsdf| - |user changed password| -
Mon, 22 Aug 2016 13:15:49 +0200|178.57.66.215|terdsfsdfsdf| - |user logged off| -
Mon, 22 Aug 2016 13:15:59 +0200|178.57.66.205|erdsfsdfsdf| - |user logged in| -
Mon, 22 Aug 2016 13:15:59 +0200|178.57.66.205|erdsfsdfsdf| - |user logged in| -
Mon, 22 Aug 2016 13:15:59 +0200|178.57.66.205|erdsfsdfsdf| - |user changed password| -
Mon, 22 Aug 2016 13:15:59 +0200|178.57.66.205|erdsfsdfsdf| - |user logged off| -
Mon, 22 Aug 2016 13:17:50 +0200|178.57.66.205|abcbbabab| - |user logged in| -
Mon, 22 Aug 2016 13:17:50 +0200|178.57.66.205|abcbbabab| - |user changed password| -
Mon, 22 Aug 2016 13:17:50 +0200|178.57.66.205|abcbbabab| - |user changed profile| -
Mon, 22 Aug 2016 13:17:50 +0200|178.57.66.205|abcbbabab| - |user logged off| -
Mon, 22 Aug 2016 13:19:19 +0200|178.56.66.225|fxsciulmla| - |user logged in| -
Mon, 22 Aug 2016 13:19:19 +0200|178.56.66.225|fxsciulmla| - |user changed password| -
Mon, 22 Aug 2016 13:19:19 +0200|178.56.66.225|fxsciulmla| - |user logged off| -
Mon, 22 Aug 2016 13:20:42 +0200|178.57.67.225|faaaa0a1111| - |user logged in| -

In this file, i have a lot ip, logins, usernames. first of all, i thought about uniq and count of ip. So, what i should do. I should use events only: user logged in, user changed off, user logged off. Next, i should show the users, that logged in the same time. First three lines, user has logged in, changed password, logged off. This time is 13:15:39. And user fxsciaqulmlk from ip 178.57.66.225 is a bot, because operation of event has made at the same time. My script:

log_file=/root/log
log_after=/root/after_log
temp_file=/root/temp
temp_file2=/root/temp2
uniq_file=/root/uniq
uniq_file2=/root/uniq2
result_uniq=/root/result_uniq
result_file=/root/result

cat /dev/null > $log_after
cat /dev/null > $temp_file
cat /dev/null > $temp_file2
cat /dev/null > $uniq_file
cat /dev/null > $uniq_file2
cat /dev/null > $result_file

grep "changed password\|logged in\|logged off" $log_file > $log_after
cat $log_after | awk '{print $6}' | awk -F "|" '{print $2,$3}' | tail -n 20 > $temp_file
cat $log_after | awk '{print $5}' | tail -n 20 > $temp_file2
uniq -c $temp_file | awk '{print $1}' > $uniq_file
uniq -c $temp_file2 | awk '{print $1}' > $uniq_file2

awk 'FNR==NR{a[$1]++;next}!a[$1]' $uniq_file  $uniq_file2  > $result_uniq

if [ -s $result_uniq ] && [ -f $result_uniq ]; then
 echo "File is not  empty"
 echo "Differences:"
 cat $result_uniq
 echo "Need to think"
 exit 0
else
 echo "File is empty"
 echo "We can use one file from uniq"
fi

for i in `uniq -c $temp_file | awk '{print $1}'`; do
if [ $i -gt 2 ]; then
s=`uniq -c $temp_file | awk '$1 == '$i | awk '{print $3}'`
ss=`uniq -c $temp_file | awk '$1 == '$i | awk '{print $2}'`
echo "Tho boot is user $s with ip $ss"
fi
done

This question is an exact duplicate of:

Script to search bots from log file 1 answer

I have written this message before. So, i have done my task, but i have a little not correct output. So... I will start. I have a log file:

Mon, 22 Aug 2016 13:15:39 +0200|178.57.66.225|fxsciaqulmlk| - |user logged in| - Mon, 22 Aug 2016 13:15:39 +0200|178.57.66.225|fxsciaqulmlk| - |user changed password| - Mon, 22 Aug 2016 13:15:39 +0200|178.57.66.225|fxsciaqulmlk| - |user logged off| - Mon, 22 Aug 2016 13:15:42 +0200|178.57.66.225|faaaaaa11111| - |user logged in| - Mon, 22 Aug 2016 13:15:49 +0200|178.57.66.215|terdsfsdfsdf| - |user logged in| - Mon, 22 Aug 2016 13:15:49 +0200|178.57.66.215|terdsfsdfsdf| - |user changed password| - Mon, 22 Aug 2016 13:15:49 +0200|178.57.66.215|terdsfsdfsdf| - |user logged off| - Mon, 22 Aug 2016 13:15:59 +0200|178.57.66.205|erdsfsdfsdf| - |user logged in| - Mon, 22 Aug 2016 13:15:59 +0200|178.57.66.205|erdsfsdfsdf| - |user logged in| - Mon, 22 Aug 2016 13:15:59 +0200|178.57.66.205|erdsfsdfsdf| - |user changed password| - Mon, 22 Aug 2016 13:15:59 +0200|178.57.66.205|erdsfsdfsdf| - |user logged off| - Mon, 22 Aug 2016 13:17:50 +0200|178.57.66.205|abcbbabab| - |user logged in| - Mon, 22 Aug 2016 13:17:50 +0200|178.57.66.205|abcbbabab| - |user changed password| - Mon, 22 Aug 2016 13:17:50 +0200|178.57.66.205|abcbbabab| - |user changed profile| - Mon, 22 Aug 2016 13:17:50 +0200|178.57.66.205|abcbbabab| - |user logged off| - Mon, 22 Aug 2016 13:19:19 +0200|178.56.66.225|fxsciulmla| - |user logged in| - Mon, 22 Aug 2016 13:19:19 +0200|178.56.66.225|fxsciulmla| - |user changed password| - Mon, 22 Aug 2016 13:19:19 +0200|178.56.66.225|fxsciulmla| - |user logged off| - Mon, 22 Aug 2016 13:20:42 +0200|178.57.67.225|faaaa0a1111| - |user logged in| -

So, what i should do. I should use events only: user logged in, user changed off, user logged off. Next, i should show the users, that logged in the same time. First three lines, user has logged in, changed password, logged off. This time is 13:15:39. And user fxsciaqulmlk from ip 178.57.66.225 is a bot, because operation of event has made at the same time. My script:

#!/bin/bash
# you should add it script in crontab, like this
#*/2 * * * *      /name_of_this_script.sh
# you should change variable way in $log_file to your own way

log_file=/root/log
log_after=/root/after_log
temp_file=/root/temp
temp_file2=/root/temp2
uniq_file=/root/uniq
uniq_file2=/root/uniq2
result_uniq=/root/result_uniq
result_file=/root/result

cat /dev/null > $log_after
cat /dev/null > $temp_file
cat /dev/null > $temp_file2
cat /dev/null > $uniq_file
cat /dev/null > $uniq_file2
cat /dev/null > $result_file

grep "changed password\|logged in\|logged off" $log_file > $log_after
cat $log_after | awk '{print $6}' | awk -F "|" '{print $2,$3}' | tail -n 20 > $temp_file
cat $log_after | awk '{print $5}' | tail -n 20 > $temp_file2
uniq -c $temp_file | awk '{print $1}' > $uniq_file
uniq -c $temp_file2 | awk '{print $1}' > $uniq_file2

awk 'FNR==NR{a[$1]++;next}!a[$1]' $uniq_file  $uniq_file2  > $result_uniq

if [ -s $result_uniq ] && [ -f $result_uniq ]; then
 echo "File is not  empty"
 echo "Differences:"
 cat $result_uniq
 echo "Need to think"
 exit 0
else
 echo "File is empty"
 echo "We can use one file from uniq"
fi

for i in `uniq -c $temp_file | awk '{print $1}'`; do
if [ $i -gt 2 ]; then
s=`uniq -c $temp_file | awk '$1 == '$i | awk '{print $3}'`
ss=`uniq -c $temp_file | awk '$1 == '$i | awk '{print $2}'`
echo "Tho boot is user $s with ip $ss"
fi
done

Everything ok. But i have bad result:

Tho boot is user fxsciaqulmlk
terdsfsdfsdf
abcbbabab
fxsciulmla with ip 178.57.66.225
178.57.66.215
178.57.66.205
178.56.66.225
Tho boot is user fxsciaqulmlk
terdsfsdfsdf
abcbbabab
fxsciulmla with ip 178.57.66.225
178.57.66.215
178.57.66.205
178.56.66.225
Tho boot is user erdsfsdfsdf with ip 178.57.66.205
Tho boot is user fxsciaqulmlk
terdsfsdfsdf
abcbbabab
fxsciulmla with ip 178.57.66.225
178.57.66.215
178.57.66.205
178.56.66.225
Tho boot is user fxsciaqulmlk
terdsfsdfsdf
abcbbabab
fxsciulmla with ip 178.57.66.225
178.57.66.215
178.57.66.205
178.56.66.225

Where i have a mistake ? I can't understand, where i have a mistake in code ? I am so sorry for repeating question, but i have done this script, i need a little help.

score 0 · Answer 1 · answered Jan 18 '17 at 15:42

It is difficult to tell what you are asking for due to the language barrier. I am not clear if you are asking for a way to loop through all the entries in your log file, or if you are trying to determine how many times a user triggers a specific event. In either case, it seems you would need a way to look at every line of your log file and split out the different fields.

There are many ways to go about this. A simple approach could look something like this:

#!/bin/bash
log="/root/auth.log"
result="/root/bots.result"

# we probably don't need the intermediate file 
# temp=/root/log.temp

# a simple redirect can clear the output file
> $result

# read every line of the log file
while read line; do
   # split fields with 'cut' using whitespace as a delimiter
   date=`echo "$line" | cut -d' ' -f1-5`

   # split using pipe as a delimiter
   ip=`echo "$line" | cut -d'|' -f2`
   user=`echo "$line" | cut -d'|' -f3`
   event=`echo "$line" | cut -d'|' -f5`

   # report results
   echo "date:$date user:$user from ip:$ip $event" >> "$results"
done < "$log"

Ok. The bot do auth, change password & log out in the same time. For example, look at first 3 linesin my log. Time, when bot made auth, changed password & logged out is the same time. 13:15:39. From this conditions, i have understood. This is a boot — Piduna Valeriu, Jan 18 '17 at 15:52
if user made in the same time operations of auth, change password & log out, i should mark this user, like a bot — Piduna Valeriu, Jan 18 '17 at 15:56

Script to search bots from log file

1 Answers1