2

This problem comes under the context of pop-before-smtp / Postfix / Dovecot, but if I knew Perl string parsing, I could come up with an answer myself. However, I'm so lost I don't even know the precise question. To wit:

We've been using Postfix for a LONG time now and are kind of hooked on it. Now we need to "move into the modern era" and let people SEND email from our SMTP server(s) even when they're outside our network. So, tasked with this job, I've found pop-before-smtp.

You can find it here.

So, I've got it all configured but it fails in testing. I've troubleshot it using the directions here, and determined that the Perl that's trying to parse the log appears to be incorrect. We're using Dovecot as our IMAP / POP server, and there are three choices given in the configuration file. Here is an excerpt from the config file showing the three sets:

# For Dovecot POP3/IMAP when using syslog.
#$pat = '^[LOGTIME] \S+ (?:dovecot: )?(?:imap|pop3)-login: ' .
#    'Login: .*? (?:\[|rip=)[:f]*(\d+\.\d+\.\d+\.\d+)[],]';
#$out_pat = '^[LOGTIME] \S+ (?:dovecot: )?(?:imap|pop3)-login: ' .
#    'Disconnected.*? (?:\[|rip=)[:f]*(\d+\.\d+\.\d+\.\d+)[],]';

# For Dovecot POP3/IMAP when it does its own logging.
##$logtime_pat = '(\d\d\d\d-\d+-\d+ \d+:\d+:\d+)';
#$pat = '^dovecot: [LOGTIME] Info: (?:imap|pop3)-login: ' .
#    'Login: .+? rip=[:f]*(\d+\.\d+\.\d+\.\d+),';
#$out_pat = '^dovecot: [LOGTIME] Info: (?:imap|pop3)-login: ' .
#    'Disconnected.*? rip=[:f]*(\d+\.\d+\.\d+\.\d+),';

# For older Dovecot POP3/IMAP when it does its own logging.
#$pat = '^(?:imap|pop3)-login: [LOGTIME] Info: ' .
#    'Login: \S+ \[[:f]*(\d+\.\d+\.\d+\.\d+)\]';
#$out_pat = '^(?:imap|pop3)-login: [LOGTIME] Info: ' .
#    'Disconnected.*? \[[:f]*(\d+\.\d+\.\d+\.\d+)\]';

One is supposed to uncomment the ones that apply, however, none of them work.

I surmise that 'pat' is the pattern for login, and out-pat is the pattern for logging out or otherwise disconnecting.

The actual log record format is clearly different than any of these three, but they're close. Here are an example pair:

Mar 11 17:53:55 imap-login: Info: Login: user=<username>, method=PLAIN, rip=208.54.4.205, lip=192.168.1.1, TLS

Mar 11 17:59:10 IMAP(username): Info: Disconnected: Logged out bytes=352/43743

When using POP, 'imap-login' is replaced by 'pop-login', and on log-out, 'POP' replaces 'IMAP' - why the changes in capitalization I can't say!

Importand data are: The timestamp, the username, and, when logging in, the "remote" ip ("rip").

Given enough time, I may be able to piece together something that works, but since I don't actually know Perl, this is kind of tough. Please help me write new rules to parse the logging output used with our Dovecot package.

Charles
  • 50,943
  • 13
  • 104
  • 142
Richard T
  • 4,570
  • 5
  • 37
  • 49

1 Answers1

1

The (:?.. portion of a Perl regular expression asks for clustering but not capturing; this allows entire groups to be matched or ignored as as group without influencing the capture group numbers; all the lines capture exactly one field, the IP to allow. (Which is a little odd, I might have expected both username and IP, but this might be easier in the long run.)

# For Dovecot POP3/IMAP when using syslog.
$pat = '^[LOGTIME] \S+ (?:imap|pop3)-login: Info: ' .
    'Login: .*? (?:\[|rip=)[:f]*(\d+\.\d+\.\d+\.\d+)[],]';
# not necessary? see comment header START OF PATTERNS
# $out_pat = '^[LOGTIME] \S+ (?:IMAP|POP3)\(\S+\): Info: ' .
#    'Disconnected.*';

I've removed the dovecot pieces since they weren't in your input. I added the Info: to both lines. I've modified the $out_pat to use IMAP(username) instead of the no-longer-there imap-login from the original. (The use of \S+ will break if usernames have spaces. Since this assumption was made elsewhere in the file, I hope it's fine.)

Since there is no longer any IP address to capture for the logout line, it is probably best to not define $out_pat -- the START OF PATTERNS comment block includes the phrase If the entry of your choice also provides $out_pat, you should uncomment that variable as well, which allows us to keep track of users who are still connected to the server (e.g. Thunderbird caches open IMAP connections).

I haven't tested this but I have good feelings about it.

sarnold
  • 102,305
  • 22
  • 181
  • 238
  • THANK YOU. Unfortunately, it doesn't appear to work. Still, I'm getting closer. I think the reason it thinks it only needs the IP is because the disconnect (out_pat) seems to be looking for a matching IP address, which it won't get. This means it's not really going to work quite right, I think. ...I cannot really afford the time to become a PERL programmer, on top of all the other programming I have to do! But it appears that now might be the time. Any pointers to solving ONLY this problem (get BOTH username and IP and then match-up the logout) - the MINIMUM I need to learn, etc? Tnx. – Richard T Mar 12 '12 at 03:02
  • Ah crap I thought it seemed too easy. Chalk this up to always running _tests_ before releasing software. Can you modify `dovecot` to force it to emit the IP address on logout as well? (Note that the _worst_ case is probably not too bad -- after all, someone _did_ use that IP address legitimately, it won't matter too much if one of your users is a zombie, you'll get either five or ten minutes of spam sent through or hours sent through and either way you'll probably notice it "quickly".) – sarnold Mar 12 '12 at 03:34
  • I don't think it's too much to be concerned about it as there's a configurable timeout, and it would ONLY open up to spam on the specific IP the client used, so the odds of it being safe are very high. ...Now, I'm trying to figure out why your code doesn't work! -smile- A fall-back plan may be to either change Dovecot's logging characteristics OR maybe replace Dovecot with a different IMAP server, though I'm not anxious to work out new password authentication... – Richard T Mar 12 '12 at 03:46
  • I've updated the `$out_pat` to a rule that should match -- but since there is nothing captured, it is liable to _still_ fail. But try _not_ defining `$out_pat`, I'm not sure it is necessary. – sarnold Mar 12 '12 at 07:41