3

I run a VPS, and I have to say that by far, getting SpamAssassin working right has been the most frustrating thing of all. It keeps missing spam mails and I see autolearn=ham in the header, yet when I find the message in /var/vmail on the server and run spamassassin -t to test it, it yields a score of above 100 (because the sender is on my blacklist). Doesn't make any sense.

This is the X-Spam-Status in the email's header:

X-Spam-Status: No, score=-0.5 required=3.4 tests=BAYES_05,HTML_MESSAGE,
    NO_RECEIVED,NO_RELAYS autolearn=ham version=3.3.2

Why would the score here by -0.5 and why is it being autolearned as ham?? Yet running spamassassin -t on this message yields:

Content analysis details:   (103.0 points, 3.4 required)

 pts rule name              description
---- ---------------------- --------------------------------------------------
 100 USER_IN_BLACKLIST      From: address is in the user's black-list
 0.0 FREEMAIL_FROM          Sender email is commonly abused enduser mail provider
                            (advertise.bz222hwpxo[at]gmail.com)
-0.0 NO_RELAYS              Informational: message was not relayed via SMTP
 3.0 BAYES_95               BODY: Bayes spam probability is 95 to 99%
                            [score: 0.9502]
 0.0 T_OBFU_HTML_ATTACH     BODY: HTML attachment with non-text MIME type
 0.0 T_HTML_ATTACH          HTML attachment to bypass scanning?
-0.0 NO_RECEIVED            Informational: message has no Received headers

I'm thoroughly confused. Any help would be greatly appreciated!

Config Files

/etc/postfix/master.cf

#
# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master").
#
# Do not forget to execute "postfix reload" after editing this file.
#
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       -       -       -       smtpd
  -o content_filter=spamassassin
#smtp      inet  n       -       -       -       1       postscreen
#smtpd     pass  -       -       -       -       -       smtpd
#dnsblog   unix  -       -       -       -       0       dnsblog
#tlsproxy  unix  -       -       -       -       0       tlsproxy
submission inet n       -       -       -       -       smtpd
  -o content_filter=spamassassin
#  -o syslog_name=postfix/submission
#  -o smtpd_tls_security_level=encrypt
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING
smtps     inet  n       -       -       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_auth_only=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject_unauth_destination,reject
  -o smtpd_sasl_security_options=noanonymous,noplaintext
  -o smtpd_sasl_tls_security_options=noanonymous
#  -o milter_macro_daemon_name=ORIGINATING
#628       inet  n       -       -       -       -       qmqpd
pickup    fifo  n       -       -       60      1       pickup
cleanup   unix  n       -       -       -       0       cleanup
qmgr      fifo  n       -       n       300     1       qmgr
#qmgr     fifo  n       -       n       300     1       oqmgr
tlsmgr    unix  -       -       -       1000?   1       tlsmgr
rewrite   unix  -       -       -       -       -       trivial-rewrite
bounce    unix  -       -       -       -       0       bounce
defer     unix  -       -       -       -       0       bounce
trace     unix  -       -       -       -       0       bounce
verify    unix  -       -       -       -       1       verify
flush     unix  n       -       -       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       -       -       -       smtp
relay     unix  -       -       -       -       -       smtp
#       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq     unix  n       -       -       -       -       showq
error     unix  -       -       -       -       -       error
retry     unix  -       -       -       -       -       error
discard   unix  -       -       -       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       -       -       -       lmtp
anvil     unix  -       -       -       -       1       anvil
scache    unix  -       -       -       -       1       scache
#
# ====================================================================
# Interfaces to non-Postfix software. Be sure to examine the manual
# pages of the non-Postfix software to find out what options it wants.
#
# Many of the following services use the Postfix pipe(8) delivery
# agent.  See the pipe(8) man page for information about ${recipient}
# and other message envelope options.
# ====================================================================
#
# maildrop. See the Postfix MAILDROP_README file for details.
# Also specify in main.cf: maildrop_destination_recipient_limit=1
#
maildrop  unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
#
# ====================================================================
#
# Recent Cyrus versions can use the existing "lmtp" master.cf entry.
#
# Specify in cyrus.conf:
#   lmtp    cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
#
# Specify in main.cf one or more of the following:
#  mailbox_transport = lmtp:inet:localhost
#  virtual_transport = lmtp:inet:localhost
#
# ====================================================================
#
# Cyrus 2.1.5 (Amos Gouaux)
# Also specify in main.cf: cyrus_destination_recipient_limit=1
#
#cyrus     unix  -       n       n       -       -       pipe
#  user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
#
# ====================================================================
# Old example of delivery via Cyrus.
#
#old-cyrus unix  -       n       n       -       -       pipe
#  flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user}
#
# ====================================================================
#
# See the Postfix UUCP_README file for configuration details.
#
uucp      unix  -       n       n       -       -       pipe
  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
#
# Other external delivery methods.
#
ifmail    unix  -       n       n       -       -       pipe
  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp     unix  -       n       n       -       -       pipe
  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix  -   n   n   -   2   pipe
  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
mailman   unix  -       n       n       -       -       pipe
  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
  ${nexthop} ${user}

dovecot      unix   -        n      n       -       -   pipe
        flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/dovecot-lda -d $(recipient)

spamassassin unix   -        n      n       -       -   pipe
        user=spamd argv=/usr/bin/spamc -f -e
        /usr/sbin/sendmail -oi -f ${sender} ${recipient}

#sp-order     unix   -        n      n       -       -   pipe
#        user=sara argv=/home/sara/order-notify -f ${sender} -- ${recipient}

/etc/spamassassin/local.cf

# This is the right place to customize your installation of SpamAssassin.
#
# See 'perldoc Mail::SpamAssassin::Conf' for details of what can be
# tweaked.
#
# Only a small subset of options are listed below
#
###########################################################################

bayes_path /var/lib/spamassassin/.spamassassin/bayes

#   Add *****SPAM***** to the Subject header of spam e-mails
#
rewrite_header Subject [***** SPAM _SCORE_ *****]


#   Save spam messages as a message/rfc822 MIME attachment instead of
#   modifying the original message (0: off, 2: use text/plain instead)
#
report_safe 0


#   Set which networks or hosts are considered 'trusted' by your mail
#   server (i.e. not spammers)
#
# trusted_networks 212.17.35.


#   Set file-locking method (flock is not safe over NFS, but is faster)
#
# lock_method flock


# Network checks
skip_rbl_checks 0
use_razor2 0
#use_dcc 0
use_pyzor 0

#   Set the threshold at which a message is considered spam (default: 5.0)
#
required_score 3.4


#   Use Bayesian classifier (default: 1)
#
use_bayes 1
use_bayes_rules 1

#   Bayesian classifier auto-learning (default: 1)
#
bayes_auto_learn 1


#   Set headers which may provide inappropriate cues to the Bayesian
#   classifier
#
# bayes_ignore_header X-Bogosity
# bayes_ignore_header X-Spam-Flag
# bayes_ignore_header X-Spam-Status


#   Some shortcircuiting, if the plugin is enabled
# 
ifplugin Mail::SpamAssassin::Plugin::Shortcircuit
#
#   default: strongly-whitelisted mails are *really* whitelisted now, if the
#   shortcircuiting plugin is active, causing early exit to save CPU load.
#   Uncomment to turn this on
#
# shortcircuit USER_IN_WHITELIST       on
# shortcircuit USER_IN_DEF_WHITELIST   on
# shortcircuit USER_IN_ALL_SPAM_TO     on
# shortcircuit SUBJECT_IN_WHITELIST    on

#   the opposite; blacklisted mails can also save CPU
#
# shortcircuit USER_IN_BLACKLIST       on
# shortcircuit USER_IN_BLACKLIST_TO    on
# shortcircuit SUBJECT_IN_BLACKLIST    on

#   if you have taken the time to correctly specify your "trusted_networks",
#   this is another good way to save CPU
#
# shortcircuit ALL_TRUSTED             on

#   and a well-trained bayes DB can save running rules, too
#
# shortcircuit BAYES_99                spam
# shortcircuit BAYES_00                ham

endif # Mail::SpamAssassin::Plugin::Shortcircuit

blacklist_from mike.newsletter30@gmail.com
blacklist_from advertise*@gmail.com
CaptSaltyJack
  • 638
  • 2
  • 13
  • 36
  • 3
    1. How are you calling spamassassin (amavisd-new, through clamd, or some other method)? 2. Is this a sitewide install, or just for your account? 3. Posting your configuration files would probably help.. – NickW Jan 07 '14 at 15:50
  • What NickW said. `spamassassin` is extremely configurable, which means its results vary greatly depending on who it's run as, and how it's invoked. – MadHatter Jan 07 '14 at 15:56
  • Via postfix I believe. I'll post the master.cf file above. This is sitewide, running spamd. I'll post some other related config files as well. – CaptSaltyJack Jan 07 '14 at 15:56
  • Arrrgh postfix. I'm not good with that; I don't know if it invokes spamassassin personally or impersonally, if you see what I mean. Hopefully someone else does. – MadHatter Jan 07 '14 at 16:08
  • I'm actually willing to pay money to get this solved. It's been a thorn in my side for many moons. – CaptSaltyJack Jan 07 '14 at 16:10
  • 1
    Calling spamassassin from postfix means (I believe) that spamassassin will be called with the postfix user, not your user, which is why on the spamassassin page, they recommend using a script to call spamassassin, so you can pass the user. See: http://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix – NickW Jan 07 '14 at 16:23
  • 1
    Obviously, if the blacklist is under your user, the postfix user won't see it, and the Bayes DB will not be trained with information specific to you. – NickW Jan 07 '14 at 16:24
  • blacklist is under the spamd user home (`/var/lib/spamassassin`). Also, isn't postfix using spamc which is the client to spamd? So it would actually be invoked using the spamd user? – CaptSaltyJack Jan 07 '14 at 16:33
  • The blacklist would need to be under the user who calls `spamc`, or the user specified in the `spamc` call. So if user "postfix" is calling spamc, then that user's blacklist would be getting used. I don't know how to configure postfix for SA, but in my configuration spamc gets called with the `-u $USER` command line argument to specify what user's Bayesian filters, blacklists, and filtering to load. – Chris S Jan 07 '14 at 16:42
  • What about the `user=spamd` part in this line in `master.cf`? Wouldn't that cause `spamc` to run as the `spamd` user? `user=spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}` – CaptSaltyJack Jan 07 '14 at 17:09
  • I will try the script on this page and see if it works out: http://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix – CaptSaltyJack Jan 07 '14 at 17:10
  • Since you are using the postfix pipe method, you might want to have a look at the Flags you can set with the method, I think that possibly the pipe is not passing the values correctly. The spamassassin example uses 'Flags=Rq', 'q' being the important bit, as it makes sure that all white space is passed to the command. http://www.postfix.org/pipe.8.html – NickW Jan 07 '14 at 17:11
  • Someone feel free to post an answer BTW. @NickW, do you want to post one using that link you suggested? http://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix – CaptSaltyJack Jan 09 '14 at 23:57
  • @CaptSaltyJack you managed to get it working with the method suggested on the spamassassin page? – NickW Jan 10 '14 at 09:12
  • @NickW Trying it out. I'm using the spamfilter.sh script suggestion and giving it a few days to see if spam is properly filed away and not mistaken as ham. – CaptSaltyJack Jan 10 '14 at 18:25

1 Answers1

2

I suspect you are running your spamassassin -t test as a different user than postfix is invoking it as. If postfix has already downgraded privileges from root, it may not be able to run as any user other than itself.

Since it appears you're running every message in SpamAssassin as the same user, you don't need separate accounts in SpamAssassin. Try using /etc/spamassassin/local.cf (or wherever the system config lives) rather than ~/.spamassassin/user_prefs for your blacklist and, if that works, make sure other per-user items are done on a global basis as well, especially sitewide bayes. (The only other one I can think of is AWL.)

If you do want per-user items, you'll have to figure out which user you're running as. Either dig through your logs or maybe you can run top and watch as a test message comes in (though it'll be fast...).

Adam Katz
  • 951
  • 8
  • 17
  • You can look in /var/log/mail.log to see some messages from spamassassin which include information about what user it is running as. – David Grayson Oct 29 '14 at 20:06