Incoming emails are mostly unchecked by SpamAssassin, although we can see in the headers that they were checked with Amavisd-new for viruses.
We've tried many things to fix it, including asking chatGPT for help, but it gave us lots to check but unfortunately no result.
The following is relevant extract from the syslog, showing the details of an incoming email, which was received but was not scanned for spam.
May 24 12:29:53 s1 postfix/smtpd[27204]: NOQUEUE: filter: RCPT from mail-vi1eur04on2122.outbound.protection.outlook.com[40.107.8.122]: <john.doe@external-domain.com>: Sender address triggers FILTER lmtp:[127.0.0.1]:10026; from=<john.doe@external-domain.com> to=<peter@OUR-SERVER-DOMAIN.net> proto=ESMTP helo=<EUR04-VI1-obe.outbound.protection.outlook.com>
May 24 12:29:53 s1 postfix/smtpd[27204]: NOQUEUE: filter: RCPT from mail-vi1eur04on2122.outbound.protection.outlook.com[40.107.8.122]: <john.doe@external-domain.com>: Sender address triggers FILTER lmtp:[127.0.0.1]:10024; from=<john.doe@external-domain.com> to=<peter@OUR-SERVER-DOMAIN.net> proto=ESMTP helo=<EUR04-VI1-obe.outbound.protection.outlook.com>
May 24 12:29:53 s1 postfix/qmgr[27178]: 0F196405AC7: from=<john.doe@external-domain.com>, size=21840, nrcpt=1 (queue active)
May 24 12:29:53 s1 amavis[15925]: (15925-17) LMTP< MAIL FROM:<john.doe@external-domain.com> SIZE=21840\r\n
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup [debug_sender] => undef, "john.doe@external-domain.com" does not match
May 24 12:29:53 s1 amavis[15925]: (15925-17) LMTP> 250 2.1.0 Sender <john.doe@external-domain.com> OK
May 24 12:29:53 s1 amavis[15925]: (15925-17) LMTP :10024 /var/lib/amavis/tmp/amavis-20230524T112825-15925-1_tO9DwM: <john.doe@external-domain.com> -> <peter@OUR-SERVER-DOMAIN.net> SIZE=21840 Received: from s1.OUR-SERVER-DOMAIN.net ([127.0.0.1]) by localhost (s1.OUR-SERVER-DOMAIN.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP for <peter@OUR-SERVER-DOMAIN.net>; Wed, 24 May 2023 12:29:53 +0800 (PST)
May 24 12:29:53 s1 amavis[15925]: (15925-17) Checking: aW84wRL8SoVA [127.0.0.1] <john.doe@external-domain.com> -> <peter@OUR-SERVER-DOMAIN.net>
May 24 12:29:53 s1 amavis[15925]: (15925-17) 2822.From: <john.doe@external-domain.com>
May 24 12:29:53 s1 amavis[15925]: (15925-17) wbl: checking sender <john.doe@external-domain.com>
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup_acl(john.doe@external-domain.com), no match
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup [local_domains] => undef, "john.doe@external-domain.com" does not match
May 24 12:29:53 s1 amavis[15925]: (15925-17) query_keys: john.doe@external-domain.com, @external-domain.com, @.external-domain.com, @.com, @.
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup_sql sel_wblist "john.doe@external-domain.com", query args: "4", [john.doe@external-domain.com,12], [@external-domain.com,12], [@.external-domain.com,12], [@.com,12], [@.,12]
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup_sql, "john.doe@external-domain.com" no match
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup_sql_field(wb), "john.doe@external-domain.com" no matching records
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup => undef, "john.doe@external-domain.com" does not match
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup [blacklist_sender<john.doe@external-domain.com>,blacklist_sender] => undef, "john.doe@external-domain.com" does not match
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup_acl(john.doe@external-domain.com), no match
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup [whitelist_sender<john.doe@external-domain.com>,whitelist_sender] => undef, "john.doe@external-domain.com" does not match
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup_re("john.doe@external-domain.com"), no matches
May 24 12:29:53 s1 amavis[15925]: (15925-17) lookup [score_sender<john.doe@external-domain.com>] => undef, "john.doe@external-domain.com" does not match
May 24 12:29:54 s1 amavis[15925]: (15925-17) about to connect to smtp:127.0.0.1:*, aW84wRL8SoVA FWD from <john.doe@external-domain.com> -> <peter@OUR-SERVER-DOMAIN.net>
May 24 12:29:54 s1 amavis[15925]: (15925-17) smtp cmd> MAIL FROM:<john.doe@external-domain.com> BODY=7BIT
May 24 12:29:54 s1 amavis[15925]: (15925-17) rw_loop sent 112> MAIL FROM:<john.doe@external-domain.com> BODY=7BIT\r\nRCPT TO:<peter@OUR-SERVER-DOMAIN.net> ORCPT=rfc822;peter@OUR-SERVER-DOMAIN.net\r\nDATA\r\n
May 24 12:29:54 s1 postfix/qmgr[27178]: 34E17405AF3: from=<john.doe@external-domain.com>, size=22290, nrcpt=1 (queue active)
May 24 12:29:54 s1 amavis[15925]: (15925-17) aW84wRL8SoVA FWD from <john.doe@external-domain.com> -> <peter@OUR-SERVER-DOMAIN.net>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 34E17405AF3
May 24 12:29:54 s1 amavis[15925]: (15925-17) DSN: sender NOT credible, SA: 0.000, <john.doe@external-domain.com>
May 24 12:29:54 s1 amavis[15925]: (15925-17) lookup [spam_dsn_cutoff_level_bysender] => true, "john.doe@external-domain.com" matches, result="100", matching_key="(constant:100)"
May 24 12:29:54 s1 amavis[15925]: (15925-17) dsn: from MTA 250 NonBlocking:Clean <john.doe@external-domain.com> -> <peter@OUR-SERVER-DOMAIN.net>: on_succ=0, on_dly=1, on_fail=1, never=0, warn_sender=, DSN_passed_on=1, destiny=1, mta_resp: "250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 34E17405AF3"
May 24 12:29:54 s1 amavis[15925]: (15925-17) DSN: SUCC from MTA 250 NonBlocking:Clean, no DSN requested: <john.doe@external-domain.com> -> <peter@OUR-SERVER-DOMAIN.net>
May 24 12:29:54 s1 amavis[15925]: (15925-17) one_response_for_all <john.doe@external-domain.com>: success, r=0,b=0,d=0, ndn_needed=0, '250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 34E17405AF3'
May 24 12:29:54 s1 amavis[15925]: (15925-17) Passed CLEAN {RelayedInbound}, [127.0.0.1] [40.107.8.122] <john.doe@external-domain.com> -> <peter@OUR-SERVER-DOMAIN.net>, Message-ID: <DB9P193MB1339236416A88C8CAFCD9FCCD7419@db9p193mb1339.eurp193.prod.outlook.com>, mail_id: aW84wRL8SoVA, Hits: 0, size: 21833, queued_as: 34E17405AF3, 1095 ms
This is our /etc/postfix/master.cf:
(note that the 2nd line was added by us today, based on other posts we read, but the email above was delivered AFTER we added it, so it doesn't seem to have worked).
smtp inet n - - - - smtpd
-o content_filter=spamassassin
submission inet n - - - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
smtps inet n - - - - smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_wrappermode=yes
-o smtpd_sasl_auth_enable=yes
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
pickup unix n - - 60 1 pickup
cleanup unix n - - - 0 cleanup
qmgr unix n - n 300 1 qmgr
tlsmgr unix - - - 1000? 1 tlsmgr
rewrite unix - - - - - trivial-rewrite
bounce unix - - - - 0 bounce
defer unix - - - - 0 bounce
trace unix - - - - 0 bounce
verify unix - - - - 1 verify
flush unix n - - 1000? 0 flush
proxymap unix - - n - - proxymap
proxywrite unix - - n - 1 proxymap
smtp unix - - - - - smtp
relay unix - - - - - smtp
-o syslog_name=postfix/$service_name
showq unix n - - - - showq
error unix - - - - - error
retry unix - - - - - error
discard unix - - - - - discard
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - - - - lmtp
anvil unix - - - - 1 anvil
scache unix - - - - 1 scache
maildrop unix - n n - - pipe
flags=DRhu user=vmail argv=/usr/bin/maildrop -d vmail ${extension} ${recipient} ${user} ${nexthop} ${sender}
uucp unix - n n - - pipe
flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
ifmail unix - n n - - pipe
flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp unix - n n - - pipe
flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix - n n - 2 pipe
flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
mailman unix - n n - - pipe
flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
${nexthop} ${user}
dovecot unix - n n - - pipe
flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${nexthop}
amavis unix - - - - 2 smtp
-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes
# -o smtp_bind_address=
127.0.0.1:10025 inet n - - - - smtpd
-o content_filter=
-o local_recipient_maps=
-o relay_recipient_maps=
-o smtpd_restriction_classes=
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_end_of_data_restrictions=
-o mynetworks=127.0.0.0/8
-o strict_rfc821_envelopes=yes
-o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
-o smtp_send_xforward_command=yes
-o disable_dns_lookups=yes
127.0.0.1:10027 inet n - n - - smtpd
-o content_filter=
-o local_recipient_maps=
-o relay_recipient_maps=
-o smtpd_restriction_classes=
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_end_of_data_restrictions=
-o mynetworks=127.0.0.0/8
-o strict_rfc821_envelopes=yes
-o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
-o smtp_send_xforward_command=yes
-o milter_default_action=accept
-o milter_macro_daemon_name=ORIGINATING
-o disable_dns_lookups=yes
This is our /etc/postfix/main.cf:
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no
append_dot_mydomain = no
delay_warning_time = 4h
readme_directory = /usr/share/doc/postfix
compatibility_level = 2
smtpd_tls_cert_file = /etc/postfix/smtpd.cert
smtpd_tls_key_file = /etc/postfix/smtpd.key
smtpd_use_tls = yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = s1.OUR-SERVER-DOMAIN.net
alias_maps = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
alias_database = hash:/etc/aliases, hash:/var/lib/mailman/data/aliases
myorigin = /etc/mailname
mydestination = localhost, localhost.localdomain
relayhost =
mynetworks = 127.0.0.0/8 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
html_directory = /usr/share/doc/postfix/html
virtual_alias_domains = proxy:mysql:/etc/postfix/mysql-virtual_alias_domains.cf
virtual_alias_maps = hash:/var/lib/mailman/data/virtual-mailman, proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, proxy:mysql:/etc/postfix/mysql-virtual_alias_maps.cf, proxy:mysql:/etc/postfix/mysql-virtual_email2email.cf
virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql-virtual_domains.cf
virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailboxes.cf
virtual_mailbox_base = /var/vmail
virtual_uid_maps = mysql:/etc/postfix/mysql-virtual_uids.cf
virtual_gid_maps = mysql:/etc/postfix/mysql-virtual_gids.cf
sender_bcc_maps = proxy:mysql:/etc/postfix/mysql-virtual_outgoing_bcc.cf
smtpd_sasl_auth_enable = yes
broken_sasl_auth_clients = yes
smtpd_sasl_authenticated_header = yes
smtpd_restriction_classes = greylisting
greylisting = check_policy_service inet:127.0.0.1:10023
smtpd_recipient_restrictions = reject_invalid_hostname,
reject_unauth_pipelining,
permit_mynetworks,
reject_unknown_recipient_domain,
permit_sasl_authenticated,
reject_non_fqdn_recipient,
reject_unauth_destination,
check_client_access hash:/etc/postfix/rbl_override,
check_recipient_access proxy:mysql:/etc/postfix/mysql-virtual_recipient.cf,
reject_unlisted_recipient,
check_recipient_access mysql:/etc/postfix/mysql-virtual_policy_greylist.cf,
check_policy_service unix:private/quota-status,
permit
smtpd_tls_security_level = may
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_delay_reject = yes
transport_maps = hash:/var/lib/mailman/data/transport-mailman,
proxy:mysql:/etc/postfix/mysql-virtual_transports.cf
relay_domains = mysql:/etc/postfix/mysql-virtual_relaydomains.cf
relay_recipient_maps = mysql:/etc/postfix/mysql-virtual_relayrecipientmaps.cf
smtpd_sender_login_maps = proxy:mysql:/etc/postfix/mysql-virtual_sender_login_maps.cf
proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $sender_bcc_maps $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $smtpd_sender_login_maps $smtpd_recipient_restrictions
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_sasl_authenticated,
permit_mynetworks,
check_helo_access regexp:/etc/postfix/helo_access,
reject_non_fqdn_helo_hostname,
reject_invalid_helo_hostname,
reject_unknown_helo_hostname,
check_helo_access regexp:/etc/postfix/blacklist_helo,
permit
smtpd_sender_restrictions = check_sender_access mysql:/etc/postfix/mysql-virtual_sender.cf,
check_sender_access regexp:/etc/postfix/tag_as_originating.re,
permit_mynetworks,
permit_sasl_authenticated,
check_sender_access regexp:/etc/postfix/tag_as_foreign.re,
permit
smtpd_client_restrictions = permit_mynetworks,
permit_sasl_authenticated,
reject_unknown_client_hostname,
check_client_access mysql:/etc/postfix/mysql-virtual_client.cf,
reject_rbl_client cbl.abuseat.org,
reject_rbl_client b.barracudacentral.org,
strict_rfc821_envelopes = yes
postscreen_greet_action = enforce
smtpd_client_message_rate_limit = 2
anvil_rate_time_unit = 60s
maildrop_destination_concurrency_limit = 2
maildrop_destination_recipient_limit = 2
virtual_transport = lmtp:unix:private/dovecot-lmtp
header_checks = regexp:/etc/postfix/header_checks
mime_header_checks = regexp:/etc/postfix/mime_header_checks
nested_header_checks = regexp:/etc/postfix/nested_header_checks
body_checks = regexp:/etc/postfix/body_checks
owner_request_special = no
smtp_tls_security_level = may
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_protocols = !SSLv2,!SSLv3, !TLSv1, !TLSv1.1
smtp_tls_protocols = !SSLv2,!SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_exclude_ciphers = RC4, aNULL
smtp_tls_exclude_ciphers = RC4, aNULL
smtpd_tls_mandatory_ciphers = medium
tls_medium_cipherlist = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
tls_preempt_cipherlist = no
enable_original_recipient = yes
dovecot_destination_recipient_limit = 1
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
message_size_limit = 0
receive_override_options = no_address_mappings
content_filter = lmtp:[127.0.0.1]:10024
This is my /etc/spamassassin/local.cf:
required_score 4.0
ifplugin Mail::SpamAssassin::Plugin::Shortcircuit
endif # Mail::SpamAssassin::Plugin::Shortcircuit
The line `required_score 4.0` was added by us. Apparently it raises the maximum message size that will be scanned, from the default of 256kb, to 4MB. Didn't solve the problem though!
We don't see any logs for Amavis or SpamAssassin in `/var/logs/`, so we have none to share.
Another point is that we're not sure whether it's connected, but we've noticed that the emails which we specifically whitelisted in ISPConfig Postfix Global Whitelist, seem to get scanned for spam and even get the "***SPAM***" subject line added!
Please help to fix this!
**System:**
Linux VPS running Ubuntu 18.04 LTS
ISPConfig v3.1 panel
Postfix v3.3.0, Dovecot v2.2.33.2, Amavisd-new v2.11.0 , SpamAssassin v3.4.2
