Fully transparent outbound anti-spam

From Halon, SMTP software for hosting providers
Jump to: navigation, search

The Halon SMTP software can be used as a fully transparent spam filter, with outbound traffic destined for port 25 being re-routed to the system by the firewalls or routers.

Considerations

We recommend doing outbound anti-spam as a store-and-forward MTA (configure it as a relay/smarthost) or semi-transparent proxy whenever possible, because transparent filtering comes with a few disadvantages. Most importantly, transparent filtering disables TLS (in order to be able to intercept the outbound traffic), and if the sending server is configured to use for example DANE email delivery for such messages will fail. From an anti-spam effectiveness perspective it performs slightly worse than alternative methods, because it makes smart queue scheduling and other deliverability techniques such as dedicated bulk IPs and source hashing impossible to use. In other words, before you deploy a transparent spam filter, consider the alternatives. If you do it because of SPF, consider using SRS to rewrite the sender address or semi-transparent proxy mode instead. We do however realise that fully transparent filtering is needed in some cases; for example when deployed as a drop-in replacement in front of servers that you do not control yourselves (VPS/cloud hosting).

Installation

The typical setup consists of (at least) two proxy servers running proxsmtp and two Halon (spam filtering) nodes.

Proxy nodes

The proxy server(s) can be any Linux distro of your choice. Follow the instructions to install and configure proxsmtpd and haproxy.

Halon (filter) nodes

On the Mail > Relay table page, configure a listener (mail_server) on a port of your choice (as configured in proxsmtp/haproxy), and type both proxy servers' IPs in the XCLIENT field, and add a catch-all (*) domain to it. The RCPT and DATA contexts can be used, but in this guide we'll only use DATA. Below is an example where we use the sending server's IP as rate limit identifier, and also use API calls to determine the customer ID for a given IP in order to rate limit that as well.

include "file:1"// Logging server http://wiki.halon.se/End-user
include "file:2"// External lookups https://wiki.halon.se/API_calls

if ($actionid 1Done(); // Only run once per email, not per recipient

$customer ""// In order to do rate limits per customer
$api api_call("&type=ip-to-cust&ip=$1", [$senderip]);
if (isset(
$api["id"])) {
    
$customer $api["id"];
    
SetMetaData(["customer" => $customer]);
}

// More restrictive for spammers
if (rate("outbound-spam"$senderip086400) >= 100 and rate("outbound-minute"$senderip060) > 10)
    
Defer("You may only send 10 email/minute if you've sent more than 100 spam the last day");
if (
rate("outbound-spam"$senderip086400) >= 100 and rate("outbound-hour"$senderip03600) > 100)
    
Defer("You may only send 100 email/hour if you've sent more than 100 spam the last day");

// Hard rate limit
if (rate("outbound-minute"$senderip10060) == false)
    
Defer("You may only send 100 email/minute, try later");
if (
rate("outbound-hour"$senderip10003600) == false)
    
Defer("You may only send 1000 email/hour, try later");

// Customer ID rate limits
if ($customer) {
    if (
rate("customer-outbound-minute"$customer50060) == false)
        
Defer("Customer may only send 500 email/minute, try later");
    if (
rate("customer-outbound-hour"$customer50003600) == false)
        
Defer("Customer may only send 5000 email/hour, try later");
}

// Store a few spam samples
if (ScanRPD() == 100 or ScanSA() > 8)
    if (
rate("spam-samples"$senderip10086400))
        
Quarantine("mailquarantine:1", ["done" => false"reject" => false]);

// Defer high volumes of suspect spam
if ((ScanRPD() == 50 or ScanSA() > 4) and rate("outbound-bulk"$senderip60028800) == false)
    
Defer("You are only allowed to send 300 bulk messages per 8 hours, try later $messageid");
if ((
ScanRPD() == 100 or ScanSA() > 6) and rate("outbound-spam"$senderip20028800) == false)
    
Defer("You are only allowed to send 100 spam messages per 8 hours, try later $messageid");

// Reject viruses
if (ScanRPDAV()) Reject("Rejected by virus filter ($messageid)");
if (
ScanKAV()) Reject("Rejected by virus filter ($messageid)");
if (
ScanCLAM()) Reject("Rejected by virus filter ($messageid)");

// Reject executable files (need DLP rule)
if (in_array("EXE"ScanDLP(["EXE"], ["stop_on_match" => true"timeout" => 10"recursion_limit" => 0])))
    
Reject(["File types which may contain executable code are not allowed.",
        
"Please find another way to deliver the file."]);

Delete(); // Equivalent of Deliver(), becase we're only operating at a decision engine for a proxy

function Reject($msg) {
    
sendlog("REJECT"$msg);
    
stat("actions", ["defer" => 0"reject" => 1"deliver" => 0]);
    
builtin Reject($msg);
}
function 
Defer($msg) {
    
sendlog("DEFER"$msg);
    
stat("actions", ["defer" => 1"reject" => 0"deliver" => 0]);
    
builtin Defer($msg);
}
function 
Delete() {
    
stat("actions", ["defer" => 0"reject" => 0"deliver" => 1]);
    
sendlog("DELIVER""");
    
builtin Delete();

Logging server

For easier administration and supervision, we recommend using the logging server, which includes a customisable rate limit page overview page.

Router configuration

For a fullt transparent setup to work, the firewall or routers which the outbound traffic is routed through needs to re-route outbound packets with a destination port of 25 (and the returning traffic) to the proxy servers. The exact configuration of course varies depending on the router brand; here are two links that apply for Juniper Networks: