Mail Gateway Performance

From Halon Security

It is possible for an SPG/VSP to handle incredible amounts of e-mail traffic. For example, the reference system^[1] can scan more than 200 messages per second on Content Flow (mailscand) level, and many more on connection (ippolicy) level, given our test case^[2]. In short, the theory of achieving such performance is to disable the appliance's history and statistics functionality (rely solely on remote syslog) and reject as much spam as possible in early stages.

Configuration changes to increase throughput

1. Configure 3 GB RAM
   If you are using VSP (a VMware appliance), increase the RAM to 2-3 GB. It is however important that the VMware host does not swap. Upon startup, the H/OS operating system determines the number of simultaneous threads. For example, at 2 GB, there are 160 concurrent message threads (mailpolicyd, mailscand, antvird, mailqueued, etc) available. This value can be viewed in the Web Administration on the Reporting section's tab Statistics.
2. Enable GlobalView
   Most mail gateway appliances, includes the VSP, comes with a GlobalView license. For other appliances, it is an add-on license. It provides connection-level outbreak-based zombie/botnet protection, effectively blocking 80-97% of the spam before the messages are received. The default configuration contains a GlobalView-enabled IP Policy flow, simply called "GlobalView (aclflow:1). Select that flow as "IP Policy Flow" for all your incoming SMTP listeners on the Domains section. In that way, the IP Policy (ippolicyd) reduces the load of the system, by handling more than 80% of the messages without requiring any significant amount of CPU power. Enable temporary blocking within the GlobalView block, for even better performance.
3. Disable quarantine blacklists
   If you do not use the quarantine, disable the quarantine blacklist lookups. In the default configuration. there are two of them. Go to Flows, select the Recipient tab, select the flow that you use, press the blacklist icon, and uncheck "Use User Blacklist". Also go to the Content tab, select the flow that you use, and remove any Black/Whitelist modules (and ScanBWList() function calls). Removing the black/whitelist lookups can increase performance by 20 times.
4. Run anti-virus and pattern analysis in the very end
   Some modules are faster than others. For example, the outbreak module (the ScanRPD() function) is many times faster than the pattern analysis module (the ScanSA() function). Therefore, run final action module such as reject (the Reject() function) if for example ScanRPD() returns 100, in order not to run ScanSA() for all the messages that ScanRPD() detect.
5. Disable history and statistics
   If you can rely solely on an external Syslog server for all your logging, statistics and mail message history needs; disable the internal statistics by going to Mail Gateway, select the Settings tab, and change "Enable History and Statistics" to "No". Instead of browsing the Activity section's History tab to look for messages, search your syslog log files. It can be a good idea to create a Content Flow script that outputs additional information to the syslog server. You don't need to worry about logging (the echo statement) for virus detections, as antivird output any virus detections to the log automatically. The code could look like:

$subject = GetHeader("Subject");
$rpd = ScanRPD();
if ($rpd == 100) {
  echo "REJECT from: $sender to: $recipient subject: $subject rpd: $rpd";
  Reject("Message was detected as spam (RPD)");
  }
$sa = ScanSA();
if ($rpd == 50 and $sa > 5) {
  echo "REJECT from: $sender to: $recipient subject: $subject rpd: $rpd sa: $sa";
  Reject("Message was detected as spam (RPD and SA)");
}
if ($sa > 18) {
  echo "REJECT from: $sender to: $recipient subject: $subject rpd: $rpd sa: $sa";
  Reject("Message was detected as spam (SA)");
}

Notes

^1. VMware ESXi 4.0.0 on one CPU core at 2 GHz with 2 GB RAM on Sun Fire X2250
^2. Typical mail traffic, between 1-50 KB in size each