Logging

From Halon, SMTP software for hosting providers
Jump to: navigation, search

The Halon email gateway has extended logging and debugging capabilities. All events (system, mail, rpc) are logged to multiple facilities in the web administration, email tracking is available, as well as optional syslog and external end-user database logging.

Custom logging

The system provides a very high level of customisation thanks for the HSL scripting language. Just like you can create custom reporting with stat() and rate(), you can write whatever you like to the log with the syslog() function. You can also use the http() and json_encode() functions to log data externally, like we do in the end-user logging server. Finally, the built-in, database-backed email tracking/queue/archive can be extended with custom meta-data, using the SetMetaData() function.

Text logs

Logs may be found in various places in the web administration, but since the appliance itself contains limited space for logging and history, it's highly recommended to use syslog and external end-user history for long term storage of logs (for debugging, troubleshooting and accountability).

System logs

System logs are log files that doesn't directly help in message debugging, rather in system debugging and health monitoring. They are found on the System → File viewer page.

Log Description
System log Primary system log
Startup Startup log
Critical log Critical system events (also in system log)
Cluster Clustering daemon log (also in system log)
Update Software update log (also in system log)
RPC Authentication log (eg. backend, login, SSH, HTTP)
Sophos Sophos anti-virus update log
ClamAV ClamAV anti-virus update log
SpamAssassin sa-update update log

Mail logs

Mail logs will tell you what happened to a message in transit and where it ended up (deliver, reject, quarantine, queued, etc). It's found on the Activity → Text log page. It support free-text searching as well as advanced regular expressions based on time.

Syntax Example Description
127.0.0.1 127.0.0.1 Searches using a plain free text search
messageid 43de929d-cc22-11dd-90ef-0048546ae42b Searches for a message (and shows full transaction)
/<regexp>/ /127\.0\.0\.1/ Searches using a regular expression

Log rotation

Mail logs are rotated by size, and not by volume/messages, as defined by mail_log_size. There are two log files, one mail.log.old which is at least max MB and a mail.log which can be anywhere from 0 to max MB. That makes it very hard to predict the amount of messages or time range that will be stored in the logs at a specific time. So given the nature of this problem, the best way to answer the question "how far back in time will my logs go?" can only be answered by running the unit for a while, and on regular occasions do a test search on Activity → Text log and see the timestamp printed by searchlog:

Jan 16 14:49:34 (info) searchlog: Log file rotated

If you take logging seriously, use external logging to store logs permanently and with a predictable retention policy.

Email tracking

The Activity → Mail tracking page with its search filter support should be considered the main tool for finding what happened to a message, given that mail_internal_history is enabled. Its database-backed, using the mailQueue() and mailHistory() SOAP API calls. It also contains a link to the text log for each message, spawned over all mail processes (smtpd, mailpolicyd, mailscand, mailqueued, cleanupd, etc).

In case mail_internal_history is disabled for performance reasons, the same functionality can be found in the external end-user logging server.

External logging

External, consolidated tracking database

The end-user project contains a tracking database, with support for search filters. It can be used to provide message tracking over an extended period of time, especially if local mail_internal_history is disabled for performance reasons.

Syslog

Syslog is one of the most useful tools for debugging and monitoring the system over an extended period of time, next to the end-user log. By using a external syslog server, you can have almost unlimited logging traceability. Enabling syslog is as easy as inputing the following information under Configuration → Syslog destinations → Add destination.