End-user

From Halon, SMTP software for hosting providers
Jump to: navigation, search
Watch our video guide on YouTube or try out the demo system

There are multiple ways of giving users access to the system. Administrators and read-only support personnel can use the standard, built-in web administration interface. End-users however, probably need something simpler, while still being able to control their own messages in the system in a controlled manner. For that, we have an open source end-user web interface on GitHub. It can be found on https://github.com/halon/sp-enduser and used either as it is, modified to fit your needs, or only as an inspiration for your own code.

Installation

Our end-user interface is a PHP web interface, that you can run on any web server of your choice:

  1. Acquire a web server with PHP (either your own server, or through a web hosting company)
  2. Copy the files to a folder on that server; if you have shell access, run the git or svn commands detailed below
  3. Edit the configuration file (settings-default.php) and when you're done rename it to settings.php
  4. Surf to the new website and follow the instructions that are shown
    • Remove install.php when you're told to

If you encounter any issues during the installation or configuration of the end-user take a look at the common errors section or contact support@.

Additional platform support

In addition to running on a regular PHP installation, we also support;

  1. Google App Engine
  2. cPanel

Version control

The best way to download the files, because of the ease of updating when making local changes, is with git or svn. To do this, run either

# git clone git://github.com/halon/sp-enduser.git

or

# svn checkout https://github.com/halon/sp-enduser/trunk sp-enduser/

svn also support the svn export command if you don't want to have your working directory in a exported web folder.

Updating

You can update the end-user to the latest version at any time by running, in that folder,

# git pull

or

# svn up ; rm install.php

Configuration

The end-user interface is configured by editing its settings.php file and some of the settings are described below.

Authentication methods

The interface supports many authentication methods, such as statically configured users, LDAP sources, using SMTP AUTH (SASL) lookups, an internal database of your choice (with users possibly created from your anti-spam system using the trigger URL API), or any other method that you can add yourself.

SMTP lookup

If the usernames are e-mail addresses, SMTP authentication (SASL) can be used as an authentication source. Add to the settings file:

$settings['authentication'][] = array('type' => 'smtp''host' => '10.2.0.30''port' => 25); 

LDAP (msexchange)

To authenticate an e-mail address (userPrincipalName) and password against Exchange/Active Directory, add this code block to the settings file:

$settings['authentication'][] = array(
         
'type' => 'ldap',
         
'uri' => 'ldap://10.2.0.30',
         
'base_dn' => 'DC=example,DC=local',
         
'schema' => 'msexchange',
         
'options' => array(LDAP_OPT_PROTOCOL_VERSION => 3),

         
// optional, bind username/password to search the LDAP source
         
'bind_dn' => 'CN=ldapuser,OU=ServiceAccounts,DC=example,DC=local',
         
'bind_password' => 'changeme',

         
// optional, specify which group user has to be a member of
         
'memberof' => 'CN=Enduser,OU=Groups,DC=example,DC=local'
         ); 

Black/whitelist

First of all, the black/whitelist requires that you configure a database backend. The most simple one is SQLite, which can be enabled by adding

$settings['database']['dsn'] = 'sqlite:/var/db/foo.db'

to the settings file, and making sure that the specified file is read- and writable by PHP. Then, configure an API key in settings.php

$settings['api-key'] = 'badsecret'

And add the ScanBWList() function to your DATA flow (and use it).

Quarantine digest messages

To begin sending quarantine digest message start by enabling this feature in the settings.php file. You also have to make sure that the mail settings are configured correctly in your php.ini file and that Sendmail/Exim is installed and working if you're running the end-user on a Linux server.

When you have done this you can edit the crontab file on your Linux server, that runs once every 24 hours, by typing crontab -e in the terminal and add the following line at the bottom:

0 0 * * * /usr/bin/php /var/www/html/sp-enduser/cron.php.txt digestday

where the path should be the directory that you installed the end-user to.

Halon node integration

SOAP authentication

It's possible to restrict the access of the configured API user to a limited set of SOAP calls. When you run the install script on the End-user server you will be presented with an example of such a configuration that you can add to the API script under the Configuration > Email engine > Code editor (IDE) page on your Halon node. If you are missing the API script you will have to create a new script and choose API as type.

Automatic user creation

If you want users to be automatically created when a message is received, add the following script to your data flow.

$triggerurl "http://end-user-url/api.php?api-key=badsecret";

function 
Quarantine(...$args)
{
    global 
$triggerurl$recipient;
    
$r = isset($args[1]["recipient"]) ? $args[1]["recipient"] : $recipient;
    
http($triggerurl "&type=trigger&recipient=$1", ["timeout" => 10"background" => true"background_retry_count" => 1"ssl_default_ca" => true], [$r]);
    return 
builtin Quarantine(...$args);

Caching of blacklist and whitelist

Used to lookup against the blacklist and whitelist on the Enduser (with a cache). The function should be called before spam checks.

$triggerurl "http://end-user-url/api.php?api-key=badsecret";

function 
ScanBWList()
{
    global 
$triggerurl$senderip$senderdomain$sender$recipientdomain$recipient;
    
$data cache [
                
"ttl_function" => API_ttl,
                
"update_function" => API_update,
                
"namespace" => "ScanBWList",
            ]
            
http($triggerurl "&type=bwlist", ["timeout" => 10"ssl_default_ca" => true]);
    
$list json_decode($data);
    if (!
is_array($list))
        return 
50;
    
$blacklist false;
    foreach (
$list as $item) {
        if (
$item["access"] == "" or $item["access"] == $recipientdomain or $item["access"] == $recipient) {
            if (
$item["value"] == $senderip or $item["value"] == $senderdomain or $item["value"] == $sender) {
                if (
$item["type"] == "whitelist")
                    return 
0;
                if (
$item["type"] == "blacklist")
                    
$blacklist true;
            }
            if (
$item["value"][0] == "." and is_subdomain($senderdomain$item["value"])) {
                if (
$item["type"] == "whitelist")
                    return 
0;
                if (
$item["type"] == "blacklist")
                    
$blacklist true;
            }
        }
    }  
    if (
$blacklist)
        return 
100;
    return 
50;
}

function 
API_ttl($new)
{
    if (
is_array(json_decode($new)))
        return 
300;
    return 
60;
}

function 
API_update($old$new)
{
    if (
is_array(json_decode($new)))
        return 
$new;
    return 
$old;

ScanBWList with per-user check

This implementation is suboptimal as it queries each $sender and $recipient for the black/white list. It's not necessary to cache these as cache hits would be close to none.

$triggerurl "http://end-user-url/api.php?api-key=badsecret";

function 
ScanBWList()
{
    global 
$triggerurl$senderip$sender$recipient;
    
$data http($triggerurl "&type=bwcheck&senderip=$1&sender=$2&recipient=$3",
            [
"timeout" => 10"ssl_default_ca" => true], 
            [
$senderip$sender$recipient]);
    if (
$data == "whitelist")
        return 
0;
    if (
$data == "blacklist")
        return 
100;
    return 
50;

Caching of spam settings

If you want to fetch the spam settings from the End-user interface.

$triggerurl "http://end-user-url/api.php?api-key=badsecret";

function 
SpamSettings()
{
    global 
$triggerurl$recipientdomain$recipient;
    
$data cache [
                
"ttl_function" => API_ttl,
                
"update_function" => API_update,
                
"namespace" => "SpamSettings",
            ]
            
http($triggerurl "&type=spamsettings", ["timeout" => 10"ssl_default_ca" => true]);
    
$list json_decode($data);
    if (!
is_array($list))
        return 
0;
    foreach (
$list as $item) {
        if (
$item["access"] == $recipient)
            return 
$item["settings"];
        if (
$item["access"] == $recipientdomain)
            
$domain $item["settings"];
        if (
$item["access"] == "")
            
$everyone $item["settings"];
    }
    if (isset(
$domain)) return $domain;
    if (isset(
$everyone)) return $everyone;
    return [
"level" => "medium"];
}

function 
API_ttl($new)
{
    if (
is_array(json_decode($new)))
        return 
300;
    return 
60;
}

function 
API_update($old$new)
{
    if (
is_array(json_decode($new)))
        return 
$new;
    return 
$old;

Caching of Datastore settings

If you want to fetch the datastore settings from the End-user interface.

function Datastore($query)
{
    global 
$triggerurl;
    
$data cache [
                
"ttl_function" => API_ttl,
                
"update_function" => API_update,
                
"namespace" => "Datastore",
            ]
            
http($triggerurl "&type=datastore&$query", ["timeout" => 10"ssl_verify_peer" => false]);
    
$list json_decode($data);
    if (!
is_array($list))
        return -
1;
    return 
$list;
}

function 
API_ttl($new)
{
    if (
is_array(json_decode($new)))
        return 
300;
    return 
60;
}

function 
API_update($old$new)
{
    if (
is_array(json_decode($new)))
        return 
$new;
    return 
$old;

History log

There's a limit to how many messages can stored in the email gateway (SP) nodes' database, for performance reasons. In order to store large volumes of email history we encourage the use of the end user interface's built-in history log feature. Note that this feature currently requires that you have configured the End-user to use a MySQL/MariaDB database. Simply append the following script to the email gateway nodes' DATA flow, to push logging information to the End-user:

$triggerurl "http://end-user-url/api.php?api-key=badsecret";

$logdata = [
    
"owner" => $recipient,
    
"serialno" => serial(),
    
"msgid" => $messageid,
    
"msgactionid" => $actionid,
    
"msglistener" => $serverid,
    
"msgtransport" => $transportid,
    
"msgsasl" => $saslusername,
    
"msgfromserver" => $senderip,
    
"msgfrom" => $sender,
    
"msgto" => $recipient,
    
"msgsubject" => GetHeader("subject"),
    
"msgsize" => MIME("0")->getSize()
];
function 
sendlog($action$desc) {
    global 
$triggerurl$logdata$messageid;
    
$logdata += [
        
"msgaction" => $action,
        
"msgdescription" => $desc,
        
"msgts" => timelocal()
    ];
    
http($triggerurl "&type=log",
        [
"timeout" => 10"background" => true"background_hash" => hash($messageid), "background_retry_count" => 1"ssl_default_ca" => true], [], $logdata);
}
function 
Reject($msg) {
    
sendlog("REJECT"$msg);
    
builtin Reject($msg);
}
function 
Deliver(...$args) {
    
sendlog("QUEUE""");
    
builtin Deliver(...$args);
}
function 
Defer($msg) {
    
sendlog("DEFER"$msg);
    
builtin Defer($msg);
}
function 
ScanRPD($arg = []) {
    global 
$logdata;
    
$logdata += [ "score_rpd" => builtin ScanRPD(), "score_rpd_refid" => builtin ScanRPD([ "refid" => true ]) ];
    return 
builtin ScanRPD($arg);
}
function 
ScanSA() {
    global 
$logdata;
    
$logdata += [ "score_sa" => builtin ScanSA(), "score_sa_rules" => builtin ScanSA([ "rules" => true ]) ];
    return 
builtin ScanSA();
}
function 
ScanKAV() {
    global 
$logdata;
    
$logdata += [ "score_kav" => builtin ScanKAV() ? : "" ];
    return 
builtin ScanKAV();
}
function 
ScanCLAM() {
    global 
$logdata;
    
$logdata += [ "score_clam" => builtin ScanCLAM() ? : "" ];
    return 
builtin ScanCLAM();
}
function 
ScanRPDAV() {
    global 
$logdata;
    
$logdata += [ "score_rpdav" => builtin ScanRPDAV() ];
    return 
builtin ScanRPDAV();

The code above could be placed in a virtual text file, and included in the top of the script. Please note that the "owner" field should probably be changed to $sender instead of $recipient for outbound traffic. For delivery status updates, the following script should be called from (again, preferably by including code from a virtual text file) in the Post-delivery flow.

$triggerurl "http://end-user-url/api.php?api-key=badsecret";

$logdata = [
    
"serialno" => serial(),
    
"msgid" => $messageid,
    
"msgactionid" => $actionid,
    
"msgdescription" => $errormsg,
    
"msgaction" => "ERROR"
];
if (
$errorcode >= 400 and $errorcode 500 and $retry $retries$logdata["msgaction"] = "QUEUE";
if (
$errorcode == 250$logdata["msgaction"] = "DELIVER";
if (
$retry 0$logdata["msgdescription"] .= " (retry $retry/$retries)";
http($triggerurl "&type=logupdate",
    [
"timeout" => 10"background" => true"background_hash" => hash($messageid), "background_retry_count" => 1"ssl_default_ca" => true], [], $logdata); 

Removing old logs

When using the history log feature you should also edit the crontab file to periodically remove old logs from the database. You do that by typing crontab -e in the terminal and add the following line at the bottom:

0 * * * * /usr/bin/php /var/www/html/sp-enduser/cron.php.txt cleanmessagelog

where the path should be the directory that you installed the end-user to.

Database stats

When using the database stats feature you should also edit the crontab file to periodically generate RRD graphs. You do that by typing crontab -e in the terminal and add the following line at the bottom:

* * * * * /usr/bin/php /var/www/html/sp-enduser/cron.php.txt rrd

where the path should be the directory that you installed the end-user to.

Common errors

Missing cURL extension

If the following warning message appear when installing the End-user

 WARNING: cURL extension is missing. Without it, the UI will run slower (not being able to run searches in parallel).

it means that the cURL extension for PHP is missing or that it has not been enabled in the php.ini file. You can install it on a Debian/Ubuntu based server by running the following command

 apt-get install php5-curl

No database

While the End-user can function without a database, some features will not work such as the black/whitelist and therefore we warn you about this during the installation

 WARNING: No database. Database users and black/whitelist will not be available until created.

You can enable the use of a database by editing the settings.php file.

Database error

If you encounter the following error message during the installation

 Database error: could not find driver

It means that you have configured the use of a database in the settings.php file but either have not installed the required PHP extension or it has not been enabled in the php.ini file. To install these extensions on a Debian/Ubuntu based server you can run

 apt-get install php5-mysql

or

 apt-get install php5-sqlite

Missing LDAP module

If you encounter the following error message when trying to login

 PHP module LDAP missing

it means that you have configured a LDAP server as an authentication source in your settings.php file but the ldap module for PHP is missing or it has not been enabled in the php.ini file. You can install it on a Debian/Ubuntu based server by running the following command

 apt-get install php5-ldap

bwcheck failed: unknown

If bwcheck fails with valid responses on the VSP, it's highly possible that the settings.php file has got an BOM (byte order mark) if it was edited using for example notepad. This can be resolved by either saving the file with an editor which doesn't add BOM or enabling zend.multibyte[1] in php.ini.

Wrong sender domain in Envelope/Return-Path

If you're running the End-user on a Linux based distribution, what domain it uses for the sender in the Envelope/Return-Path of email sent from the End-user is usually set either in the /etc/mailname file or set explicitly in the settings for the MTA.