Installing Sender History Database and Configuring Delay Queue

To install and configure the Sender History Database role and configure the Delay Queue, see the instructions below:

Installing the Sender History Database role

  1. Use the PMX installer. It will ask for permission to update itself. Select Yes.
  2. Select Custom Installation.
  3. Select Sender History Database.
  4. Specify the email address for notification messages.
  5. Specify the mail server to send notifications to.
  6. Specify the directory in which the sender history database file should be stored.
  7. Specify the internal host IP addresses and subnets, if any.
  8. Proceed with the installation.
Note

If you have Full installations and want to upgrade to 6.4.0:

  • Upgrade to 6.4.0 (new packages for existing roles will be upgraded)

  • Run the installer, go to Add Additional Components, select Sender History Database Role and install.

Configuration of the Sender History Database role

The Sender History Database is maintained using Redis (https://redis.io/). You need to configure: <pmx_directory>/etc/redis.conf. An explanation of the parameters is given below. Check the values of: bind, port, dir and dbfilename.

Basic configuration: Network

  1. Specify the network interface on which Redis should listen for connections.
    Important The default is set to listen on all available network interfaces. This configuration is not secure if the system is exposed to the Internet. If you choose to listen on all interfaces (we DO NOT recommend this configuration if the system is exposed to the internet), set protected-mode to no.

    The default port is 6379.

    For example:
    bind 192.168.56.29
          port 6379 
  2. Specify the working directory location. The database will be saved here.
    For example:
     dir /opt/pmx6/var/redis
  3. Specify the name of the database file (this file will be created in the directory specified above).
    For example:
    dbfilename edge.domain-c.vbox_shdb.rdb

Advanced configuration: (this affects performance, so only fine tune this configuration if required)

  1. Specify backup frequency (when to save the database on disk). Use the following options:
    save every 7 minutes, irrespective of keys changed:
    save 420 1
    save every 4 minutes, if (at least) 500 keys changed:
    save 240 500
    save every minute, if (at least) 1300 keys changed:
    save 60 1300
  2. Specify the maximum number of simultaneous client connections allowed:
    maxclients 15000

    You will need to verify whether your Linux system (particularly if you are using CentOS) is configured to allow the number of open files:

    grep 'nofile' /etc/security/limits.conf; cat /proc/`pidof redis-server`/limits|grep "Max open files.

  3. Do not use the AOF method of saving records:
    appendonly no
  4. Verify whether the redis service can be reached through the edge server. You may need to adjust the settings on your firewall or router.
  5. Make sure the resources system is initialized on the CSM role, as this is needed for the delay queue to function properly:
    perldoc pmx-resources-init

Optional: Using CSM to share three configuration files: redis_location.conf, delay.conf and policy.siv

These three files are necessary for the delay queue. You can either share them with all edge servers using CSM or you can manage them individually on each edge server.

  1. redis_location.conf

    Each edge server needs to know how the redis server can be contacted in order to set and get sender history database entries. The file redis_location.conf should be present on the edge server at the following location: <pmx_directory>/etc/redis_location.conf. Syntax of the file is:

    server_ip = 10.200.193.229
                 port = 6379
    Note Communication between the milter and historian DB role will be in plain text. You need to restart the milter service if there is any change made to this file.
  2. delay.conf

    This file contains parameters that affect the decisions about delaying messages.

    Note Communication between the milter and historian DB role will be in plain text. You need to restart the milter service if there is a change in this file.

    This file should be present on each edge server at the location: <pmx_directory>/etc/delay.conf

    delay_status = Collect
    delay_min_minutes = 10
    delay_max_minutes = 60
    delay_max_size = 1024
    send_delayed_mail_to = 127.0.0.1:10025
    delay_status

    Possible values: Collect, On, and Off

    Default: Collect

    Example:
    delay_status = Collect

    Collect: Do not delay mails based on sender history, instead build the sender history database.

    On: Enable snow shoe spam protection

    Off: Disable snow shoe spam protection.

    delay_min_minutes
    Minimum time (in minutes) a message can be delayed.

    Default: 2

    Example:
    delay_min_minutes = 10

    Possible values: The value must not be less than 2 and it must not be greater than or equal to delay_max_minutes.

    delay_max_minutes
    Maximum time (in minutes) a message can be delayed.

    Default: 60

    Example:
    delay_max_minutes = 60

    Possible values: The value must not be greater than 60 and it must not be less than or equal to delay_min_minutes.

    delay_max_size
    Delay queue disk size limit (MB).

    Default: 1024

    Example:
    delay_max_size = 1024

    Possible values: The value must not be less than 256 and it must not be greater than 1024.

    send_delayed_mail_to

    It describes the destination of delayed message when it is time to re-inject the message into the milter.

    For PMX supplied Postfix MTA, refer to the port parameter in the milter section of 'Policy' in the file pmx.conf.

    Example: send_delayed_mail_to = 127.0.0.1:10025

    For PMX supplied Sendmail MTA, use the IP address and port that Sendmail is receiving mails on.

    Example: send_delayed_mail_to = 192.168.1.1:25

    We recommend you use the IP and port so that the mail is reinjected into the same edge server that has processed (and delayed) it before.

    Note Make sure that relay is accepting the mails (mails that are getting re-injected). If not, mails (that will fill the disk up to the value of delay_max_size) will stack in the delay queue directory until the size of the directory goes beyond the delay_max_size configuration parameter.
  3. policy.siv

    A new test pmx_delayed_mail and a new action pmx_suspect_delay have been added to be used in policy.siv.

    The test pmx_delayed_mail returns true if the message being processed was delayed earlier and is getting reinjected into the milter. Use this test to prevent the actions and test, which are to be applied only to the message when it first arrived, from being applied to the delayed message. For more details, click policy.siv
    Note It is recommended to use command pmx-policy to edit the policy.siv file to search for syntax error. If you are using any other method to edit the policy (example: a text editor), use command pmx-policy to check the correctness of the policy.

    The action pmx_suspect_delay delays the message for a period specified by the antispam engine. The message will be delayed only if the delay_status is 'On' in <pmx_directory>/etc/delay.conf file. This action should be used in conjuction with the test pmx_spam_hit like following:

    if pmx_spam_hit :comparator "i;ascii-casemap" :matches ["DQ_SUSP_?"]
                     {
                         pmx_suspect_delay;
                      }
    Note Do not use stop action. It is implicit, if the message is delayed, message processing will stop internally.

    A sample policy script is given that will help learning how this new action and test should be used.

Upgrading the edge servers

Upgrade the edge servers.