Exim, Virtual Domain Setup

From Ubuntuwiki.net

Jump to: navigation, search

COPIED AND PASTED FROM: http://struction.de/projects/HOWTO_VirtualMail_Exim-MySQL-Spamassassin-ClamAV-Dovecot/

I haven't tried any of this crud yet. This just seemed like the most useful info I've found to date for potentially migrating to a newer/better-supported MTA from Qmail/Vpopmail.

Sad to say, frankly, it looks from here like Qmail/Vpopmail is STILL far, far, FAR better suited to hosting virtual domains well than any of the newer MTAs. SIGH.

--Jimbo

See Also: http://www.debian-administration.org/articles/140

Looks like the best one yet: http://koivi.com/exim4-config/

SMTP Auth... hm :-\ http://www.debian-administration.org/articles/280

more on SMTP Auth: http://typo.submonkey.net/articles/2005/11/22/smtp-auth-with-exim 



This HOWTO is updated frequently and still not complete. E.g. at the moment you'll have to edit user preferences by hand or something like phpMyAdmin. However, it's running for over a year on production systems and is doing well ;-)

Introduction

This HOWTO presents a way for configuring a Virtual Mail System using Exim, MySQL, SpamAssassin, ClamAV and Dovecot. If you find this guide useful or have any ideas / improvements don't hesitate to drop me a mail.

Exim will be the main component in this HOWTO, as it's the mail server. Why exim? Because I like it! Have a look at a MTA Comparison.

A quick overwiew of this projects goals and features. Some assumptions I made about the setup and howto use it. You'll get the whole plot when I'm describing the Idea. All the techniques used will be explained there. The implementation consists of installing and configuring the tools / software components that are connected. Either follow these instructions, or just grab all configfiles and adapt them to your needs. Some additional material is provided at the end.

Best viewed with Clean Style. Update pending...

I'm planning to give this HOWTO an update. hopefully it'll be ready this or next month.

I'll upload new versions of my config files, add some new features, clean up some stuff and - belive it or not - there's some work going on with the web-interface. So just stay tuned in... ChangeLog

2007-09-22 + added feedback section to this site in order to list some successfully used distributions 2007-06-21 + give a hint for adding a new user (SQL), as web-frontend still isn't available 2007-04-22 + small anecdote about broken harvesters used by spammers (see plussed under Aliases) 2007-03-12 + new ClamAV config syntax (since version 0.90) + mod_auth_mysql-patch to do Apache Basic Auth against the user-database + added changelog ;-)

2007-02-27 + added Selective GreyListing support

2007-02-13 + initial version


Successful Feedback

I've received some feedback about successful installations using this guide under a number of Distributions. Perhaps you'll have to tweak a litte bit here and there but it already worked on: 

   * Gentoo Linux
   * Debian Etch (4.0) [adapted HOWTO (introducing additional features)]
   * Ubuntu
         o Ubuntu 8.04 (LTS)
         o Ubuntu 8.10
   * Fedora Core 4
   * FreeBSD
         o FreeBSD 6.2 (AMD64)
         o FreeBSD 7.0
         o FreeBSD 7.1 (AMD64 & i386)
   * Solaris 10 Zone on Sun 420R (Sparc64)

Remember: try to contribute and give feedback. Goals / Features already implemented 

  1. Virtual Users (they only exist in the database, so there's no need to give each user a real account including ssh-access)
  2. All User-data is stored centralized in a Database
  3. All mail (less than 20M) is virus-scanned. Virus-mails are rejected during SMTP-session (before accepting it), so the sender gets to know and we aren't responsible for bounces
  4. All mail (less than 1M) is spam-scored. Mail is tagged in headers only. User may define a personal spam-score-threshold and how to rewrite the subject (if at all). Very-highscore-mails are rejected during SMTP-session (before accepting it), so the sender gets to know and we aren't responsible for bounces
  5. Sending mail to foreign domains is only permitted after successful SMTP-authentification. The Connection has to be encrypted using SSL/TLS
  6. Receiving mail is only possible using SSL/TLS via IMAP/POP3
  7. Aliases (including plussed addresses), Catchall, allusers and conditional email-addresses (see Idea for complete overview!)
  8. Mails are stored in a MailDir /var/mail/DOMAIN/USER for user USER@DOMAIN
  9. Selective GreyListing support for mails that are likely to be spam, but not enough to be rejected
 10. Passwords are stored as base64-encoded SHA1-hashes
 11. Every daemon is running under an own user
 12. Flat Database structure (as we have no frontend yet, this works pretty well with phpMyAdmin)

TODO 

  1. reject archive attachments for files containing executeable double extensions (to reject unknown worms that embedds itself as *.pdf.exe in a zip-file)
  2. Users may change their preferences (spam-score-threshold) and passwords (not yet covered: a web-frontend is in work)
  3. Site-Admins may create / edit users in their domain
  4. limit users in sending rate or message size (this probably won't be implemented as I can't see any need in domineering a user)
  5. ...

Assumptions

The following assumptions are made:

I will not go into details installing the needed software, as it's dependent on your distro. I'll just assume it's all right there. As configuration may vary accross different distros and I'm using Gentoo, you'll have to figure out the difference to your one. However it should not be that hard, perhaps some different config file locations or user names running the services. Keep in mind to adapt them. So far I've got success stories from Debian and FreeBSD users. Please provide feedback!

You are familiar with the used programs or at least with their functionality. This guide does not make sense, if you don't know what e.g. MySQL or an IMAP-Server may be useful for. The reason is that you should be able to adjust the config to your needs. This way you may exchange a component through another piece of software. If you don't want to use ClamAV, but another or additional virus-scanner it's helpful to understand the Exim-config. Or perhaps you don't like the idea of GreyListing and want to get rid of this functionality. So if you understand the way of configuring it you may use this guide as the glue for connecting the components. This way you're able to adapt it better to your needs as they may be different to mine.

As various used Programs need to communicate with each other, I configure them to use UNIX Domain Sockets, whenever possible. Otherwise they are bound to the loopback-interface. This way no unnecessary ports are opened to the public, thus avoiding additional attack-vectors.

Of course this does not affect the SMTP-server or IMAP/POP3-server, as we do want to provide these services to the public. ;-)

Apart from that it is possible to deploy each tool onto another hardware, for better scalability. They all are able to communicate via IP.

I'm not using dovecots new SASL-Authenticator to implement SMTP-Auth in exim, instead both of them authenticate against MySQL on their own.

I'm not using dovecots delivery feature (nor any other maildropper) to store incomming mails in MailDir, instead exim stores them directly there. If you experience problems with dovecot getting slow on building an index on access, you may take the benefit of using dovecots deliver, which updates the index on every new incomming mail. That depends on your hardware, but this average server will take just about a second in folders with 16k+ mails for a full index-recreation, so in general you shouldn't even notice... Tools / Software used

The following software is used in this HOWTO, so I suggest installing it via the packet-management of your favorite distro. The exact versions I used are noted, but you should be fine with the features the major-version-number gives you. 

   * Exim (4.62) - Powerful and flexible MTA / SMTP server
   * MySQL (5.0.26) - Database that stores user-information and their preferences
   * SpamAssassin (3.1.8) - well-known spam-scoring-engine
   * ClamAV (0.88.7) - free signature-based virus-scanner
   * Dovecot (1.0_beta15) - small but powerful and secure IMAP and POP3 server

Idea

The idea of the whole virtual mail system is to reduce administrative overhead. All settings are stored in a MySQL database and all services such as SMTP and IMAP should use them. Concepts Users

A user is identified by his email-address (served by the system) and his password. You can consider it as an account. For each domain the system should serve, a user has to exist. (Usually the Domain-Admin.)

Users are able to send mail using SMTP-AUTH and read mail using IMAP or POP3. These privileges may be revoked.

There are several settings a user may set. please see SPAM for a detailed description.

In all further examples let's say user@struction.de is a user. Aliases

Aliases are email-addresses that are redirected to another address (local or remote, even aliases again). E.g. alias@struction.de may redirect all mails to user@struction.de.

An alias may redirect a mail to multiple recipients. E.g. office@struction.de may redirect an incomming mail to user@struction.de and anotheruser@example.com.

By default every user gets some aliases defined automatically according to the "plussed addressing scheme". Mail to user+string@struction.de gets redirected to the user's mailbox, where string can be anything valid in an email-local-part. string gets introduced by the +-sign. Multiple +-signs are valid.

This can be used to publish different mail-addresses to different people in order to track who sold your address to whom. E.g. you give the address user+company.A@struction.de to company A and one day you receive a mail to that alias-address from company B, you'll notice where they got the address from.

By the way I noticed that most harvesters used to gather email-addresses from web-pages seem to choke on these plussed addresses. I've setup a page with fake addresses that is only visible to crawlers/harvesters and was disappointed that this spamtrap didn't received a mail for months (half a year as of 2007-04-22). On examining the mail-server logs I discovered a lot of rejected mails due to non-existent local email-addresses. Taking a closer look on these failed addresses I found out that most of them where cropped after the +-sign. E.g. from this page harvesters may find user+onlyforspam@struction.de, but are adding onlyforspam@struction.de to their database. Obviously these broken harvesters can be tricked by using such a plussed address where the wrongly cropped address is invalid and rejected by the mail-server. Conditionals

Taking the last mentioned method to the next level, you can publish alias-addresses that are valid only if a condition that is embeded in the address matches. Conditionals are introduced by the #-sign and take a parameter that's seperated from the condition by another #-sign. The general syntax for a conditional email-address is user#condition#parameter@struction.de for user. Remember: they do not need to be setup by the user. Just use them!

For now there a two conditions implemented: before

An email-address like user#before#YYYYMMDD@struction.de is valid and the mail server accepts it, if the current date is not higher than YYYYMMDD, which is a day encoded like 20070308 and the user exists.

This way you can publish a temporary valid address e.g. for an one-time online-game. Or you may embed such an address on your webpage that's dynamically generated to be valid one week, like user#before#20090102@struction.de, which is generated with the following PHP-code: <?="user#before#" . date("Ymd", ( time() + (7 * 24 * 60 * 60) )) . "@struction.de"?> People may contact you, but crawlers harvesting email-addresses won't be able to spam you after the defined time. fromdomain

An email-address like user#fromdomain#domain@struction.de is valid and the mail server accepts it, if the domain of the sender's address matches domain and the user exists.

E.g. you can publish user#fromdomain#company-A.com@struction.de for the newsletter of company A which is expected to send news from domain company-A.com. If they trade this address to some other company, mail from them will get rejected. CatchAll

This one is quite simple. If a mail could not be accepted because there's no user like doesnotexist@struction.de and you've defined a CatchAll for the domain struction.de, incomming mail gets redirected like an alias to another address.

Somebody requested this feature. In my opinion it just increases spam in your inbox, but on the other hand if you've plenty of unused domains... AllUsers

For all hosted domains there's an email-address named alle@domain, that redirect incomming mail to all users of domain. The only limitation is that it's only for internal use, which means to send a mail to this address you have to be authenticated using SMTP-AUTH. Maintaining the System

There's a fully buzzword-compliant web-interface in progress. Expect it to be released soon. This section is work-in-progress, too. But it's not that hard to figure out how to create users using e.g. phpMyAdmin, as all tables and columns are documented as comments in their database-structure. adding a user

Just create a line in the user-table. The password-hash can be generated with

dovecotpw -s SHA1


Make sure, you chop the prefix {SHA1} from the output hash. The database-structure itself should be self-documenting. If you're unsure, check out the table-comments. The following SQL should create a new user test@example.com, using default values for optional fields:

INSERT INTO `user` (`username`, `domain`, `password`, `Full Name`) VALUES ('test', 'example.com', 'W6ph5Mm5Pz8GgiULbPgzG37mj9g=', 'test user entry for demo')


There are more ways to create the password hash. Just have a look at the Addon Section. adding an alias adding a catchall SPAM & VIRUS

There are several techniques implemented to catch spam & virii. All of them have their advantages and disadvantages, so our goal is to mix them up at a good ratio. Tagging & Rejecting at SMTP-time vs. User-Thresholds

It's important to handle such incomming mail in a transparent way. No mail should get lost without a trace, so we cannot accept it in the first place and silently drop it later. In this situation the sender doesn't know that his mail never reaches the recipient. As spam & virus mails often use faked sender-addresses, we end up in clogged mail-queues or flooding the wrong person, if we send out notifications.

So the whole decision has to be made while receiving the mail. On the one hand this is quite ressource-intensive, but on the other hand we can reject the mail and the sending mail-server recognizes that and is responsible for notifying the sender. So our mail-queue keeps clean and we don't bother the wrong person.

But now there's another inconvenience. As we want different users to have different spam-thresholds we cannot reject spam because an incomming mail may have multiple users with different thresholds. Aliases need to be resolved and the final recipient may even reside on a foreign mail-server.

Thus we just tag the spammy mail with the score in a header or reject it, if its score is far above the maximum threshold in all user preferences. There are ways to limit the number of recipients per mail, but I dont't like the idea because of broken mail-servers. Later at delivery-time, we know the user the mail is locally deliverd to. So we can evaluate the spam-score with his preferences, but it's to late to reject the mail.

So finally it's the users choice to automatically filter out mails with a high spam-score in his client.

For the same reason we only can use a system-wide bayes-database.

I know these are a lot of disadvantages, but for me it's important to have a reliable mail system and I definitely like the advantages of getting rid of most unwanted mail at SMTP-time. If you know a way to solve this problem, please let me know! Selective GreyListing

I don't like GreyListing in general, because it may delay legitimate mail. But as the advantage of rejecting spam- & worm-spreading bot-nets is inviting, I've implemented a simple selective version within Exim.

All incomming mails that scores more than GREYLIST_SPAM_THRESHOLD (in /etc/exim/exim.conf) will be temporary rejected if seen for the first time and on consecutive attempts for at least GREYLIST_TIMEOUT seconds.

This way we catch low scored mail that's likely spam, while not delaying most legitimate mail. Through the delay period we increase the chance that updated Colaborative Checksum Lists, URIBL or DNSBL will score the mail higher in the meantime.

If you have backup MX servers for you domain, or some foreign accounts forwarded, you'll probably won't benefit that much from this method. However - it won't harm you, so give it a try... Rejecting dangerous extensions Aliases & Conditional addresses

See Aliases and Conditionals to learn how to avoid spam by protecting your mail-address. Exim hints to SpamAssassin

Exim adds several headers to an incomming mail that will give SpamAssassin some hints, later (via /etc/spamassassin/99_struction_EXIM.cf). Sender Address Verification

Exim performs a callout verification on the sender address. This tests wheather this address accepts mail, initiating a SMTP-connection to the senders mail-server. Valid and existing addressed will get some negative spam-points, while non-existant ones will score real hard.

Some spammers use the recipents address as the senders address, or an address of the recipients domain, thinking they may be whitelisted. As we only allow our users to send mail using SMTP-AUTH and spammers can't do this, we can reject those mails. Authenticated Sender

Authenticated senders will receive some negative spam-points, as we trust them. However - since they may have a worm / bot installed that uses their authentication credencials, we still scan 'em! Non-RFC-conformant (E)HELO

Most spam uses non-RFC-conformant (E)HELO greetings at the beginning of their SMTP-connection. These failures will score some spam-points later. As there are some MTAs out there, which seem to be broken, it's better not to reject them.

Some spammers try to impersonate our mail-server, thinking it's more trusted or whitelisted. These are rejected, as they can't have our IP ;-). Implementation Installation

Install ClamAV, SpamAssassin, Dovecot and Exim. It's obvious that you'll need MySQL, too - but that may be installed as a dependency. Just make sure you're using at least MySQL version 5, since older versions lacked handling of trailing blanks in strings. (That's annoying if a user configures the system to prepend 'SPAM ' to the subject of a spammy mail and MySQL forgets about that blank.)

Be sure to install/compile your packages with all needed features. In Gentoo it'll be usefull to add support for mysql, ssl, maildir as global USE-flags for all packages.

After installing all services, keep in mind to add them to your startup-scripts as you want to run them after booting ;-) SSL-Certificate / Key

In order to use SSL encrypted connections you need a certificate and key. There are several possibilities to create them. If you need to manage some more you may want to use a tool like TinyCA.

For simplicity I use the same cert/key for both programs: Exim and Dovecot. They should be placed under /etc/exim/exim.crt and /etc/exim/exim.key.

Don't forget to set the file-permissions correctly. (owner=mail / exim.key should be readable only for owner) Exim

Include support for exiscan-acl. Optional lmtp, spf, srs. Exclude gnutls, as ssl will provide the same features.

Daemon should be run as user mail. Dovecot

Optional include POP3 support: pop3d

Daemon should be run as user dovecot. ClamAV

Daemon should be run as user clamav. Make sure it's compiled with gmp-support to verify signature updates. SpamAssassin

This service isn't installed the same way as the others in Gentoo. Somehow they prefer to run it on a per-system-user basis, but actually we only want users to be virtual. That's why this setup is described more detailed.

We are running SpamAssassin in a system-wide config under user spamassassin, so create him and his home-directory /home/spamassassin with appropriate permissions. (bayes database will be stored there):

adduser --home-dir /home/spamassassin --create-home --comment "added for spamd" --user-group spamassassin


Create a directory /var/run/spamassassin owned by the user spamassassin. I prefer all runtime-files for a service like sockets or pid-files in a own directory.

Make sure to start spamd with correct options. This is done in /etc/conf.d/spamd in Gentoo. (Debian e.g. uses /etc/default/spamassassin and a slightly different syntax):

SPAMD_OPTS="--max-children 15 --max-conn-per-child=25 -H -u spamassassin --socketpath=/var/run/spamassassin/spamd.sock --socketowner=spamassassin --socketgroup=spamassassin" PIDFILE="/var/run/spamassassin/spamd.pid"


As all mail is content-scanned during the SMTP-session it's nessessary to react fast. Some seconds scan-time per mail is ok, but when SpamAssassin is expiring bayes tokens during a session that may take long and timeout the connection. Therefore we create a daily cronjob /etc/cron.daily/spamassassin-expire-bayes that expires bayes tokens. Make sure, it's executeable! Configuration

All packages installed? Everything works so far? Then let's get started... MySQL

Install the database scheme from system-scheme.sql:

mysql -p <system-scheme.sql


Create MySQL users for exim and dovecot with read-only permissions on the new database. In addition exim needs INSERT-permission for the greylist-table. Substitute EXIMPASSWORD and DOVECOTPASSWORD with some random secret (e.g. created with 'pwgen --secure 32 2').

GRANT USAGE ON *.* TO 'exim'@'localhost' IDENTIFIED BY 'EXIMPASSWORD' WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0; GRANT SELECT ON `system`.* TO 'exim'@'localhost'; GRANT INSERT ON `system`.`greylist` TO 'exim'@'localhost';

GRANT USAGE ON *.* TO 'dovecot'@'localhost' IDENTIFIED BY 'DOVECOTPASSWORD' WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0; GRANT SELECT ON `system`.* TO 'dovecot'@'localhost';


Dovecot

Create /etc/dovecot/dovecot-sql.conf with following content (and substitute your DOVECOTPASSWORD):

driver = mysql connect = host=/var/run/mysqld/mysqld.sock dbname=system user=dovecot password=DOVECOTPASSWORD default_pass_scheme = SHA1 password_query = SELECT password, '/var/mail/%d/%n' AS userdb_home, 'mail' AS userdb_uid, 'mail' AS userdb_gid FROM user WHERE username = '%n' AND domain = '%d' AND IMAP_allowed = 'YES'


Edit /etc/dovecot/dovecot.conf to suit your needs and be sure it contains:

protocols = imap imaps ssl_cert_file = /etc/exim/exim.crt ssl_key_file = /etc/exim/exim.key disable_plaintext_auth = yes login_user = dovecot verbose_proctitle = yes default_mail_env = maildir:/var/mail/%d/%n

protocol pop3 { # fix broken pop3 logins pop3_uidl_format = %08Xu%08Xv }

auth default { mechanisms = plain passdb sql { args = /etc/dovecot/dovecot-sql.conf } userdb prefetch { } user = nobody }


ClamAV

Because ClamAV needs to access the temporary files created by Exim for scanning, make sure the user running clamd (clamav) is in mail-group (the GID that Exim runs = mail) by adding to /etc/group a line like:

mail:x:12:mail,clamav


Since version 0.90 the configuration syntax has changed. Pick your version: current versions (since 0.90)

Edit /etc/clamd.conf to suit your needs and make it contain:

PidFile /var/run/clamav/clamd.pid TemporaryDirectory /tmp DatabaseDirectory /var/lib/clamav LocalSocket /var/run/clamav/clamd.sock FixStaleSocket yes StreamMaxLength 32M FollowDirectorySymlinks yes FollowFileSymlinks yes User clamav AllowSupplementaryGroups yes


Edit /etc/freshclam.conf to suit your needs and contain:

UpdateLogFile /var/log/clamav/freshclam.log PidFile /var/run/clamav/freshclam.pid DatabaseOwner clamav DatabaseMirror db.xx.clamav.net DatabaseMirror database.clamav.net ScriptedUpdates yes Checks 96 NotifyClamd /etc/clamd.conf


older versions (before 0.90)

Edit /etc/clamd.conf to suit your needs and make it contain:

AllowSupplementaryGroups DetectBrokenExecutables ScanRAR ArchiveLimitMemoryUsage


Edit /etc/freshclam.conf to suit your needs and contain:

NotifyClamd


SpamAssassin

Configure SpamAssassin to your needs, but ensure to have all needed plugins installed (edit *.pre)

Add /etc/spamassassin/99_struction_EXIM.cf to take advantage of spam-hints configured in Exim.

Add /etc/spamassassin/wrongmx.pm and /etc/spamassassin/99_struction_WrongMX.cf to catch spam that is send through a low priority MX, while your server is up.

Add /etc/spamassassin/iXhash.pm and /etc/spamassassin/99_struction_IXHASH.cf to catch spam with fuzzy checksums by iX magazine.

Add /etc/spamassassin/99_struction_DNSRBL.cf to use some additional DNSRBLs.

Make sure your /etc/spamassassin/local.cf contains the following directives:

  1. we run spamassassin through exiscan-acl from exim.
  2. thus, in exim we cannot take modified spamd output as mail.
  3. no message modifications will be visible to exim!
  4. the only way to get debugging output into the mail is a report.

report_safe 0

  1. somehow bayes did not work, so try to specify database path directly

bayes_path /home/spamassassin/.spamassassin/bayes

  1. Enable the Bayes system

use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 bayes_use_hapaxes 1 bayes_expiry_max_db_size 400000

  1. do not automatically expire database, as it causes long scantimes (and timeouts for exim)
  2. make sure the cronjob doing this in an external process is setup!

bayes_auto_expire 0

  1. Enable or disable network checks

skip_rbl_checks 0 use_dcc 1 use_pyzor 1 dcc_path /usr/bin/dccproc pyzor_path /usr/bin/pyzor

  1. as the server is unreliable in these times, increase the timeout

pyzor_timeout 10

  1. new template. Try to keep it under 78 columns (inside the the dots below).
  2. ........................................................................

clear_report_template report SpamAssassin _VERSION_ on host _HOSTNAME_ report scan-date = _DATE_ report score = _SCORE_ report bayes-score = _BAYES_ report bayes-token-summary = _TOKENSUMMARY_ report bayes-token-spam-count = _BAYESTCSPAMMY_ report bayes-token-ham-count = _BAYESTCHAMMY_ report bayes-token-spam = _SPAMMYTOKENS(16,short)_ report bayes-token-ham = _HAMMYTOKENS(16,short)_ report bayes-auto-learned = _AUTOLEARN_ report possible-languages = _LANGUAGES_ report relayed-countries = _RELAYCOUNTRY_ report pyzor = _PYZOR_ report RBL = _RBL_ report ==== ====================== ================================================== report " pts rule name description" report ---- ---------------------- -------------------------------------------------- report _SUMMARY_

  1. ........................................................................


Exim

Use exim.conf.dist as initial /etc/exim/exim.conf and edit it to suit your needs by inserting/modifing the following lines:

Insert MySQL-connection-config with substituted EXIMPASSWORD:

  1. mysql auth

hide mysql_servers = localhost/system/exim/EXIMPASSWORD


Set primary_hostname to your fully qualified server-name.

Make Exim listen on SMTP-SSL, too:

  1. ports to listen on (smtps is forced to use TLS/SSL via tls_on_connect_ports)

daemon_smtp_ports = smtp : smtps


localdomains are fetched from database. if mail is comming for USER@DOMAIN, it will return DOMAIN, so to make Exim feel responsible for a domain just add a user in the database.

domainlist local_domains = ${lookup mysql {SELECT DISTINCT domain FROM user WHERE domain='${quote_mysql:$domain}'}}


Some SSL/TLS configuration. use your certificates and enforces SSL on SMTP-SSL

  1. SSL/TLS config

tls_advertise_hosts = *

  1. additionally listen on ssl/smtp

tls_on_connect_ports = 465 tls_certificate = /etc/exim/exim.crt tls_privatekey = /etc/exim/exim.key

  1. log some details

log_selector = +tls_cipher +tls_peerdn


Define our ACLs to be called on RCPT and DATA SMTP-command:

acl_smtp_rcpt = acl_check_rcpt acl_smtp_data = acl_check_data


Define ClamAV-connection via socket:

av_scanner = clamd:/var/run/clamav/clamd.sock


Define SpamAssassin-connection via socket:

spamd_address = /var/run/spamassassin/spamd.sock


Add these lines after 'accept hosts = :' in acl_check_rcpt:

  1. temporary reject message, if already greylisted and entry hasn't expired yet
  2. authenticated users skip this

defer message = Your Message is currently still greylisted! Please try again later. log_message = message from ${sender_address} over ${sender_host_address} is still GreyListed !authenticated = * # true, if triple is in db and not yet GREYLIST_TIMEOUT seconds since first seen # false, else (older or not in db) condition = ${if >={GREYLIST_TIMEOUT}{${lookup mysql{ SELECT (UNIX_TIMESTAMP()-MAX(`first_seen`)) AS QueueTime FROM `greylist` WHERE `SenderIP` = '${quote_mysql:$sender_host_address}' AND `SenderAddress` = '${quote_mysql:$sender_address}' }{$value}{${eval:GREYLIST_TIMEOUT+1}}}}{true}{false}}


  1. save authenticated user in header, if nessessary (intentionally done before spamcheck, to use it's headers)

warn authenticated = * message = X-Authenticated-User: $authenticated_id\n\ X-Authenticator: $sender_host_authenticated

  1. deny, if foreign, unauthenticated connection claims to come from a local domain

deny message = Sender claims to have a local address, but is not authenticated nor relayed (try using SMTP-AUTH!) log_message = Forged Sender address (claims to be local user [${sender_address}], but isn't authenticated) !hosts = +relay_from_hosts !authenticated = * condition = ${if match_domain{$sender_address_domain}{+local_domains}}

  1. we're doing HELO checks here, because we can't add headers in acl_smtp_helo

warn message = X-Invalid-HELO: HELO is IP only (See RFC2821 4.1.3) log_message = HELO ($sender_helo_name) is IP only (See RFC2821 4.1.3) condition = ${if isip{$sender_helo_name}} warn message = X-Invalid-HELO: HELO is no FQDN (contains no dot) (See RFC2821 4.1.1.1) log_message = HELO ($sender_helo_name) is no FQDN (contains no dot) (See RFC2821 4.1.1.1) # Required because "[IPv6:<address>]" will have no .s condition = ${if match{$sender_helo_name}{\N^\[\N}{no}{yes}} condition = ${if match{$sender_helo_name}{\N\.\N}{no}{yes}}

warn message = X-Invalid-HELO: HELO is no FQDN (ends in dot) (See RFC2821 4.1.1.1) log_message = HELO ($sender_helo_name) is no FQDN (ends in dot) (See RFC2821 4.1.1.1) condition = ${if match{$sender_helo_name}{\N\.$\N}}

warn message = X-Invalid-HELO: HELO is no FQDN (contains double dot) (See RFC2821 4.1.1.1) log_message = HELO ($sender_helo_name) is no FQDN (contains double dot) (See RFC2821 4.1.1.1) condition = ${if match{$sender_helo_name}{\N\.\.\N}}

warn message = X-Invalid-HELO: Host impersonating [$primary_hostname] log_message = HELO ($sender_helo_name) impersonating [$primary_hostname] condition = ${if match{$sender_helo_name}{$primary_hostname}{yes}{no}}

warn message = X-Invalid-HELO: $interface_address is _my_ address log_message = HELO ($sender_helo_name) uses _my_ address ($interface_address) condition = ${if eq{[$interface_address]}{$sender_helo_name}}

warn message = X-Invalid-HELO: no HELO log_message = no HELO ($sender_helo_name) condition = ${if !def:sender_helo_name}


Add these lines after 'require verify = sender' in acl_check_rcpt:

  1. embed a header flag, if sender callout verification fails. this may lead to rejection in future, or give a hint to bayes filter
  2. the next both directives have complement verify conditions, so only one matches

warn message = X-Sender-Verify: FAILED ($sender_verify_failure) log_message = Sender ($sender_address) could not be verified using callout: $acl_verify_message ($sender_verify_failure) !verify = sender/callout=10s,random

warn message = X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) verify = sender/callout=10s,random


Edit the line after 'accept authenticated = *' in acl_check_rcpt:


control = submission/sender_retain/domain=


Exchange all lines of DATA-ACL acl_check_data with:

  1. Unpack MIME containers and reject file extensions
  2. used by worms. Note that the extension list may be
  3. incomplete.

deny message = $found_extension files are not accepted here demime = com:exe:vbs:bat:pif:scr


  1. Reject messages that have serious MIME errors.
  2. This calls the demime condition again, but will return cached results.

deny message = Serious MIME defect detected ($demime_reason). demime = * condition = ${if >{$demime_errorlevel}{2}{1}{0}}

  1. Deny if the message contains a virus. Before enabling this check, you
  2. must install a virus scanner and set the av_scanner option above.

deny message = This message contains a virus ($malware_name) and is rejected. 

 	log_message 	= rejected VIRUS ($malware_name) from $sender_address to $recipients
 	condition 	= ${if < {$message_size}{20M}}
 	demime 	  	= *

malware = *

  1. Add headers to all messages (:true). Before enabling this,
  2. you must install SpamAssassin. You may also need to set the spamd_address
  3. option above.

warn message = X-Spam-Score: $spam_score\n\ 

                   X-Spam-Score-Int: $spam_score_int\n\
                   X-Spam-Bar: $spam_bar\n\
                   X-Spam-Report: $spam_report

condition = ${if < {$message_size}{1M}} spam = spamassassin:true

  1. Reject spam messages with score over 10+2*max_score_from_db (fallback=15), using an extra condition.

deny message = This message is classified as UBE (SPAM) and therefore rejected. You scored $spam_score points. Congratulations! condition = ${if >={$spam_score_int}{${lookup mysql{ SELECT ((max(spam_threshold)*2+10)*10) AS spam_reject_threshold FROM user WHERE SMTP_allowed='YES'}{$value}{15}}}{true}{false}}

  1. temporary reject message for greylisting, if spamscore is above 2.0 and the message (sender address + IP) is seen for the first time
  2. authenticated users skip this

defer message = Your Message will be greylisted! Please try again in GREYLIST_TIMEOUT seconds. log_message = message from ${sender_address} over ${sender_host_address} will be GreyListed !authenticated = * condition = ${if >={$spam_score_int}{20}{true}{false}}

  1. false, if triple is in db (at this point if it's in the timeout has expired)
  2. true, if not

condition = ${lookup mysql{ SELECT MAX(`first_seen`) FROM `greylist` WHERE `SenderIP` = '${quote_mysql:$sender_host_address}' AND `SenderAddress` = '${quote_mysql:$sender_address}' }{false}{true}}

  1. insert triple into database (which should succeed)

condition = ${lookup mysql{ INSERT INTO `greylist` ( `SenderIP`, `SenderAddress`, `first_seen` ) VALUES ( '${quote_mysql:$sender_host_address}','${quote_mysql:$sender_address}',UNIX_TIMESTAMP() ) }{$value}fail}


  1. save exim version and current date in header

warn message = X-Exim-Version: $version_number (build at $compile_date)\n\ X-Date: $tod_log\n\ X-Connected-IP: $sender_host_address:$sender_host_port

  1. save additional information in header

warn message = X-Message-Linecount: $message_linecount\n\ X-Body-Linecount: $body_linecount\n\ X-Message-Size: $message_size\n\ X-Body-Size: $message_body_size\n\ X-Received-Count: $received_count\n\ X-Recipient-Count: $recipients_count\n\ X-Local-Recipient-Count: $rcpt_count\n\ X-Local-Recipient-Defer-Count: $rcpt_defer_count\n\ X-Local-Recipient-Fail-Count: $rcpt_fail_count

  1. Accept the message.

accept


Define the following routers directly after the dnslookup-router in the router-section (Order is important!):

  1. alle@<domain> is an auto-generated alias for all users of <domain>, which is only available for authenticated senders
  2. NOTE: we need to respect SMTP_allowed for every user!

mysql_all_domain_alias: 

 driver = redirect
 # restriction to local domains only may be a double check, as data takes care of it already ;-)
 domains = +local_domains
 local_parts = alle
 data = ${lookup mysql{SELECT CONCAT(username,'@',domain) AS sendto FROM user WHERE domain='${quote_mysql:$domain}' AND SMTP_allowed='YES'}}
 condition = ${if or {{\
 			def:authenticated_id\

}{\ eq {$sender_host_address}{127.0.0.1}\ }\ }\ } 

 file_transport = address_file
 pipe_transport = address_pipe
  1. an alias can be specified by giving one or more db-entries that match username and domain,
  2. or return a comma-seperated list of recipients.
  3. when no domain is specified in db-entry, recipients are taken from all domains with a matching username
  4. setting internal='YES' only allows sending mail to this alias, if authenticated (for internal usage)

mysql_alias: 

 driver = redirect
 # restriction to local domains only may be a double check, as data takes care of it already ;-)
 domains = +local_domains
 file_transport = address_file
 pipe_transport = address_pipe
 data = ${if or {{\
 		   def:authenticated_id\

}{\ eq {$sender_host_address}{127.0.0.1}\ }\ }{\ 

 			${lookup mysql{SELECT sendto FROM alias WHERE ( username='${quote_mysql:$local_part}' AND (domain='${quote_mysql:$domain}' OR domain=) )}}\

} {\ ${lookup mysql{SELECT sendto FROM alias WHERE ( ( username='${quote_mysql:$local_part}' AND (domain='${quote_mysql:$domain}' OR domain=) ) AND internal='NO' )}}\ }} 

 local_part_suffix = +*
 local_part_suffix_optional
  1. virtual user in mysql-db? and suffixed with a condition?
  2. currently supported:
  3. <user>#before#<yyyymmdd>@<localdomain> e.g.: pille#before#20061003@struction.de will accept mail for existing user pille@struction.de, if current date is before 20061003
  4. <user>#fromdomain#<senderdomain>@<localdomain> e.g.: pille#fromdomain#difu.de@struction.de will accept mail for existing user pille@struction.de, if current domain of sender is difu.de

mysql_user_condition: 

 driver = accept
 # restriction to local domains only may be a double check, as the condition takes care of it already ;-)
 domains = +local_domains
 # as we embed base64 encoded strings in local_part_suffix, and these are case sensitive, we must take care of them.
 # NOTE: this results in the missing feature, that conditional-mails in this router are case-sensitive! (pille#...#...@struction.de != Pille#...#...@struction.de)
 caseful_local_part = true
 condition = ${if and {{\
 			# existing user
 			eq {${lookup mysql{ SELECT DISTINCT CONCAT(username,'@',domain) AS email FROM user WHERE username='${quote_mysql:$local_part}' AND domain='${quote_mysql:$domain}' AND SMTP_allowed='YES'}{true}{false}}}{true}\

}{\ # different conditions or {{\ # suffix contains #before# and date (yyyymmdd) is not yet #before#yyyymmdd and {{\ eq {${sg{$local_part_suffix}{^#([^#]+)#[0-9]\{8\}\$}{\$1}}}{before}\ }{\ lt {$tod_logfile}{${sg{$local_part_suffix}{^#[^#]+#([0-9]\{8\})\$}{\$1}}}\ }\ }\ }{\ # suffix contains #fromdomain# and the domain-name of sender and {{\ eq {${sg{$local_part_suffix}{^#([^#]+)#.*\$}{\$1}}}{fromdomain}\ }{\ eq {$sender_address_domain}{${sg{$local_part_suffix}{^#[^#]+#(.*)\$}{\$1}}}\ }\ }\ }{\ # suffix contains #b64from# and the base64 encoded address of sender DOES NOT WORK YET! and {{\ eq {${sg{$local_part_suffix}{^#([^#]+)#.*\$}{\$1}}}{b64from}\ }{\ eq {${str2b64:$sender_address}}{${sg{$local_part_suffix}{^#[^#]+#(.*)\$}{\$1}}}\ }\ }\ }\ }\ }\ }\ } 

 local_part_suffix = #*
 transport = local_mysql_delivery


mysql_user: 

 driver = accept
 # restriction to local domains only may be a double check, as the condition takes care of it already ;-)
 domains = +local_domains
 condition = ${lookup mysql{ SELECT DISTINCT CONCAT(username,'@',domain) AS email FROM user WHERE username='${quote_mysql:$local_part}' AND domain='${quote_mysql:$domain}' AND SMTP_allowed='YES'}{true}{false}}
 local_part_suffix = +*
 local_part_suffix_optional
 transport = local_mysql_delivery
 no_more
  1. a catchall domain can be specified by giving one or more db-entries that match the domain,
  2. or return a comma-seperated list of recipients.
  3. this router acts as a fallback, so it has to be placed below all routers that react on 'users'.
  4. any mail to a not otherwise (in another router above) defined local_prefix in these domains are forwarded.
  5. so keep in mind that this mostly may forward unsolicited mail and should not be used at all ;-)

mysql_catchall: 

 driver = redirect
 # restriction to local domains only may be a double check, as data takes care of it already ;-)
 domains = +local_domains
 file_transport = address_file
 pipe_transport = address_pipe
 data = ${lookup mysql{SELECT sendto FROM catchall WHERE domain='${quote_mysql:$domain}'}}


Add the following line to the remote_smtp-transport in the transport-section to remove some headers for outbound mail. 

 headers_remove = X-Spam-Report:X-Spam-Bar


Define a new transport local_mysql_delivery in the transport-section. This one will be called for virtual users to deliver the mail into their maildir:

local_mysql_delivery: 

 driver = appendfile
 directory = /var/mail/${domain}/${local_part}/
 maildir_format
 delivery_date_add
 envelope_to_add
 return_path_add
 user = mail
 group = mail
 mode = 0660
 # at this time, we know a local user to get his individual preferences to tag the mail
 # the '${eval:$header_X-Spam-Score-Int:}' is is a hack to cope with negative ints that seem to be parsed as strings, thus failing the comparsion
 # if there's no X-Spam-Score-Int header set by data-acl above, don't panic ;-)
 # another hack is that we remove important headers, we add later to be sure there are no multiple versions from earlier relays, or forged ones (this is BUGGY right now as it merges all equal headers!)
 #     therefore I implemented the ${sg{$header_X-Spam-Score-Int:}{^.*\n}{}} regex hack, that strips all
 headers_remove = Subject : X-Spam-Flag : X-Spam-Score-Int : X-Spam-Score : X-Spam-Bar : X-Spam-Report 
 headers_add = "X-Spam-Threshold: ${lookup mysql{ SELECT DISTINCT spam_threshold FROM user WHERE username='${quote_mysql:$local_part}' AND domain='${quote_mysql:$domain}' AND SMTP_allowed='YES'}{$value}{ERROR}}\n\
 		 X-Spam-Score: $header_X-Spam-Score:\n\

X-Spam-Score-Int: $header_X-Spam-Score-Int:\n\ X-Spam-Bar: $header_X-Spam-Bar:\n\ X-Spam-Report: $header_X-Spam-Report:\n\ 

                X-Spam-Flag: ${if def:header_X-Spam-Score-Int:{\

${if >={${eval:${sg{$header_X-Spam-Score-Int:}{^.*\n}{}}}}\ {${eval:10*${lookup mysql{ SELECT DISTINCT spam_threshold FROM user WHERE username='${quote_mysql:$local_part}' AND domain='${quote_mysql:$domain}' AND SMTP_allowed='YES'}{$value}{ERROR}}}}{YES}{NO}}\ }{\ UNKNOWN\ }}\n\ Subject: ${if def:header_X-Spam-Score-Int:{\ ${if >={${eval:${sg{$header_X-Spam-Score-Int:}{^.*\n}{}}}}\ {${eval:10*${lookup mysql{ SELECT DISTINCT spam_threshold FROM user WHERE username='${quote_mysql:$local_part}' AND domain='${quote_mysql:$domain}' AND SMTP_allowed='YES'}{$value}{ERROR}}}}{${lookup mysql{ SELECT DISTINCT spam_tag FROM user WHERE username='${quote_mysql:$local_part}' AND domain='${quote_mysql:$domain}' AND SMTP_allowed='YES'}{$value}{ERROR}}$h_subject:}{$h_subject:}}\ }{$h_subject:}}\n\ X-Delivered-To: $original_local_part@$original_domain ($local_part@$domain)\n\ X-Message-Age: $message_age"


Define the following two authenticators in authenticators-section. They will verify the password for SMTP-AUTH from the database, but advertise this functionality only if the connection is encrypted to avoid disclosure:

plain: driver = plaintext public_name = PLAIN server_advertise_condition = ${if eq{$tls_cipher}{}{no}{yes}} server_condition = ${if crypteq {$3}{\{sha1\}${lookup mysql {SELECT DISTINCT password FROM user WHERE CONCAT(username,'@',domain)='${quote_mysql:$2}' AND SMTPAUTH_allowed='YES'}}}{yes}{no}} server_set_id = $2

login: driver = "plaintext" public_name = "LOGIN" server_prompts = Username:: : Password:: server_advertise_condition = ${if eq{$tls_cipher}{}{no}{yes}} server_condition = ${if crypteq {$2}{\{sha1\}${lookup mysql {SELECT DISTINCT password FROM user WHERE CONCAT(username,'@',domain)='${quote_mysql:$1}' AND SMTPAUTH_allowed='YES'}}}{yes}{no}} server_set_id = $1


List of Config-Files

Have a look for complete (Gentoo-based) config-files.

Please do not use without reading them first. there may be references to my domain-name in them and I don't want to read your mail! MySQL 

   * System-Database-Scheme

Exim 

   * /etc/exim/exim.conf

ClamAV current versions (since 0.90) 

   * /etc/clamd.conf
   * /etc/freshclam.conf

old versions (before 0.90) 

   * /etc/clamd.conf
   * /etc/freshclam.conf

SpamAssassin 

   * /etc/spamassassin/local.cf
   * /etc/spamassassin/init.pre
   * /etc/spamassassin/v310.pre
   * /etc/spamassassin/v312.pre
   * /etc/spamassassin/99_struction_IXHASH.cf
   * /etc/spamassassin/99_struction_WrongMX.cf
   * /etc/spamassassin/99_struction_DNSRBL.cf
   * /etc/spamassassin/99_struction_EXIM.cf
   * /etc/spamassassin/iXhash.pm
   * /etc/spamassassin/wrongmx.pm
   * /etc/conf.d/spamd
   * /etc/cron.daily/spamassassin-expire-bayes

Dovecot 

   * /etc/dovecot/dovecot-sql.conf
   * /etc/dovecot/dovecot.conf

Addons change password plugin for squirrelmail

For SquirrelMail (an IMAP-WebMail) there's a plugin called change_sqlpass which does not seem to be maintained for some years now. I've written a patch against the last version 3.3 to file change_sqlpass/functions.php to support the base64 encoded sha1 password hashes.

Configure it by creating a MySQL-user for squirrelmail and containing the following lines in change_sqlpass/config.php:

$csp_dsn = 'mysql://squirrelmail:SQUIRRELMAILPASSWORD@unix(/var/run/mysqld/mysqld.sock)/system'; $lookup_password_query = 'SELECT count(*) FROM user WHERE username = "%2" AND domain = "%3" AND password = %4'; $password_update_queries = array( 'UPDATE user SET password = %4 WHERE username = "%2" AND domain = "%3"' ); $force_change_password_check_query = ; $password_encryption = 'B64SHA1'; $csp_salt_static = ; $csp_salt_query = ;


Apache mod_auth_mysql encryption scheme patch

For Apache's mod_auth_mysql I've written a patch against the last version 3.0.0 to file mod_auth_mysql.c to support the base64 encoded sha1 password hashes in our database. (see it on SourceForge's project patch page)

Configure it by creating a MySQL-user called apache with read-only access to the relevant username/domain/password fields using the password APACHEPASSWORD:

GRANT USAGE ON *.* TO 'apache'@'localhost' IDENTIFIED BY 'APACHEPASSWORD' WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0; GRANT SELECT ( `username` , `domain` , `password` ) ON `system`.`user` TO 'apache'@'localhost';


After compiling the patched module, you can create a password authenticated directory, all users of the mail-system can login to, by placing the following content in a .htaccess file (or the apache config):

AuthName "MySQL authenticated zone" AuthType Basic

AuthMySQLUser apache AuthMySQLPassword APACHEPASSWORD AuthMySQLDB system AuthMySQLUserTable user AuthMySQLNameField CONCAT(username,'@',domain) AuthMySQLPasswordField password AuthMySQLPwEncryption sha1base64

require valid-user

Retrieved from "http://localhost/index.php/Exim,_Virtual_Domain_Setup"
Personal tools