| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| email_advanced [2024/06/23 18:40] – slugmax | email_advanced [2024/09/04 05:37] (current) – “// hc9 |
|---|
| |
| This tutorial discusses advanced topics concerning e-mail on SDF: spam filtering, automatic processing, forwarding. It is probably only useful for ARPA members. | This tutorial discusses advanced topics concerning e-mail on SDF: spam filtering, automatic processing, forwarding. It is probably only useful for ARPA members. |
| |
| |
| ===== Introduction ===== | ===== Introduction ===== |
| | |
| ** and what you //will not// find here** | ** and what you //will not// find here** |
| |
| Here we discuss advanced topics for e-mail processing on SDF, mainly automatic processing of incoming messages with ''procmail'' and forwarding issues. As ''procmail'' is only available to ARPA members, the contents of this tutorial will be mostly useless to //user//-level members. For basic info about e-mail (addresses, reader programs, file size, etc), please read the corresponding [[http://sdf.lonestar.org/index.cgi?faq?EMAIL|FAQ]] entry and [[email_at_sdf|Accessing your SDF Email]]- and if you still have problems with basic sending and receiving of messages via SDF, this tutorial is most probably not for you! | Here we discuss advanced topics for e-mail processing on SDF, mainly automatic processing of incoming messages with ''procmail'' and forwarding issues. As ''procmail'' is only available to ARPA members, the contents of this tutorial will be mostly useless to //user//-level members. For basic info about e-mail (addresses, reader programs, file size, etc), please read the corresponding [[http://sdf.org/index.cgi?faq?EMAIL|FAQ]] entry and [[email_at_sdf|Accessing your SDF Email]] - and if you still have problems with basic sending and receiving of messages via SDF, this tutorial is most probably not for you! |
| |
| ===== Forwarding E-Mail ===== | ===== Forwarding E-Mail ===== |
| The ''sendmail'' system allows for automatic forwarding of incoming mail to other addresses or even programs (filters). The file //$HOME/.forward// may contain the following types of lines: | The ''sendmail'' system allows for automatic forwarding of incoming mail to other addresses or even programs (filters). The file //$HOME/.forward// may contain the following types of lines: |
| |
| somebody@somewhere.com | ^ line ^ description ^ |
| external e-mail address, where a copy of each mail will be sent | | ''somebody@somewhere.com'' | external e-mail address, where a copy of each mail will be sent | |
| ''|filterprogram'' | | ''|filterprogram'' | will pipe the incoming messages through ''/bin/sh'' with the argument ''filterprogram'' | |
| will pipe the incoming messages through ''/bin/sh'' with the argument ''filterprogram'' | |
| |
| However, if you want to do more than forwarding to one address, use of ''procmail'' is highly recommended! In fact, it is best to set up ''nospam'' (see below), which will generate a ''.forward'' file with a line calling ''procmail'', and then to modify the resulting ''.procmailrc''. | However, if you want to do more than forwarding to one address, use of ''procmail'' is highly recommended! In fact, it is best to set up ''nospam'' (see below), which will generate a ''.forward'' file with a line calling ''procmail'', and then to modify the resulting ''.procmailrc''. |
| **Note:** On the SDF cluster hosts, your .forward should contain the following: | **Note:** On the SDF cluster hosts, your .forward should contain the following: |
| |
| "|IFS=' '&&exec /usr/libexec/procmail -f-||exit 75 #username" | ''%%"%%|IFS=' '&&exec /usr/libexec/procmail -f-||exit 75 #username%%"%%'' |
| |
| Replace 'username' with your username. On the MetaArray, a .forward is not needed - the mail server on ma enables procmail processing for all users with a .procmailrc in their home directory. | Replace ‘username’ with your username. On the MetaArray, a .forward is not needed - the mail server on ma enables procmail processing for all users with a .procmailrc in their home directory. |
| |
| ===== Fetching E-Mail ===== | ===== Fetching E-Mail ===== |
| - SDF also supports plus-addressing; you can use this to join mailing lists and do filters. This can be used by adding +something before the @ in your email address. | - SDF also supports plus-addressing; you can use this to join mailing lists and do filters. This can be used by adding +something before the @ in your email address. |
| - The [[http://sdf.org/index.cgi?faq?EMAIL?07|FAQ]] has more information about [[#nospam|nospam]], available on SDF. | - The [[http://sdf.org/index.cgi?faq?EMAIL?07|FAQ]] has more information about [[#nospam|nospam]], available on SDF. |
| - The automatic mail processor "procmail" (see "man procmail" for more information) allows to filter incoming mail following user defined rules; in particular, you can generate a [[#whitelist|white list]] of sender addresses you know as good, and put messages from other addresses into a special folder which you browse through now and then, to check for good messages and adding the corresponding addresses to your white list. | - The automatic mail processor “procmail” (see “man procmail” for more information) allows to filter incoming mail following user defined rules; in particular, you can generate a [[#whitelist|white list]] of sender addresses you know as good, and put messages from other addresses into a special folder which you browse through now and then, to check for good messages and adding the corresponding addresses to your white list. |
| |
| ==== nospam ==== | ==== nospam ==== |
| |
| ''nospam'' is a script which can set up .forward and .procmailrc files so that certain e-mails will be stored in "bulk folders". | ''nospam'' is a script which can set up .forward and .procmailrc files so that certain e-mails will be stored in “bulk folders”. |
| |
| ''nospam -i'' gives some additional information, and ''nospam -e'' will enable this filtering. We will suppose that this has been done also for the following discussion of [[#whitelist|white lists]]. | ''nospam -i'' gives some additional information, and ''nospam -e'' will enable this filtering. We will suppose that this has been done also for the following discussion of [[#whitelist|white lists]]. |
| //Note:// For this to be working, the following //must// have been set up: | //Note:// For this to be working, the following //must// have been set up: |
| |
| * a ''.forward'' file which pipes incoming e-mail through "procmail" | * a ''.forward'' file which pipes incoming e-mail through “procmail” |
| * a standard ''.procmailrc'' file, where this fragment is attached or inserted | * a standard ''.procmailrc'' file, where this fragment is attached or inserted |
| |
| </file> | </file> |
| |
| This will compare the address in the From: field of incoming messages to the ones saved in the file ''$MAILDIR/.whitelist'' (of course, name and location can be changed), and when an address is //not found// in this file, the message will be saved in the folder ''$MAILDIR/quarantine'' (which can be modified as well) and not show up in the normal Inbox (and not be processed further). Now and then, one can look through the quarantine folder for "good messages", and add the corresponding addresses to the ''.whitelist'' file. | This will compare the address in the From: field of incoming messages to the ones saved in the file ''$MAILDIR/.whitelist'' (of course, name and location can be changed), and when an address is //not found// in this file, the message will be saved in the folder ''$MAILDIR/quarantine'' (which can be modified as well) and not show up in the normal Inbox (and not be processed further). Now and then, one can look through the quarantine folder for “good messages”, and add the corresponding addresses to the ''.whitelist'' file. |
| |
| If you want to use the mail directories proposed by nospam, you may prepend the above noted fragment by | If you want to use the mail directories proposed by nospam, you may prepend the above noted fragment by |
| |
| MAILDIR=$HOME/mail | ''MAILDIR=$HOME/mail'' |
| |
| which will result in storing the "possibly bad" messages in ''~/mail/quarantine'' and expect the whitelist in ''~/mail/.whitelist''. | which will result in storing the “possibly bad” messages in ''~/mail/quarantine'' and expect the whitelist in ''~/mail/.whitelist''. |
| |
| A sample ''.whitelist'' may look like this: | A sample ''.whitelist'' may look like this: |
| </file> | </file> |
| |
| All messages not coming from these three addresses would end up in the "quarantine" folder defined in the ''.procmailrc'' fragment. | All messages not coming from these three addresses would end up in the “quarantine” folder defined in the ''.procmailrc'' fragment. |
| |
| ===== Using Procmail With a DNSBL ===== | ===== Using Procmail With a DNSBL ===== |
| |
| DNSBL stands for "DNS blacklist", and it is essentially a collection of IP addresses known to send out spam and other bad stuff. Two popular DNSBLs are [[http://www.spamcop.net|SpamCop]] and [[http://www.spamhaus.org|Spamhaus]]. Typically the mail server will check the source of incoming mail with the DNSBL before even accepting the message. SDF does not do this, but using Procmail to check a DNSBL at the user level is easy. In this tutorial we will use Spamhaus. | DNSBL stands for “DNS blacklist”, and it is essentially a collection of IP addresses known to send out spam and other bad stuff. Two popular DNSBLs are [[http://www.spamcop.net|SpamCop]] and [[http://www.spamhaus.org|Spamhaus]]. Typically the mail server will check the source of incoming mail with the DNSBL before even accepting the message. SDF does not do this, but using Procmail to check a DNSBL at the user level is easy. In this tutorial we will use Spamhaus. |
| |
| (Thank you [[http://www.benya.com/procmail/#dnsbl|Benya]] for the original instructions.) | (Thank you [[http://www.benya.com/procmail/#dnsbl|Benya]] for the original instructions.) |
| Next Procmail should extract the IP address from which the message was sent. This is done using ''formail'' to get the headers, ''grep'' to find the correct line, and ''sed'' to find the actual IP address, which is then saved as ''SENDERIP''. | Next Procmail should extract the IP address from which the message was sent. This is done using ''formail'' to get the headers, ''grep'' to find the correct line, and ''sed'' to find the actual IP address, which is then saved as ''SENDERIP''. |
| |
| | <file config SENDERIP> |
| <file> | |
| SENDERIP = `formail -c -XReceived | grep "by mx.sdf.org" | \ | SENDERIP = `formail -c -XReceived | grep "by mx.sdf.org" | \ |
| grep -v "from mx.sdf.org" | \ | grep -v "from mx.sdf.org" | \ |
| </file> | </file> |
| |
| Then we'll begin the actual Procmail recipe, which will only be executed after checking to make sure that the "SENDERIP" variable exists and is in the correct format. Anything written to the "LOG" variable will be inserted into the log file. (New lines must be explicitly stated.) | Then we'll begin the actual Procmail recipe, which will only be executed after checking to make sure that the “SENDERIP” variable exists and is in the correct format. Anything written to the “LOG” variable will be inserted into the log file. (New lines must be explicitly stated.) |
| |
| <file> | <file> |
| |
| ===== Not Having Your Mail Look Like Spam ===== | ===== Not Having Your Mail Look Like Spam ===== |
| | |
| If you are using SDF's VPM service for your VHOST or other domain email accounts hosted here, you will want to add an SPF record so other services don't consider *your* email as spam: [[add_spf_record_for_your_vpm_mail|How to Add an SPF Record For Your VPM Mail]] | If you are using SDF's VPM service for your VHOST or other domain email accounts hosted here, you will want to add an SPF record so other services don't consider *your* email as spam: [[add_spf_record_for_your_vpm_mail|How to Add an SPF Record For Your VPM Mail]] |
| |
| |
| **example:** | **example:** |
| Use port //443// on SDF host //"otaku"//, tunnel local port //2525// to port //25// on SDF host //"mx"// (SMTP server); restrict ssh to IPv4, no remote commands, run in background after connecting: | Use port //443// on SDF host “//otaku//”, tunnel local port //2525// to port //25// on SDF host “//mx//” (SMTP server); restrict ssh to IPv4, no remote commands, run in background after connecting: |
| % ssh -4 -fN -p 443 -L 2525:mx:25 sdf_user@otaku.sdf.org | % ssh -4 -fN -p 443 -L 2525:mx:25 sdf_user@otaku.sdf.org |
| |
| ==== Tunnel Test: ==== | ==== Tunnel Test: ==== |
| |
| //telnet(1)// can be used; type //"quit"// to exit session: | //telnet(1)// can be used; type “//quit//” to exit session: |
| |
| > % **telnet localhost 2525** | > % **telnet localhost 2525** |
| ==== Tunnel Usage: ==== | ==== Tunnel Usage: ==== |
| |
| Configure MUA to use //"127.0.0.1"// (localhost), port //2525// for out-going messages. Also ensure the //From:// and //ReplyTo:// addresses are routeable. Handling of rejected email is MUA-dependent - some do queuing, others write to //$HOME/dead.letter// or ///dev/null//: | Configure MUA to use “//127.0.0.1//” (localhost), port //2525// for out-going messages. Also ensure the //From:// and //ReplyTo:// addresses are routeable. Handling of rejected email is MUA-dependent - some do queuing, others write to //$HOME/dead.letter// or ///dev/null//: |
| |
| **example:** | **example:** |
| Configure and test the //Heirloom Mailx// MUA for user //"frog"// on localhost //"mud.bog"// to use //"localhost:2525"// for SMTP (off-site email): | Configure and test the //Heirloom Mailx// MUA for user “//frog//” on localhost “//mud.bog//” to use //"localhost:2525"// for SMTP (off-site email): |
| > # **//$HOME/.nailrc//**" | > # **//$HOME/.nailrc//**" |
| > set smtp="localhost:2525" | > set smtp="localhost:2525" |
| |
| ===== References ===== | ===== References ===== |
| | |
| * Setting up [[http://sdf.org/?tutorials/SSH-SDF#public_key|Public Key Authentication]] | * Setting up [[http://sdf.org/?tutorials/SSH-SDF#public_key|Public Key Authentication]] |
| * OpenSSH FAQ: How do I use [[http://openssh.com/faq.html#2.11|Port Forwarding?]] | * OpenSSH FAQ: How do I use [[http://openssh.com/faq.html#2.11|Port Forwarding?]] |
