The email delivery process

Between the time an email is sent and the time it is received at the other end, it will have gone through a number of servers and checks in order to make sure it is a valid message and is being organized into the right place. This process can be complicated, but this page details the precise process of email delivery at Fastmail, which can be useful for filtering your mail, or just understanding how email works in general.

Email delivery: a brief overview

This is a general overview on how an email is delivered on a Fastmail account with no custom Sieve scripts and standard spam protection. Some custom Sieve scripts might change the order of some of these steps.

When an email arrives at Fastmail, it comes in via SMTP. SMTP is designed to move your email across different networks, and helps transfer messages between different mail providers.

When we receive a message, we immediately check for two things: the sender's email address and the recipient's email address. If either of these is missing, we reject the email.

After we check that the addresses are included, we look further at the target address (the address the message is sent to, which might be the To:, Cc:, or Bcc: address). We look up the domain in our system, then the address at the domain, and check that these match an address at a user account (or an external address). If we can't find a matching account, the message is rejected.

We begin adding headers to messages that contain server information on the message (for a more detailed explanation on this process, see the technical information below). If the message is redirected to an external mail service through an alias, we send it through to their servers. Spam checking is run on messages and we add headers with the results. If the message is detected as spam, we then sort it into the Spam folder on the user's account.

If the email passes the spam checking process, we run any Forwarding and Organize rules that the user has saved on the account. If one of these rules match, we take the rule's action, or if no rules match, the email is then sorted into the Inbox or into a custom folder if the message was sent to a matching plus or subdomain address.

A technical deep dive

A complete technical guide to the email delivery process is as follows:

  1. Email arrives via SMTP. The SMTP protocol requires two parameters during the transaction, a "MAIL FROM" value and a "RCPT TO" value.

    The "MAIL FROM" parameter is supposed to contain the email address of the sending party. If the email is from the "postmaster" address at the remote site, an empty value of <> is used. The value used here is completely independent of what is shown in the actual email header From line.

    The "RCPT TO" parameter specifies the recipients of the message. The value used here is completely independent of what is shown in the actual email header To/Cc lines. This is how emails that are BCC'ed to you, or have forged headers, can arrive at your account even though your address is not in the header.

  2. Any aliases, domains and subdomains are matched and translated to the "Target" values. This is done repeatedly until no more translations occur. The order is as follows:

    a) Any address of the form user@subdomain.srcdomain.com is translated to subdomain+user@srcdomain.com. A more detailed explanation of this can be found in our documentation of subdomain addressing.

    b) For an address of the form name+plus@srcdomain.com, if an alias name@srcdomain.com exists, then the address is replaced with the "Target" of the alias.

    c) If address contains a "+" part (e.g. name+srcplus@srcdomain.com) and the "Target" also contains a "+" part (e.g. target+trgplus@targetdomain.com) then the two are combined into target+trgplus.srcplus@targetdomain.com.

    d) If no specific name@srcdomain.com alias exists, but a 'catch all' alias exists (e.g. *@srcdomain.com), then the target of that is used. If the "Target" contains an asterix (*) in it, between the "+" part and the "@" part, then it is replaced with the original name part of the address. For example, suppose:

    • The alias *@srcdomain.com exists with a "Target" of yourname+*@targetdomain.com.
    • An email sent to john@srcdomain.com will be rewritten to yourname+john@targetdomain.com.
    • If you have a folder in your account called "John", then this email will automatically be filed straight into that folder. You can also deliver straight to a subfolder by using a "Target" of yourname+parentfolder.*@targetdomain.com.

    e) If the "Target" contains commas, then it is separated on the commas into multiple target values. Each of the actions described above occurs on each of the target addresses.

  3. If the resultant target (or targets) is an external address, the email is then queued for sending to the external address.

  4. If the address is local, it is delivered to the appropriate local mailbox. During delivery, the following also occurs:

    • The original SMTP "MAIL FROM" parameter is added to the email as an X-Mail-from header.
    • The original SMTP "RCPT TO" parameter is added to the email as an X-Delivered-to header.
    • The result delivery address based on the alias and subdomain translations above is added to the email as an X-Resolved-to header.
    • If the email has any attachments, the names of the attachments are added as X-Attached headers.
    • The email is scanned by our content filter.
    • The matching user account's Sieve script is checked for matching Discard rules, and if any tests match, the message is discarded.
    • If the user has spam checking enabled, the email is spam checked.

      • Any SpamAssassin "hits" are put in the X-Spam-hits header. See SpamAsassin's documentation for a list of the main tests performed and their scores.
      • A "spam score" is put in the X-Spam-score header. It's clamped to be always >= 0, even if the actual score was negative.
  5. After that, the user's "Sieve scripts" for Forwarding and Organize rules are run on the email, in addition to any custom Sieve added after spam filtering. This is a script generated from the Settings → Rules screen which controls the discard, forwarding, and file-into-folder actions for the email.

  6. If the final delivery address has a "+" in it, and no particular Sieve rule catches the email to file it into a specific folder, then the email is delivered to the folder name after the '+' part. Folder name matching is not case sensitive, and with the characters '_', '-', and the space character all being regarded as the same. For example, email to username+sent_items@domain.com will be placed into the "Sent Items" folder. If no such folder exists, then the email will be delivered to the standard Inbox. You can deliver to subfolders by using '.' to separate folder names.

  7. The 'envelope' value seen by the Sieve script is the resultant target value, not the original 'MAIL FROM' or 'RCPT TO' parts.

What headers are added by Fastmail?

While your email is being processed, a number of headers are added:

X-Spam-known-sender

The X-Spam-known-sender header tells whether an email should be considered as a known sender and whitelisted. Currently we check the user's contacts against the MAIL FROM address and From header address as well, with the localpart replaced by * (e.g. foo@bar.com → *@bar.com). The checks done may be expanded in the future.

Currently the following values are possible, though these may change in the future. In general you should only rely on the field starting with "yes", containing "in-addressbook" or containing one of the contact group UUIDs.

Where:

In addition, on the end of that list (comma separated), we include any contact groups the address was in as:

Example headers

An example set of the above headers as they might appear on a message (although it's unlikely these would all appear on all messages):

Return-Path: <hannahm@teachdev.io>
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
by sloti2d1t19 (Cyrus 3.1.6-731-g19d3b16-fmstable-20190627v1) with LMTPA;
Tue, 16 Jul 2019 00:43:28 -0400
X-Cyrus-Session-Id: sloti2d1t19-1563252208-3122307-2-17030829279071251430
X-Sieve: CMU Sieve 3.0
X-Spam-known-sender: no
X-Spam-score: 0.0
X-Spam-hits: FREEMAIL_FROM 0.001, HTML_MESSAGE 0.001, ME_ZS_CLEAN -0.001,
RCVD_IN_DNSWL_NONE -0.0001, SPF_HELO_NONE 0.001, SPF_PASS -0.001,
LANGUAGES en, BAYES_USED none, SA_VERSION 3.4.2
X-Spam-source: IP='209.85.166.47', Host='mail-io1-f47.google.com', Country='US',
FromHeader='com', MailFrom='com'
X-Spam-charsets: plain='UTF-8', html='UTF-8'
X-Resolved-to: hannah_martin@fastmail.com
X-Delivered-to: hannah_martin@fastmail.com
X-Mail-from: hannahm@teachdev.io