The Paradox Of The Mail Server On The Cloud

Cloud Mail ParadoxProviding your web application with a mail service that works flawlessly is probably essential for your business. You need to send activation emails to users, password reset emails, newsletters and probably a whole bunch of other emails that have to do with interactions with your application.

When there were only physical servers and static IP addresses, everything worked perfectly. But now, when your application is in the cloud, setting up a working mail server next to your application is ridiculously impossible. If your application is successful and you would like to send emails to your millions of satisfied users, your options come down to:

  1. Use a physical hosted server.
  2. Use a 3rd party email service.
  3. Set up a mail server in the cloud and compromise on some/most being marked as spam.

For us cloud oriented developers, option 1 is as useful as somebody suggesting you’d use a cassette tape recorder to put your favorite songs on. It’s old, unreliable, can’t scale. Option 2 is very costly if your business is successful, and most of these services don’t deal with the amount of mails you need to send if you have a large scale user base. Option 3 will make your email communication efforts with your users almost non-existent, which means you can’t afford it as well. So your only option is to compromise somewhere.

Why is sending email from the cloud so difficult?

In order for your mail server to operate successfully and be trusted by mail services around the world, you need to abide by the following rules:

  1. Don’t be an open relay.
  2. Implement (and follow) SPF policy (and DKIM if possible).
  3. Have a PTR record that resolves back exactly to your mail server hostname.
  4. Don’t let your public IP address be listed in any RBLs.

Rule #1 is easily implemented in any mail server configuration, and there are also a number of online tools to test if you’re an open relay or not. Option #2 is also pretty easy to implement, assuming you control your DNS zone files and know your way around it.

The problem of mail on the cloud begins with rules #3 and #4. A PTR record, which is a reverse DNS entry, must be present and correct for your mail server to not be considered spammy. If your mail server is at 1.2.3.4 and is called mail.example.com, the PTR query for 1.2.3.4 (well, for 4.3.2.1.in-addr.arpa) must return mail.example.com. The PTR record can only be changed by the owner of the IP address, or by a delegation of his authority to you. Amazon Web Services do not let you control PTR records, so there goes the option for a mail server on EC2.

Other clouds let you control the PTR records for the IP addresses they assigned to you. But they fail on Rule #4. While your specific IP address might not be blacklisted in RBLs, the entire block that it belongs to might be blacklisted, because these IP addresses are assigned dynamically and therefore are always suspected as spammy by these lists. This is the case with Rackspace Cloud for example, and is the only thing left to be solved before you can run a mail server there. And although they’re trying to get their address block de-listed, this problem still persists.

Other clouds I’ve examined in this space are GoGrid and Joyent. GoGrid want you to fill up a questionnaire, and only then they open up port 25 for you. This sounds absurd, and against all the on-demand nature of the cloud (and I also personally don’t trust ServePath, the company that operates GoGrid). Joyent’s offering seem to disregard the option of hosting a mail server with them, and I couldn’t get their response on this matter.

So unless Rackspace Cloud solve their IP block blacklisting problem, or AWS offer a PTR setting option (plus no blacklisting as well), we’re left with the need to compromise.

The only feasible solution right now — seems like it’s back to physical hosting.

About these ads

5 thoughts on “The Paradox Of The Mail Server On The Cloud

  1. Thanks Oren. I think that your post is very useful for readers who might be confused with how to properly implement a mail server within a cloud hosting environment.

    I would like to offer some additional details from the GoGrid standpoint. For starters, we have always had the policy that people request that port 25 be opened and is closed from the getgo. While this extra step is, I agree, a bit of a hassle, the unblocking usually doesn’t take long to do. Because it is extremely easy to set up (and tear down) servers, many “spam shops” have looked at the Cloud as a great way to set up on-the-fly spamming environments. By requiring this extra step, we have dramatically reduced the number of abuses within the GoGrid environment, thus making it “cleaner” for other users of the service. Users simply have to open a Support Case requesting that port 25 be unblocked and the change is done quickly.

    However, our prevention methods don’t stop there. One of the issues with other providers (AWS, for example) is that frequently the IP address that you are allocated is often from a pool of public IP resources. What that means is that you “could” run into the case where an IP address you are given was used in the past by a spammer, and thus, since you now “lease” that IP, your service could be listed in an RBL. It is possible to get off those lists, but it does take a bit of legwork. So, part of the way that GoGrid helps you avoid that type of situation is that fact that you are given a block of 16 static, contiguous IP addresses from the beginning. You only need to fill out an “IP Justification Form” should you want more beyond the initial 16. More information on that is here. This, too has helped us reduce the number of occurrences of IP addresses being blacklisted. When you get your initial block, those 16 IP addresses are essentially reserved for you to use and “protect.”

    Lastly, as you mention at the end of your post is to move to “physical hosting.” This too, can be accomplished easily with GoGrid using a Hybrid Hosting solution where you have your web/app servers in the Cloud, with GoGrid, and your back-end infrastructure (e.g., DBs or in this case perhaps your sendmail server) hosted on physical hardware with ServePath, all privately connected via a dedicated, physical LAN connection. Just another option I thought I would throw out there.

    True, a spammer could have gone through the process of creating an account, requesting that port 25 be unblocked, used the service maliciously, then cancelled the account (or had it shut down). Then there is a slim possibility that a new user could be assigned the same block, but the odds of that happening are pretty low.

    AWS recently implemented a similar procedure for SMTP limiting on new accounts. Read the thread here for more information. All cloud providers should recognize that this is an issue and take steps to prevent email abuse.

    Regardless, I think your points that you detail in your post are good to have written out. We (at GoGrid) will always work with our customers to help prevent abuse and blacklisting as best as we can.

    Thank you,
    Michael Sheehan
    Technology Evangelist for GoGrid

  2. I have been trying to setup a customer support oriented website where I have a contact form for general inquiry, a user signup page (I am using Drupal) and a help desk support application (OSTicket). Everything depends on my server being able to send out notifications anytime any of the above happens. I singed up with Rackspace and it has been an ongoing battle with them. One technical support just pointed me to their knowledgebase of how to install postfix and mail server and closed the ticket without answering my questions on how to resolve the issue of the IP I am assigned is in Spamhaus’s PBL (Policy based blocking list) – The PBL is implemented by Rackspace itself. They supplied the list to spamhaus and according to spamhaus. Here is an extract from Spamhaus: ” Outbound Email Policy of Rackspace US, Inc. for this IP range:

    It is the policy of rackspace that unauthenticated email sent from this IP address should be sent out only via the designated outbound mail server allocated to rackspace customers. To find the hostname of the correct mail server to use, customers should consult the original signup documentation or contact rackspace Technical Support.”

    I have made several requests on providing information on their outbound mail server. All I need is a single email address originating from my server notifying me and my users. I am frustrated. What is the point of a cloud server hosting? What are we supposed to host on a cloud server? only wikis or 90′s era website? If these functionality are absent from cloud then I do not thing that the cloud business model will succeed.

  3. One way to satisfy requirement #3 (PTR record) is if the cloud provide complies with traditional best practices for anyone assigned a public IP: Give ALL their IP addresses a valid PTR record (such as server1234.cloudhosting.example.com) with a matching A record, Then the cloud e-mail server can simply look up its own PTR record and use that name in its HELO/EHLO SMTP messages, and that is satisfied.

    As for requirement #4 (making sure the cloud e-mail server is not RBL listed), the proper solution is for the cloud provider to offer dedicated e-mail cloud servers that do not allow customer code on them, and to manage those cloud servers the way good business ISPs handle their mail servers. This is similar to how many cloud providers (such as the pioneer Akamai) offers dedicated cloud web servers for scalable high bandwidth non-programmatic content such as downloads.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s