Starting last month, Rackspace Cloud Sites has not been delivering mail sent through PHP’s mail() function in a timely matter, and sometimes not at all. The company I work at has been using RS Cloud Sites for a few years now, and while mail delivery has always been slower than average, emails tended to arrive within 20 minutes. Now we have emails showing up 3 days late, and there were some tests that I never received.

Why is there no uproar about this? And why has Rackspace failed to notify its customers of this ongoing issue?

Most websites rely critically on scripts that send email for their contact forms and web applications. For any brochure-type website, the main goal is to get the visitor to contact them. Call to actions point to a contact form. The contact form submits to a script that either sends mail via PHP’s mail() (sendmail) or via SMTP.

If this email suddenly starts arriving 3 days late, the client either losing business, or, at the least, has created an initial impression of being slow to respond. If someone is using WordPress, their password recovery is not going to arrive and they won’t be able to access their blog. If a confirmation email is sent after a user signs up, that user now scratches their head and gets upset because they didn’t receive it.

Did you notice that I tagged this under rants?

Should I expect this mail to be delivered? Is PHP mail() reliable?

Usually when sending using PHP mail(), the biggest concern is whether it is flagged as spam and the headers are right, not whether it actually arrives or not. I know other web hosts send the mail faster than Rackspace Cloud ever has, when it was acting normally. We have been using a script that uses mail() on dozens of sites for a few years, and have no had issues until now.

I’m talking about on a Linux server, where mail() is usually configured by PHP.ini to use sendmail. WordPress uses the PHPMailer class and defaults to using the PHP mail function. I think the best description of the downsides of the function itself can be found in the third section on this SwifterMailer docs page. While the SwiftMailer library uses it as a “last resort” because it is overly simplistic and is configured different between Windows/Linux, etc, their second method of transport is indeed sendmail.

It may be better to use SMTP in general, but Mail() is as far as I know should still be generally considered to be reliable when it’s using sendmail. Server experts, correct me if I’m wrong here.

What does Rackspace have to say about this problem?

So we posed the problem to Rackspace Cloud support via Live Chat, and two different tickets. Our eyebrows first raised when our own contact form delivered a message 4 days late. Then our ticket system was sending notifications with a large delay. One of the responses from the first line of inquiry was troublesome. Bolded below:

When e-mails are sent via our SMTP relays through non-authenticated methods, they are sent through our mail relays and placed in a queue. Depending on the queue, these emails can be delayed for an extended period of time. Unfortunately, after four days these emails timeout if they have been trapped in queue and you will not get a bounce back error if this occurs.

In order to enforce our rules for sending through our mail relays our mail relays will trap any messages that do not meet certain criteria. Our rules on sending through our mail relays can be seen here:
http://cloudsites.rackspacecloud.com/index.php/Rules_for_using_Cloud_Site%27s_mail_relays

To enforce rules #2 and #3, our mail relays will only send mail if a valid From address is provided within the email as well as within its header. The first 4 lines of the following article show how you can designate this From address to the PHP mail function. Please note that the From address is designated in the headers passed as the 4th argument to the function as well as to the 5th argument.
http://cloudsites.rackspacecloud.com/index.php/How_can_I_test_PHP_mail_functionality%3F

Emails timeout?? Wha-wha-what? So they are dropped completely?

Any email sent via php mail relay can be delayed up to four days and afterwards the emails are dropped. This is an automatic process and the postmaster is setup to put all email using non-authentication methods in a queue.

This is surprising. So if things get backed up, some mail may be dropped completely without any bounceback.

A lot of the rest of the response was regarding bulk emails. If you are sending a large amount of emails in bulk, then please review Rackspace’s policies in detail. Use MailChimp or SendGrid for your large email needs. I won’t get into that discussion here. I’m only talking about occasional single emails. In our case, they were sent to an address at the same domain, and previously were being delivered within 20 minutes.

The second ticket I presented the issue I had described above, where 2 sites in development did not have contact forms being delivered, and we had been contacted by two of our clients, saying they were not receiving inquiries.

When sending via the basic php mail() function emails are going to be subject to our open relays. At times these can get congested and cause delays. We are very aware of this issue and in the long term our operations and engineering teams are working on a permanent solution. In the short term currently SMTP authentication is going to be the only workaround. Using SMTP authentication is going to be much more reliable and we highly recommend it’s use.

Here is an example script using SMTP authentication that I think may be helpful:

http://www.rackspace.com/knowledge_center/index.php/How_do_I_test_PHP_SMTP_functionality

Unfortunately this is going to be the only valid solution at this time. If you have any further questions please let us know. Thanks for being a Customer! Remember we are here for you 24x7x365! Have a great day!

This has me concerned that Rackspace Cloud Sites may be overloaded, and/or is headed downhill. Are certain people abusing the system and strong limiting needs to be put in place, so the smaller quantities and single emails are not stuck in a traffic jam? Perhaps this is a really technical problem to do with the cloud architecture is in place. This is all conjecture on my part, and I’m interested to know if anyone has any more specific details about why this is like this. Rackspace has always seemed to be positioned as “best in the biz”, and now I am doubtful. Though, RS Cloud was SliceHost. It was purchased by Rackspace in October 2008. So perhaps this is more of a reflection on the growing pains involved.

UPDATE 3/26: Rackspace has weighed in on the issue with some more details about the problem and how they are working on fixing it. There should already be improvements as a result of their tuning. See the comments below this article.  

How to fix it right now and make sure your e-mail is delivered

Okay. Time for a deep breath. If you only have your own site to deal with, it’s possible to get this resolved real quickly using PEAR mail. If you have 300 sites to deal with..well, you’re in a tough spot. Since the only solution provided by Rackspace is to use SMTP, the first thing you need to do is:

  1. Create an email address, or use the credentials for an existing email address. This may be noreply@yourdomain.com, or any other email address of your choosing, used only for sending out mail.
  2. Make note of the outgoing server and outgoing port number for this email account. If your email is at Rackspace Cloud Sites, their settings are at the link below under PEAR module.

Custom PHP Scripts – Use the PEAR Mail Module

Rackspace Cloud has the PEAR module already installed on the server for easy inclusion. Their knowledge base article has the details on how to use it. All you need to do is change a few lines of code in your existing script:

Make sure the FROM address is set to the same email being used to send. In the case of a contact form where one of the submitted fields is the person’s e-mail address, you can set that to the REPLY-TO. That way, when whoever receives the contact form hits reply, they will be replying to the person who submitted it. And the headers remain as legitimate as possible by saying the FROM is the same as the account actually sending it out.

WordPress – Install the WP Mail SMTP plugin

This free plugin changes the wp_mail function to use SMTP instead of mail(). Install the plugin and change the settings as necessary for your e-mail account and outgoing server/port. More info on the plugin page here.

CodeIgniter / Other Scripts/Libraries – Use their SMTP Options

Check your framework or library for SMTP configuration options. For CodeIgniter, see their documentation on the Email class. You will need to set the protocol to “smtp”, and then set the “smtp_host”, “smtp_user”,” smtp_pass”, and “smtp_port”. Note: For some reason, the configuration option $config['_smtp_auth'] = TRUE;  is no longer listed. I think is is for older versions of CodeIgniter only. I needed to add this for it to work on the CodeIgniter site I recently updated.

In Closing

Note: Your email server’s settings may be unique and I can’t debug them here. Please use the appropriate support channels for your email / web hosting with any issues sending mail.

I hope Rackspace figures out a way to clear up this issue soon. I was right in the middle of starting a post on the pros and cons of Rackspace Cloud Sites hosting after 2+ years of hosting, when this issue arose. In general, we’ve been fairly happy with them; especially the helpful customer support and live chat. There are a few major outstanding issues that I think are holding them back, that would not be too hard to remedy and would make everyone happy. More on that later. Till next time.

Other mentions similar to this issue I could find online:

Comments on this Article

  1. Josh Odom says:

    Hi Josh,
    I lead product management and engineering for the Cloud Sites product at Rackspace and just wanted to give you an update on this issue.

    The overall performance of our mail relays especially over the last few weeks has not been up to our standards and has been something my team has been working on correcting. Much of the problem we have been experiencing has been due to spam being sent through the Cloud Sites mail relays leading to mail being discarded or unacceptable delivery times. We’ve tackled this problem by making some significant improvements to our filtering infrastructure, which we are continuing to tune on a daily basis. In most cases, the source of this spam is due to customers running out of date applications or plug-ins, which are specifically targeted by spammers. We’re working with these customers directly to ensure that they’ve patched their application and closed the hole spammers are using to send these messages.

    We’re going to be watching this very carefully over the next few weeks. Additionally, we’re in the process of deploying better monitoring so that we are able to take more proactive steps when we start seeing out-of-bounds activity on the relays so that we can take corrective action faster.

    As of now, you should see much faster delivery times and reliability in the messages being sent through the PHP mail() function,

    Regards,
    – Josh Odom
    – Product Line Leader, Cloud Platform
    – Rackspace Hosting

    • Josh says:

      Hi Josh. I appreciate your quick response to this, and the details about what has been happening. I can only imagine how it is to deal with spammers, bot-nets, and compromised sites on a larger scale. The improvements are great news I hope things continue to stabilize.

  2. Austin says:

    Thanks for writing this. We’ve had more complaints than usual last week from our clients (over 250) so I’ve been trying to figure out the best way to handle these delays for the sites. Looks like it’s SMTP time!

  3. Chris McCauley says:

    Hi all,

    The situation is worse than you think. Rackspace are sending their own email through the same mechanism. We regularly get notifications of ‘upcoming’ system maintenance but the email has been delayed several days. Just got an email from Rackspace accounts (15mins ago) and it was actually sent on Friday (four days ago).

    Really poor service on this

    Chris

    • Josh says:

      Interesting. So had the system maintenance already happened when you received it? I thought it had leveled out. On the 14th or 15th, I sent a test from a tried-and-true sendmail script and got an email within 2 minutes; a new record.

  4. Melvyn says:

    Josh, since the end of April 2012 we have seen a very large number of emails being delayed from 3 hours to 4 days still. While we maintain our own mail server, outside clients that have accounts with RackSpace; their mail is severly delayed. When a user tells me about a message being delayed, I really don’t have to look anymore because it points to RackSpace. Called a customer rep. and they were helpful but they see in your logs that it is delayed and don’t know what to tell me. We do not have a problem with any other mail but from RackSpace. My users just have to deal with it.

  5. Nathan says:

    Excellent post, thank you! I’m still having issues with this and discovered your post. Have you switched everything to authenticated SMTP? Are you still having issues as well?

    • Josh says:

      It is not as bad as before; plaintext emails have been much speedier.

  6. 916 Networks says:

    It’s almost November and I’m still having major issues. I recently “Upgraded” a client from an old Linux server (that worked!!!) to CloudSites. The speed of my site improved, but major email delays are severely impacting their business and their confidence in the system that has been working fine for 6 years on an in-house server.

    Any update to this? Rackspace is great in so many ways, but as I’m moving people to Cloud Sites I am coming across many deal breakers for me and my clients.

    • Josh says:

      I haven’t heard of any updates since; the contact forms lately have been arriving within 15 mins.

  7. james says:

    Hi Josh, I’m experiencing a similar issue with one of my clients who’s email is with rackspace. The site is sending s VERY simple mail form using php mail() but just isn’t getting through to the rackspace based email. I ran a test on my server that showed it is not running as an open relay but I’m assuming rackspace just assume it is and bin the email. Have you had any update since last year?

    • Josh Winn says:

      The major failure that prompted this didn’t happen again AFAIK. I haven’t been working with RS servers as of last month though.

Comments are closed.