Home > amazon, business, smugmug, web 2.0 > Amazon S3: The "speed of light" problem

Amazon S3: The "speed of light" problem

March 8, 2007

I was interviewed yesterday by Beth Pariseau for an article about Amazon’s S3 at SearchStorage.com. All-in-all I think it’s a good article that covers some of Amazon’s strengths and weaknesses, but would like to clarify some of my quotes in the article.

I’m quoted as having no read speed issues, but having write speed problems. As is common in articles like this, that’s boiling down a long conversation and much is lost in the translation. 🙂 In reality, Amazon has been blazingly fast for us (both reads and writes), relatively speaking, except for the few times they’ve had problems, which I’ve blogged about before. That particular quote, especially about it being less than a 10th of a second, was my attempt to explain the “speed of light” problem, which applies to both read and writes. Even mighty Amazon hasn’t yet figured out how to transfer data at faster-than-light speeds. 🙂

Basically, we’re in California and Amazon isn’t. This means that when we initiate a read or a write to S3, we’re sending bytes to them and they have to cover, at minimum, the physical distance to Amazon’s datacenters (wherever they are) before anything can be done. Assuming that one of their datacenters in on the East Coast, and assuming we have to read or write from that one occasionally, we’re talking 60-80ms of time just to get bits there and back. No-one on Planet Earth can get around this problem, so it bears consideration when you’re planning for S3 usage.

Obviously, our data in our own datacenters suffers from this problem too – only it’s inches, instead of thousands of miles, to our servers, so it’s almost negligible. But we do have clients all over the world, so the problem is still very real. Our friends Down Under, for example, have to wait much longer for their photos to start drawing than our friends at the Googleplex down the street. If we really wanted to solve that problem, we’d have to build or use a CDN (Content Distribution Network). So far, we haven’t wanted to.

Beth mentions how Bob Ippolito at Mochi Media got better performance in Taipei with CacheFly than with Amazon S3. To me, this seems sorta obvious. To my knowledge, S3 doesn’t have a datacenter in Asia at all, and secondly, they’re not a CDN. Let me say that again – they’re not a CDN. Amazon has their issues they need to overcome with S3, but dinging them for lower performance than a CDN is sorta silly. S3 doesn’t provide web search faster than Google either. See my point?

I’m sure Amazon has thought (or is thinking?) about extending S3 to offer CDN services, but I believe the way Amazon builds these things, it’d probably be a separate service that could be layered on top of S3. They’re into offering building blocks which you can mix & match, not complicated services that do too much. (To any would-be Amazon Web Services competitors reading this, the building block approach is the Right Way to do this.)

Beth’s article is right on the money with regards to data transfer costs, though. S3 currently has two sweet spots: small companies who can’t buy large bandwidth, and companies who need a lot of storage but not a lot of transfers. There are, of course, companies which need a lot of transfers but not much storage (CDNs are probably appropriate here), and companies which need a lot of transfers AND a lot of storage. SmugMug potentially falls into this latter category, but you can imagine someone like YouTube falling into it even more than we do. How they solve the different requirements of different companies will be interesting to watch.

Let me reiterate in case it’s not abundantly clear: I love S3. It’s saved us tons of money. I’m a normal, paying customer – not an Amazon shill. It has problems and growing pains, just like every single other online site or service you can name. It may not be right for you – but it’s certainly right for a ton of us.

I address the “speed of light” issue (and some ways of minimizing it) and the whole “sweet spot” pricing issue on my ETech talk (which I’m still working on). If there’s anything specific you’d like to see, be sure to let me know – I’ll be posting the slides here.

Categories: amazon, business, smugmug, web 2.0
  1. PanMan
    March 9, 2007 at 5:51 am

    And that’s why I would like to see an S3 datacenter in europe. Or an S3 competitor, with similar offering. Where for you it’s 60ms, here it’s 200. That’s why I don’t host all my stuff there.
    But I’m guessing the European market still isn’t that interesting to amazon, and the intercontinental bandwith will be more expensive, if they decide to set up a mirror here.

  2. March 10, 2007 at 6:02 am

    To Whom It May Concern:

    Erica Roush, Captured Moments Studio, performed wedding photography services for us on October 28, 2006. Despite numerous attempts to contact Ms. Roush (voicemail and email) regarding our wedding photo CDs, we have received one return call (several months ago), two emails making empty commitments, and no wedding photo CDs.

    Ms. Roush’s photography service advertisement indicates a 2-4 week turnaround time (see attached). The last time we communicated via email with Ms. Roush she assured us our wedding photo CDs would be put in the mail the very next day, 2-3 day priority mail (December 19, 2006) ~ See attached copy of email correspondence.

    We thought you may have concerns hosting someone that is misrepresenting their services, or perhaps help us to get in contact with Ms. Roush to retrieve our wedding photo CDs.

    Ms. Roush’s Contact information is as follows:

    Erica Roush, Photographer
    Captured Moments Studio
    Mailing Address:
    1080 Carlton Place #2D
    Frederick, Maryland 21703
    Contact #1: 1-240-529-8651
    Contact #2: 1-301-663-3837
    Email: capturedmomentsstudio@msn.com
    Website: http://www.capturedmomentsstudio.smugmug.com

    We prefer not to make this a legal matter, but we feel at this point we may have no other choice. All we want are our wedding photo CDs for which we paid $650.00 to Ms. Roush, with cancelled checks to prove it.

    Please – anything you can do to help us is greatly appreciated.


    Brenda and Jeffrey Brown
    Mechanicsburg, PA

    Erica –
    Please – if you could just let us know when you will be mailing our wedding CDs. As we have said all along – that is all we want. Please…

  3. March 10, 2007 at 12:28 pm

    Good point about S3 being a building block and an Amazon CDN being another building-block. Then you can choose what you want in your S3 storage available on the CDN, or use their CDN for content not on S3.

  4. March 11, 2007 at 4:31 pm

    We have been taken by Erica Roush – Captured Moments Studio. She took our pictures on 9/3/06 and we have yet to see our pictures. According to our contract, she was suppose to give us a disk, our prints both black and white and in color, printed. 2 -8×10’s of our choice. We have left numerous messages with her, no return calls, e-mails etc. There are many more brides that were taken by her. We are taking her to court on Friday, 3/17. Please do not allow her to continue to scam more brides. She is using other names.

    Donna Lucas
    Bel Air, MD

  5. March 23, 2007 at 1:55 pm

    What do you guys use to interact this S3’s API? Did you write an app to upload files into their system or did you find/purchase a product? So far any of the products I find you either get the performance without manageability or you get manageability but dog slow upload speeds.

  6. March 23, 2007 at 2:09 pm

    We wrote our own software layer that just reads & writes to S3 whenever we need to.

    It took less than a day to write (there’ve been tweaks and modifications since then) since the S3 API is so simple, and it’s very fast.

  7. March 27, 2007 at 9:42 am

    I won’t be at your talk at ETech (unfortunately!), but I wanted to know if in using S3, you’ve had any problems from users that know that their data is stored in a place that you don’t have any physical access to. I’m the throes of constructing a web service right now that relies heavily on S3 for storage of users’ personal data. We plan to encrypt everything before sending it over to S3 (because a privacy breach would really put us under), but personally, I’m still wondering how to pacify potential customers that we’ve already talked to who are concerned about our use of S3 instead of our own physically-accessible servers for storage. Any thoughts? Thanks in advance!

  8. jenifer meadows
    May 3, 2007 at 9:01 am

    erica roush of captured moments photogarphy is a con artist she prys on people on a very vulerable time she took my wedding photos oct 14 2006 and i have yet to recieve them i am in the process of taking legal action i suggest any one else with this problem does too

  9. Jacinta F.
    May 26, 2007 at 8:27 pm

    Jenifer I would like to speak with you about erica r. Please email me.

  1. No trackbacks yet.
Comments are closed.
%d bloggers like this: