Home > amazon, business, smugmug, web 2.0 > Amazon S3: What would you like to know?

Amazon S3: What would you like to know?

February 2, 2007

As I mentioned in my article about performance issues with S3, I’m speaking on the subject at ETech this year. I’m planning on spending roughly half the time on the business ramifications and half on technical architecture. And I’ll be posting the slides or a PDF or something here after the presentation.

But I’d love some feedback about what you would like me to talk about so you can get the most out of my presentation and/or the information I put up here.

Leave a comment telling me what you’re most interested in about S3 and our implementation and I’ll re-prioritize based on your feedback.


UPDATE: Slides from ETech 2007 are up.

Categories: amazon, business, smugmug, web 2.0
  1. February 2, 2007 at 12:01 pm

    did you take a look at OmniDrive, Box.net, and others before choosing S3? What tipped the scale?

    Also, what does the contract with Amazon look like? Does it specify service levels in terms of tehnology, i.e., RAID, rpm’s, cache controllers? I’m guessing not, but would be interesting to learn how they’ve navigated around these requirements that are so common to the procurement process of many companies.

  2. February 2, 2007 at 12:05 pm


    Good questions. I’ll definitely try to include them in my presentation.

    The short answer is that Amazon is well-known in the industry for doing very solid systems and engineering work plus their API was so dead-simple and a perfect “building block” without all sorts of unnecessary stuff bolted on. Plus, the storage price can’t be beat. (The transfer cost can, though, and that’s something I’ll certainly be covering).

    There is certainly no SLA at this point whatsoever, and that’s obviously a big sticking point for a lot of corporations.

  3. February 2, 2007 at 4:05 pm

    I’ve submitted a paper to an academic conference with an evaluation of S3’s performance and availability. Drop me a note if you want a copy of the submission.

  4. February 2, 2007 at 4:36 pm

    You mentioned in your “Amazon S3: Outages, slowdowns, and problems” post the following:

    “we toggle[d] our primary endpoint to use the East Coast for awhile. This is the switch I mentioned earlier that I flipped, and it worked out beautifully.”

    I’d like to know more about how you accomplished this. I haven’t found anything in the S3 documentation that describes how to do it. My product could really use a capability like this one. Thanks!

  5. Sphinx
    February 2, 2007 at 4:51 pm

    You know, you still owe me a damned signed copy of Sin? I haven’t forgotten. I have to go to a wedding down in cali in august; I’ll hunt you if I have to.

  6. February 2, 2007 at 4:54 pm


    I have one right here for you. Might even be talked into a signed copy of Wages of Sin, too. 🙂

  7. Jake
    February 2, 2007 at 6:08 pm

    I’d love to know more about how you determine what to serve locally and what to serve out of Amazon S3, and what generalizations you can make from that.

  8. Anonymous
    February 2, 2007 at 10:39 pm

    Are you getting a volume discount from Amazon?

    If another company were to offer a similar service with an S3-compatible API, perhaps positioned as being complimentary to S3 (extra redundancy, extra location(s), different peering relationships), how interested would you be?

    Sorry for posting anonymously, but this is a potential future offering from my employer and they don’t like it when I post about unannounced products.

  9. February 2, 2007 at 10:42 pm


    I’m not getting a volume discount, or any special treatment, from Amazon. I’m billed the same rate everyone else is, straight through the AWS billing system.

    If another company were to offer a similar service, I’d be extremely interested. Assuming the price and performance were comparable or better, it’d probably be a no brainer for us to use this as well.

    A few other large companies may or may not have talked to me about things like this already. 🙂

  10. Julian
    February 3, 2007 at 8:35 am

    What I wanna know is if everything that is on S3 always has to be transferred to your servers and is then send to the users or if one happens to load images directly from amazon servers. And if it’s the latter, than how do you hide that there’s amazon behind.
    A direct answer here would be nice.

  11. February 3, 2007 at 10:46 am

    I was thinking something realted to Julian’s question… clearly you’re pulling the data from S3 to SmugMug, then pushing it back out to the viewer… but why not have S3 push it directly to the viewer? It would seem to be a straight forward thing to toggle in the html generation, could even get fancy with a bit of js to figure out which of the S3 (and smugmug) locations is fastest for a given viewer to pull from. Was it purely a brand preservation thing? that you want smugmug.com in the url?

  12. Peter
    February 6, 2007 at 3:32 am

    Hi Don,

    Are you using the REST or SOAP interface (or both)?

    If both, which one do you find faster and which more reliable (if available)?

    If one particular interface, how did you decide? Any information would be great.

    Have you found latency effects the way you consume the service (compared to local storage)? (I know its a vague question, but being in Australia Amazon S3 is not as luring when there is a 220ms+ overhead attached– damn you physics!)

    Do you employ any smarts to guess where and what your clients are going to do in regards to prefetching/caching of S3 data?

    How do you handle a failure when fetching non-cached S3 data? (this is after trying more than once and at different data centers etc.)

    I believe you mentioned in a previous article/post that you are planning on peering (directly) with Amazon to reduce bandwidth costs. Assuming this is true, more questions!

    On peering; Are Amazon going to reduce or eliminate the bandwidth cost component of S3 (as seen with EC2)? Has this been discussed?

    On peering; Are you planning on peering directly with Amazon in their data center or at the closest peering point (an internet exchange for example)? Some other location?

    On peering; Will you be peering at each of Amazon’s significant S3 data center locations?

    On peering; Will you pay anything, to Amazon, for peer directly with them?

    On peering; Are you currently peered with Amazon? (Had to ask!)

    On peering; Would I be correct in assuming that the cost and usage benefit of S3 has been significant enough (both current and predicted) to warrant the extra overhead of a peering agreement with Amazon?

    Unfortunately I won’t be attending ETech so if you could include some of the answers as a reply or in the slides I would be appreciative. Thanks!

  13. Sphinx
    February 6, 2007 at 11:54 am

    You tease! Will you mail them!? I’ll be here imagining a what a universe finally balanced in it’s principles will be like…

  14. March 2, 2008 at 2:33 am

    est il availible en Francais, my English not good

  1. February 2, 2007 at 11:21 am
  2. March 22, 2007 at 7:37 pm
Comments are closed.
%d bloggers like this: