Home > amazon, datacenter > Amazon announces SimpleDB (in Beta)

Amazon announces SimpleDB (in Beta)

December 14, 2007

Sweet! Amazon finally took the wraps off of SimpleDB. They’ve been working on this for awhile, and as you can probably tell, it’s a natural fit with S3 and EC2. There’s a great write-up about it over on inside looking out.

This is nearly a perfect solution for some of our data-related scaling challenges, except for two issues:

  • Physical proximity. Some of my datacenters aren’t close to Amazon’s, so the actual time to query SimpleDB is query time plus latency. This isn’t a problem if you’re doing all your queries from EC2, but we’re not there yet (we’d like to be, but a few pieces are missing. SimpleDB is one of those pieces, so we’re getting closer…). Amazon has promised me they’re workin the speed of light issue. 😉
  • Attribute size limits. We have some data fields that are longer than 1024 bytes (most aren’t and would work fine). We’ve thought about chunking the data up to get around this, which is a possibility, but it gets messy. Storing them in S3 is both overkill and probably too slow – if I need to get a few thousand photo captions *fast*, doing it through S3 isn’t optimal. If we could solve the latency problem I already mentioned, I’d be fine storing that specific data in some other store and working around it that way.

On the plus side, SimpleDB should be screaming fast, incredibly scalable, and almost all of our SQL queries would work with no changes other than syntax. Like many of you, I’m sure, we’re using much of our RDBMS as a fairly simple data store and aren’t using many advanced RDBMS capabilities. All of those queries could just use SimpleDB and then we could devote our DB iron to just the rare complex queries. We’re not alone – tons of web apps are gonna love this.

I’m thrilled to see the Amazon AWS stack continue to grow, and I’m shocked that they have as big of a lead as they do. I would have thought Microsoft / Google / Sun / whomever would have been out with some competition by now. It’s gonna happen – but I never would have guessed it would take this long.

Oh, and while I have your attention – SmugMug is now a fairly heavy user of EC2 and I have a write-up coming. So check back later if that’s of interest.

Categories: amazon, datacenter
  1. bd_
    December 15, 2007 at 11:37 pm

    It looks like you’d need to get the captions one at a time anyway – GetAttributes only works on one item, and Query only returns the item names.

  2. December 17, 2007 at 12:16 pm

    People are not looking at the whole picture though.

    The ‘inside looking out’ article says ‘Really large data sets’ are cool, but Amazon says ‘Amazon SimpleDB is designed to store relatively small amounts of data.’ As well, ‘store data in lexicographical ordered form’ means no native data types which will adversely affect query performance since everything is text. Which also means that having multivalued attributes is really saying ‘a comma-delimited data field.’

    Simple DB sounds like a decent offering; but it is not going to revolutionize databases. Directory services? Let’s hope so.

    For more ranting, I wrote about this a couple days ago: http://dba4life.blogspot.com/2007/12/amazons-simpledb.html

  3. December 20, 2007 at 9:36 am

    Hello Don!

    I can’t wait to hear your experience with EC2. I always enjoy reading your blog. I’m putting together some web services as well and I would love to learn from your experience with EC2.



  4. ana
    January 1, 2008 at 3:33 am

    Hi Dan

    Did you mean that you also pay for the Amazon DB?

    Why you don’t build your own DB for your organization?

    Where did you face the problem of latency?

    What is EC2 mean?

  5. Jorge Oliveira
    January 3, 2008 at 11:00 am

    I’m waiting for your article on EC2 🙂

    And since you are one of the biggest AWS customers, you could put some pressure on static ip, load balancing, CDN and persistant storage availability? 🙂

    Keep up the good work!!!

  6. November 15, 2009 at 10:02 am

    This was great last time, looking forward to it!

  1. December 14, 2007 at 7:52 pm
Comments are closed.
%d bloggers like this: