Home > business, datacenter, web 2.0 > Thoughts on Google App Engine

Thoughts on Google App Engine

April 8, 2008

First:  Very cool.

Next:  I think it’s interesting that Google has basically taken a sniper scope out and aimed it at a specific cloud computing target.  App Engine is only for web applications.  No batch computing, no cron jobs, no CPU/disk/network access, etc.  

I think this is very smart of Google.  Rather than attacking Amazon head-on, Google has realized there’s a huge playing field for cloud computing, and are attempting to dominate another portion of it, one where they have a lot of expertise.  Very good business move, imho.

Will we use it?  I wouldn’t be surprised.  I’ve long thought that we’ll continue to mix in web services from a variety of providers, and it looks like App Engine can solve a slice of our datacenter need that other providers don’t yet provide.  

I’m more than a little concerned, though, by how much vendor lock-in there is with App Engine.  At first glance, it doesn’t look like the apps will be portable at all.  If I want to switch providers, or add in other providers so I’m not relying solely on Google, I’m outta luck.  

I’m hopeful other languages get supported, too.  I think Python is great – don’t get me wrong – but we have a lot more experience with other languages, so there’ll be a learning curve.

Finally, I’m dying to find out what pricing for an application of our scale will look like.  I can see some immediate, obvious things I’d like to try to do on App Engine, but the beta limits aren’t gonna cut it for us.  😦

Will it replace Amazon?  It sure doesn’t look like it from where I sit.  In fact, I don’t see this as much of a competitor to Amazon Web Services.  There’s some overlap in some small area (hosted web apps on EC2), but I doubt that’s the bulk of Amazon’s business.  As I said, we’ll likely end up using both (and other providers as they come along, too).

My favorite bit?  In theory, Google has solved the data scaling problem.  I don’t mean raw binary (blob) storage, which S3, SmugFS, MogileFS, and plenty of other things have solved, but the “database” scaling problem.  Every popular web app runs into this problem, and it’s typically solved with a combination of memcached, federation, and replication.  But it’s messy.  In theory, Google has automated that piece for us.  I can’t wait to play with it and see if that’s true.

I also can’t wait to see who else is going to wade into this fray.  Sun?  Microsoft?  Yahoo?  IBM?  

Bring it on!

  1. April 8, 2008 at 11:12 am

    portability is a big issue and Python is a non-starter for most existing apps. agree that the database model could be the big win here. will be fun to watch.

  2. April 8, 2008 at 11:21 am

    I agree with many of your thoughts about the use of GAE with AWS. They don’t seem to have a very large overlapping area, which is great for both. They can also work really well together. There’s no reason why a GAE app couldn’t use GQL with S3, for instance. That’ll largely depend on the pricing structure, though.

    And yes, IBM will likely get in to this arena. Look up IBM’s Project Kittyhawk. If GAE hits individuals and small businesses, with AWS hitting small to medium businesses, Kittyhawk could likely be the large business and enterprise level solution in grand mainframe style.

  3. Luke Baker
    April 8, 2008 at 11:45 am

    This seems like it’ll give a good kick in the pants to the BigTable clones out there (Hbase and Hypertable). Of the APIs they offer this seems like the hardest thing to duplicate. I expect it’ll be only a matter of time before folks have put together the pieces to allow application portability. While this application portability would require more sysadmin know-how than App Engine requires, I think it’ll still be quite sufficient for the people concerned about it.

  4. April 8, 2008 at 12:10 pm

    The portability question is a big one – it looks like you can get some degree of portability by sticking with Django: http://code.google.com/appengine/articles/django.html

    One of my first projects is going to be testing this – I’m a lot more comfortable restricting myself to running on a Django host.

  5. April 8, 2008 at 12:18 pm

    SQL Server allows one to create horizontal and verticle “views” of tables. For example, for one state only(horizontal) or a couple columns only (vertical). Using the Enterprise Edition (mucho bucho) you can index these views also. I’m no expert. Oracle & DB2 probably can do this feature that facilitates scalability.

  6. April 8, 2008 at 2:21 pm

    Right there with you on the lock-in. However, as a web site development experience, it brings back a developer experience that I have missed since the late nineties.

  7. scott
    April 8, 2008 at 5:06 pm

    Cron jobs are easy with AppEngine. Here’s the recipe:

    1. Create the job you want to have done at a regular interval and give it an http-based interface.
    2. Create an EC2 instance with a cron job that calls you AppEngine HTTP interface at your desired interval! 🙂

    On a more serious note, I think that it’s best to look at AppEngine as a first step. I think it’s a great starting point because they’ve chosen to launch functionality that can inspire entrepreneurs. I am virtually certain that batch processing, mapreduce, cron jobs, etc. will happen eventually.

    Myself? I would *gladly* give up whatever programming language I happen to be using at any specific time if database scaling complexity where to disappear (from my view). I’ve been spending the day asking myself “Do I really need ‘joins’?” — it’s always a good exercise to question assumptions.


  8. April 8, 2008 at 5:29 pm

    The portability problem is definitely an issue.

    However, it’s interesting to note that, aside from live authentication, everything can run locally with the SDK you download. I haven’t tested the performance of the local implementation, but there isn’t anything to stop someone from dropping that on to a server and using it. No, it’s not scalable and it probably won’t perform as well, but it might just get the job done. And it might just provide the test bed for a third party to create a clone of the APIs.

    @scott: I totally agree with your statement about database scaling. This platform also seems to be far simpler to port over a SQL solution than to port one to Amazon’s SimpleDB (at least, without having tried either with a large schema, that’s my view).

  9. Herbert
    April 8, 2008 at 6:28 pm

    Portability is not really a problem. It’s the kind of thing people have been papering over for years with postgresql and mysql. Someone will create a compatibility wrapper.

    Google and Amazon: you should peer your data centers and provide free traffic to customers of both. There really is very little overlap between the two services, and it would really help to kick-start new websites, which would be in the interests of both companies, to enable folks to mix ‘n match between the two.

  10. April 8, 2008 at 9:34 pm

    Hmmm. On one level, there is no lock in because they have open sourced the apis. If another provider wants to build something that is GAE compliant, it would be reasonably easy to do. The trick, as always, would be the underlying tech – GFS / BigTable.

    Could this be built out on top of Hadoop? Probably. Using Jython, and some magic, I can see how to get most of the way there.

    The real change this is going to bring is bringing home the reality that building large dynamic sites doesn’t need a honkin SQL server. And this is a good thing.

  11. Tim
    April 8, 2008 at 10:26 pm

    Does Google promise any latency #’s or CPU resources.

    What I mean is, is Google promising the service will be “fast”?

  12. Alex
    April 9, 2008 at 5:31 am

    For those of us who like Python this seems ideal.

    I’ve been watching Amazon EC2 for ages, although it seems a good service it’s just that little bit beyond most developers and small time users. If I wanted to host a small application on EC2 then it’s going to cost me at least $75/month just to keep it running, as well as the additional costs for bandwidth and S3 usage. Even with EC2 I have to code my application to run on Amazons platform given that there’s no permanent disk storage apart from S3, the only permanent database available is SimpleDB.

    Google has got it right by offering a basic version for free. Developers won’t have to sign up with their credit cards to create small applications. I could create a basic site on there for free, if it gets popular then I’ll probably end up paying (and can probably pay for it at that point).

  13. April 11, 2008 at 7:42 am

    “I’m hopeful other languages get supported, too”

    I’m trying to get php support on app engine 🙂


    And while I’m waiting, I’m learning something about python…

  14. sam
    April 13, 2008 at 2:29 pm

    I’m amazed that the open source dbs have not tackled the problem head-on with an oracle rac killer… The db scale problem simply does not exist with oracle and db2 – just attach another box when load increases. Shared storage is becoming an open source commodity but the dbs are waaaay behind the curve in using it effectively.

  15. Steven Roussey
    May 2, 2008 at 12:15 pm


  16. Joshink
    May 18, 2008 at 3:18 pm

    I’ve been working with AppEngine for about a week, and I can tell you that it is an awesome platform. I’m using Django because that what I normally develop in anyway, and in many ways, there ORM is way better than Django’s. You don’t have run syncdb or anything, and the Expando model is unbelievable.

    The fact that you can code and not have to worry about the scaling is such a relief. You simply build and test locally then publish. It’s seriously that simple. Python really shines when using AppEngine and I’m very glad it was the first language they supported. It will be interesting to see what languages they pick next or if they just continue to develop the Python API.

  17. December 3, 2008 at 2:13 am

    I’ ve had occasion to try out taksi, it worked well for GDI capture, but for Direct3D capture on the engine I used it failed in CTaksiDX9:: GetFrame during GetRenderTargetData. I’ ve found a solution by disabling the avi feature (I didn’ t need it) and using screen capture through the texture api with a direct surface to file save- I used D3DXSaveSurfaceToFile. GetRenderTargetData failed with INVALIDCALL- I didn’ t investigate further, but your comments and the msdn documentation suggest it could happen due…

  18. November 6, 2009 at 4:00 pm

    Love it! You got me so excited to get one and start shooting video!

  1. April 8, 2008 at 11:22 am
  2. April 8, 2008 at 12:09 pm
  3. April 8, 2008 at 6:47 pm
  4. April 9, 2008 at 7:19 am
  5. October 4, 2008 at 3:49 am
Comments are closed.
%d bloggers like this: