Lessons from the Ritz-Carlton
At SmugMug, we pride ourselves on our customer service. We think it’s one of the key things that sets us apart from our competitors and we spend a lot of time focusing on it. Every SmugMug employee works a customer service shift every week. Think about it – how many web properties can you think of that provide over-the-top customer service?
I was lucky enough to stay at the Ritz-Carlton Half Moon Bay over the weekend for my 6th wedding anniversary (and first overnighter away from the twins!). I had heard of the Ritz’s reputation for service, of course, but I’ve stayed at nice hotels before. Just how great could it be?
Boy, was I in for a wakeup call.
A few days prior to our arrival, I received a phone call from the Ritz Guest Relations Supervisor wondering if there was anything special they could do for me since it was our anniversary. I hemmed and hawed for a minute, thinking, and she took the opportunity to suggest a few wonderful things such as having ‘our song‘ playing when we got back to our room after dinner. Together, we came up with a few more great ideas and she made it happen – after dinner, my wife was in tears and I had earned major bonus points.
When we arrived and I pulled up to the guardhouse down the road from the hotel, the gentleman manning it wished us a “Happy Anniversary”. Before we’d even driven up or checked in, everyone already knew! The valets knew, the doorman knew, the front desk knew. Everyone wished us a “Happy Anniversary” and even correctly pronounced my last name (it’s difficult for most people, and we’re very used to a variety of pronunciations – but clearly, they knew at the Ritz).
The valets gushed over my car when I pulled up, asking if it was turbo-charged (they could hear the whine – but it’s a supercharger), complimenting me on my rims, etc. Ok, so I’m car-obsessed and a sucker for people realizing that my car is unique and wanting to talk about it. Now, remember, this is the Ritz – there was a Bentley next to a Rolls-Royce next to a Ferrari 360 next to a Laborghini Murcielago out front. I love my car, but it pales in comparison – but they knew it was my car and that I’d appreciate the compliments.
I could go on-and-on, but I think you’re getting the point. From service at the spa (best. massages. ever.) to dining, everything was on par with what I’ve already described. The next day, Ritz employees were still greeting us in the halls by our name and wishing us “Happy Anniversary”.
The bottom line: We felt special. We felt pampered. We felt like the Ladies and Gentlemen of the Ritz-Carlton knew us personally and really cared about making sure we were happy. They’ve earned a customer for life.
Things we can learn from the Ritz and their approach to service:
- Attention to detail. Knowing how to pronounce my last name properly doesn’t sound like a big deal – but it is to me, and it’s but one small example. Everything was like that – no detail overlooked.
- Proactive service. Calling me a few days ahead of time, prepared with suggestions, was over the top and greatly appreciated.
- Warmth and caring. Every Lady and Gentleman at the Ritz seemed to genuinely care about us and our well-being.
- Communication. Everyone there seemed to know it was our anniversary, and that put a smile on our faces each and every time we heard “Happy Anniversary”.
- Understanding the customer. When I drove up in my car, they realized that I was into cars and reacted accordingly. Adjusting your game to be “just right” for every given customer is a diffcult but laudable goal.
- Every customer is special. I’m not a regular, I’ve never stayed at a Ritz, and I wasn’t well-dressed or anything – I was wearing my typical cheap, comfy Old Navy from head-to-toe. It didn’t matter – at the Ritz, everyone’s special.
We need to take our game to the next level. We provide excellent customer service, possibly the best on the web, but that’s no reason not to improve – and what better place to learn than from watching the gold standard.
Scoble @ The Ritz
My wife and I celebrated our 6th wedding anniversary (and first overnighter away from the twins!) last weekend. During lunch on Saturday, I spied a Google hat across the room and recognized the face beneath – Robert Scoble. I don’t know how we kept passing in the night at tech conferences, but we’d never met. I ambled on over and introduced myself and met Jeremy Wright as well.
You never know what to expect when you meet high-profile geek bloggers (I’ve been disappointed by more than a few), but I gotta tell you, Scoble is The Real Deal – a truly nice guy. Nice guys are rarer than they should be in the Valley, but Scoble was warm, friendly, funny and personable. Later that afternoon, while my wife caught some Zs, we had fun filming the inaurugal Mojito Show. I’m hoping I can make a repeat appearance this Saturday, but we’ll see.
Jeremy was fun, too, and sounds like he’s built an interesting business around blogging (and did it fast – only 11 months) with b5media. Oh, and he’s got a great business card with plenty of attitude.
Best part of the interview? Scoble said something like “Wait, you’re profitable? You can’t be Web 2.0!”. 🙂
Sun's response
Looks like I may have kicked over an anthill over at Sun – a bunch of Sun employees have contacted me. Sun’s response to my review has been very impressive – they’ve all been very polite, responsive, and anxious to get their hands dirty and see exactly what we’re doing. The people I’ve spoken to so far have been hardcore and extremely knowledgeable, so I expect some great results.
I’m gathering some additional data they asked for on Ubuntu Linux, and then we’re gonna get together at their Menlo Park offices next week and do some profiling to see both where the bottlenecks are and if there’s any other tweaking we need to do.
Linux is clearly my #1 target, since I’d rather not swich OSes unless there’s something super-compelling about Solaris (I remain open to the idea that there may be, but am skeptical), so we’ll be tackling that first. But the really good news for all the commenters is that we’ll be sticking another T1000 in one of our datacenters and try profiling Solaris on T1000 side-by-side with Ubuntu on T1000.
This should get interesting. 🙂
Phantom finally dies! Long live the Lapboard!
Shacknews reported that the Lapboard is finally for sale (and that the Phantom “console” is finally dead).
About time. It’s been obvious for years that the only thing worth having is the Lapboard and that the Phantom would suck, if it ever shipped.
Adding Solaris to the mix
So my Sun T1000 review got dugg, and commented on, and there’s one loud-and-clear message: people would like to see Solaris results.
So would I. But as I outlined in the review, I don’t have any Solaris expertise. I am a busy guy. 🙂
(I should re-iterate that Jonathan Schwartz asked for everyone to review the T1000 with Ubuntu, which is exactly what I did.)
So if anyone at Sun would like to spend a few hours with us and help us get this box configured for the same test on Solaris, I’d love to see what it could do and post a follow-up.
Any takers?
Sun Fire 'CoolThreads' T1000 review
Ever since they first announced the Niagara processors at Sun, I’ve been excited. Could Niagara change my business? Who wouldn’t want tons of physical cores coupled with tons of virtual cores? At every tech conference I’ve tried to get hard data from the people manning Sun’s booths. At MySQL User’s Conference they were hyping MySQL performance, for example – yet there’s a huge MySQL bug where performance degrades with more CPUs, so that’s clearly not a great target for us (yet).
Nonetheless, the geek in me remained intrigued – I’ve believed for years that scaling # of CPUs, rather than purely speed of CPUs, was the future. One of the great parts of my job is that I get to play around with new toys and new technology, like Amazon’s S3 and Niagara, that can enhance our business or change it in some way. And every geek wants to dream that there’s some hot new CPU around the corner that’ll solve all their problems, right?
Sun has a great 60-day Try & Buy program. They make it basically as painless as clicking on the server you want, and a few days later it arrives. Very cool. Unfortunately, I haven’t used Sun gear since 1994, when I was using SunOS 4 (remember when SunOS was BSD-based?), so it would likely be time-consuming to try out both new hardware and new software. No thanks, I’m a busy guy.
Enter Jonathan Schwartz and his famous blog. Jonathan probably doesn’t remember me, but when I was 12 years old, I’d haunt the halls at NeXT every second I got and crashed NeXTWorld every year. I remember him. He was NeXT’s most important developer, and my father got the thankless task of buffering Steve Jobs and Jonathan. Both of them needed the other, but they couldn’t stand each other. Fun fun. 🙂
I’ve been meaning to touch base with Jonathan and see how he’s doing at his new job – and to see if a small web company like ours can shed any light on Sun’s direction. I think he’s got a very tough endeavor ahead of him – he’s gotta turn a massive company with lots of inertia around to compete in a whole new ballgame. For more than a decade now, datacenter computing has been shifting more and more rapidly towards free operating systems coupled with commodity hardware, and Sun nearly missed the boat. Now they’re scrambling to catch up. I believe Jonthan “gets it”, but we’ll have to see if he has the time and energy to really make the shift.
On June 16th, Jonathan posted a blog entry where he announced that Ubuntu Linux ran on Niagara, and that anyone who writes a thorough review would get to keep the box in question. Fantastic idea – I get to run Linux, which I know like the back of my hand, play with some hot new technology, and I get to keep the hardware for my time. Sold! So here we are, 60 days later, with a thorough review.
UPDATE: Jonathan has a new blog entry this morning about Niagara’s power savings. Pretty cool that you can get a rebate for using lower-power servers – but it doesn’t materially impact the conclusion of this review.
UPDATE #2: The comments here and on digg are pretty clear – you’d like to see Solaris results. Me too. Here’s an open call for help from Sun.
digg_url = ‘http://digg.com/hardware/Amazing_comparison_of_a_32_core_Sun_running_Ubuntu_versus_4_core_AMD’;
Amazon S3 = The Holy Grail
I should have posted this a few weeks ago, but better late than never. We now use Amazon S3 for a significant part of our storage solution. We’re absolutely in love with it – and our customers are too (even if they don’t know it).
As you probably know, SmugMug has been profitable since our first year, with no investment capital. We’ve had a great track record for keeping our customers’ priceless photos safe and secure using only the profits we’ve accrued to purchase our storage (yes, I said purchase. We have no debt – we own all of our storage, we don’t lease). And every SmugMug customer gets unlimited storage – so that’s no mean feat. (Currently, unlimited means ~300TB of storage and nearly 500,000,000 images. To put that into perspective, that’s more than 65,000 DVDs or 480,000 CDs).
But Amazon’s S3 takes our storage architecture to the next level:
- Your priceless photos are stored in multiple datacenters, in multiple states, and at multiple companies. They’re orders of magnitude more safe and secure.
- We’d already built a custom, low-cost commodity-hardware redundant scalable storage infrastructure. Nonetheless, it’s significantly cheaper to use S3 than using our own – especially when you factor multiple states & datacenters into the equation.
- Perhaps even more importantly, our cash-flow situation is vastly improved. Instead of paying $25,000 for a handful of terabytes of redundant storage up-front, even before they’re used, we now pay $0.15/GB/month as we use it.
- When we have some sort of internal outage with storage, it doesn’t matter – Amazon’s always on. They eat their own dogfood – S3 is in production use on dozens of Amazon products. We’ve had storage-related internal outages a few times already, and our customers haven’t been able to tell. We’ll still have rare outages on our site, unfortunately, (everyone does), but storage is now vastly less likely to be part of the cause.
- I started writing our S3 interface on a Monday, and by that Friday, we were live and in production. It really is that simple to pick up and use, and it was basically a drop-in addition to our existing storage.
- It’s fast. I don’t mean 15K-SCSI-RAID0-fast, but I do mean internet-latency-fast. It’s basically as fast as our internal local storage + the roundtrip speed of light to Amazon. I can measure the difference with computer timing, but in blind tests, humans haven’t been able to tell the difference. Everything we serve from Amazon feels fast.
I hate to admit this, but Amazon has built a playing-field leveler. It’s now much much easier for a competitor of ours to spring fully-formed from two guys in a garage than it was. Anyone who doesn’t get on board with Amazon S3 (or the inevitable S3 competitors) may get left behind. I’m glad we’re first, but I doubt it’ll last.
Tim O’Reilly, technology visionary extraordinaire, recently said of Sun’s new ‘Thumper’, the Sun Fire X4500: “This is the Web 2.0 server.” While I think Tim has perhaps the clearest vision in the industry, and the Thumper does truly look awesome, this time I think he may have missed the mark. The Web 2.0 server is *any* cheap Linux box coupled with utility storage like S3.
Initially this post had a lot of technical detail (I am the ‘Chief Geek’, afterall), but I removed it since it was probably getting boring. So this is the quick-and-dirty ‘Business Case for Amazon S3 and How it Helps our Customers’ post. If there’s enough interest, I can write up a detailed post about exactly how we use S3, how it works in conjunction with our own local distributed filesystem, and post our S3 library (which was derived from someone else’s). Post in the comments if that’s of interest.
Also, we’ll be presenting at a storage conference in Florida in late October (I’m sorry, I don’t have the name of the con with me, but I’ll update this post when I do), and have had a few other people request conferences talks on the subject. Comment if that’s of interest, too, so we know where to go speak.
Finally, one last geek thought: Anyone using the SmugMug API is now actually using multiple APIs through ours (depending on what you’re doing, you may be using Google and/or Yahoo, but you’re almost certainly using Amazon). The stack continues to grow.
UPDATE #1: In response to a comment below, I don’t feel like we “bet the company” on S3 – every photo our customers entrust us with, we keep local copies in our existing distributed storage infrastructure. We use S3 as redundant secondary storage for use in cases of outages, data loss, or other catastrophe.
Phil Askey has a point
While I’m still upset about the tone and method of dpreview’s message to us, I have to admit, after thinking about it a little bit and exchanging some private emails with Phil, he has a point.
We’re a ‘no ads, no spam‘ company, and would hate for our brand to be associated with any sort of spammy tactics. On the other hand, we feel like people should be able to recommend products they enjoy. I love the Canon 30D, for example, and I’m not afraid to tell everyone I meet. But there’s a line, and it looks like some of our customers may have crossed it. (Why on earth a few customers crossing the line results in the banning of an entire photo community, I don’t know, though.)
To that end, we’ve added some language to our referral information in our users’s Control Panel, and in the referrals help section on SmugMug. Specifically, the text reads: “Please: As you know, SmugMug hates spam, as we’re sure you do. We know you’re thrilled with SmugMug, but please refrain from using your coupon code in a pushy way that could be construed as spam. Thanks!”. I think that about sums it up, and it helps us and our customers be better Internet citizens.
To be clear: We never intended for our referral program to be some ‘free ride’ on forums or anything. The original intent was for friends and families to hook each other up. We’re happy to pay for advertising on message forums and websites.
I guess we’re just cursed with zealot customers. Oh darn. 🙂
Between a rock and a hard place with dpreview
One of my all-time favorite sites on the net, dpreview, home to all worthy camera news, isn’t happy with us. And I’m not sure what to do about it. Here’s the deal, and I’m hoping maybe someone has some ideas on what we can do:
SmugMug customers are foaming-at-the-mouth rabid zealots. Ever been cornered at a party by a TiVo subscriber who wouldn’t let you leave until you’d promised to buy one? Yeah, they’re like that. Four years ago, when we realized what was going on, we thought it’d be great to have a referral program that lets them get discounts at SmugMug and save their friends a few bucks. Not all that different from any other customer referral program, online or off, in the world. (We do give bennies to both parties, though, which isn’t unique but is fairly rare).
It took off like gangbusters and everyone got warm-and-fuzzies because they were saving themselves andtheir friends money. So they started posting their referral codes in blogs, email signatures, and forum posts. No biggie, right?
Apparently to dpreview, it IS a biggy. Thousands of SmugMug customers are also dpreview forum members. And now Phil Askey, dpreview owner, dropped a bomb in our laps: all SmugMug links in the forums will be blocked. Why? Because he’s upset at our ‘viral marketing’ technique (aka referral program). Here’s the message we got (with no warning whatsoever):
Just to let you know that we will be blocking linking to smugmug from our forums due to the rapid increase in your ‘viral marketing’ technique of using ‘account codes’ for discounts. Numerous of our regular posters are now inadvertently promoting your site by placing such account codes in their signatures, this is considered to be commercial advertising and is against our posting rules. As we have no interest in banning such members we will instead be blocking any external linking or mention of your site.
As I mentioned, we have thousands of shared customers (likely tens of thousands, but I don’t know for sure), so with the flip of a switch, Phil will be pissing off a large portion of his user base. Lots of camera reviews in the forums come from SmugMug customers, complete with both links to their galleries and embedded links to images directly on SmugMug. They’ll all disappear in a heartbeat. If I were a non-SmugMug customer at dpreview, I’d be wondering “What’s to stop him from doing this for Flickr or Pbase or anyone else he chooses in the future?”.
I’d be tempted to say that it’s because we don’t pay him to advertise at dpreview, but the sad truth is we’ve tried numerous times to buy ads there. Phil simply ignored every single one of our requests.
So I have no idea what the motivation here is. You see paid plants for other photo sharing sites (something we’ve NEVER done) running amok over there, posting like crazy with no useful information other than a link to “the GREATEST photo sharing site in the world! Way better than [Flickr|Pbase|SmugMug]”.
I’d really like to be fair here. We owe our customers at least some effort to rectify the situation before their ability to use dpreview disappears. So, help me out here. Are we doing something wrong? Is our referral program somehow harming dpreview and/or the internet as a whole? Is there something we could do to change the program that would make things better?
Are are we just stuck?
Here’s a thread on dpreview [NOW DELETED. New thread, from which I’ve been banned.] to discuss the topic. And here’s one on our sister forum, dgrin.
Have an API Key, Zooomr
There’s some hooplah over Flickr not letting Zooomr have an API key. Just to be clear, as if my earlier post on the ‘lock-in’ subject weren’t already clear enough, we’re happy to play nice with our competitors. Grab a SmugMug API Key, make it easy to migrate your photos – we’re thrilled.
We think competition is good, lock-in is bad, and that the best company should win. We all do things a little (or more than a little, depending) differently than each other. Let the customer choose.
SmugMug's Don MacAskill 