Archive
Delta Airlines has terrible customer service
photo by: Local Sports Online
My poor little sister just got totally destroyed by awful Delta employees. She was trying to fly cross-country for the first time with her 4 month old baby. Disaster ensued, and Delta totally blew the customer service. Money quote?
“I asked [the supervisor] if she cared that I had been treated poorly by Delta employees, and she said if I wanted someone to care, I should write to “corporate” and they were required by law to respond to me in 30 days or less.”
My sister doesn’t fly as often as I do, clearly, or she would have known how terrible flying has gotten these days. She would have done online check-in, and would have arrived earlier. She made some mistakes, and seemed willing to accept the consequences. But that’s no excuse for downright rude customer service. Would an “I’m sorry” or a smile have been so hard? Shame on Delta.
Anne, for the record, flying with Virgin America has been an absolute dream, particularly when paired with Clear. (Want a free month with Clear? Use code SCA04501). I wish both were everywhere – maybe someday. In the meantime, avoid Delta like the plague.
UPDATE: I should have known Consumerist would have lots of stories like this. Enjoy!
MySQL and the Linux swap problem
Ever since Peter over at Percona wrote about MySQL and swap, I’ve been meaning to write this post. But after I saw Dathan Pattishall’s post on the subject, I knew I’d better actually do it. 🙂
There’s a nasty problem with Linux 2.6 even when you have a ton of RAM. No matter what you do, including setting /proc/sys/vm/swappiness = 0, your OS is going to prefer swapping stuff out rather than freeing up system cache. On a single-use machine, where the application is better at utilizing RAM than the system is, this is incredibly stupid. Our MySQL boxes are a perfect example – they run only MySQL and we want InnoDB to have a lot of RAM (32-64GB … and we’re testing 128GB).
You can’t just not have any swap partitions, though, or kswapd will literally dominate one of your CPU cores doing who-knows-what. But you can’t have it swapping to disk, or your performance goes into the toilet. So what to do?
Our solution is to make swap partitions out of RAM disks. Yes, I realize how insane that sounds, but the Linux kernel’s insanity drove us to it. Best part? It works. Here’s how:
mkdir /mnt/ram0
mkfs.ext3 -m 0 /dev/ram0
mount /dev/ram0 /mnt/ram0
dd bs=1024 count=14634 if=/dev/zero of=/mnt/ram0/swapfile
mkswap /mnt/ram0/swapfile
swapon /mnt/ram0/swapfile
That’ll give you a 14MB swap partition that’s actually in RAM, so it’s super-fast. This assumes your kernel is creating 16MB ramdisk partitions, but you can adjust your kernel paramenters and/or the ‘dd’ line above to suit whatever size you want.
We’ve found that anywhere from 20MB-40MB tends to be enough (so use /dev/ram1, /dev/ram2, etc), depending on load of the box. kswapd no longer uses any noticeable CPU, there’s always a few MB of free “swap”, and life is back in the fast lane. Just add those lines to your relevant startup file, like /etc/rc.d/rc.local, and it’ll persist after reboots.
Some Linux purists will probably hate this approach, others may have more efficient ways of achieving the same thing, but this works for us. Give it a shot. 🙂
Oh, and I hope it goes without saying, but make *darn* sure you know what you’re running on your box and what the maximum RAM footprint will be before you try running with only 20-40MB of swap. We’ve never OOMed (Out-Of-Memory) a production MySQL box – but that’s because we’re careful.
UPDATE: See what happens when I wait to blog? I forget that I read another related post over on Kevin Burton’s blog. Like Kevin, we’re using O_DIRECT, but unlike Kevin, this doesn’t solve the problem for us. Linux still swaps. We use the latest 2.6.18-53.1.14.el5 kernel from CentOS 5, btw. (Sorry, had posted 2.6.9 because I was dumb. We’re fully patched)
SmugMug's Don MacAskill 