December 4, 2009
An open letter to Net24 – sorry if it makes you wince but I really hope (for your sake) that you think about the issues I raised..
So I’m pretty pissed off – so sorry if this email gets a bit aggro.
We host our website with you – it’s on a VPS, something I’m not overly keen on (you see despite being a technology commentator, I’m not, and don’t want to be, a sys admin) so you sell it as an “empty container” and don’t support it. That’s fine when it works, when it doesn’t though, it sucks.
Over the time that we’ve had the VPS, there have been quite a few outages. Annoyingly they’ve been out of office hours and, despite a totally misleading name, you don’t actually have staff around 24/7. I can call an on-call engineer who only wants to hear if there is a new outage – otherwise he’ll just run through his schedule fixing things that go down.
Yesterday we sent out one of our regular newsletters to around 5000 recipients. Given Murphy’s law it was obvious that the VPS would go down just as we were sending a campaign. Luckily this time it went down an hour or two before and I checked the site before hitting “send”.
I called the engineer who told me that you’d had a hardware outage and he was going through a list of affected servers and copying them to a new node one at a time and that he’d get to us “sometime soon”. I told him that wasn’t good enough and he agreed to bump us up the list – 15 or so minutes or later we were finally up. Yay! I hit send and watched around 5000 emails hit cyberspace…
And yup – ten minutes later a buddy tweeted me to say that our site was down. It seemed that in moving the VPS as a whole (remember when you said that when moving a VPS you do it in its entirety and nothing can go wrong?) somehow a MySQL database got screwy and, whatever the propeller head reason, we lost our site.
So I gave you a call and at first you did the “no, it can’t have been us. you need to get your developer to go through the logs to find the problem”. I, of course, jumped up and down and got some extra help and you fixed the problem (thanks!) and we were back up and running. I sent an email suggesting that maybe, just maybe, it might be an idea to inform people proactively when their servers were down and not just leave them to find it out for themselves – that’s something you’re thinking about I guess.
So then this morning… our customer service rep told us that she’d not been getting emails from the e-commerce app sitting on our server. This is kind of important as the emails carry half of the credit card detail (we’re concerned about our customers security so half goes via email and half is through the app control panel) and we couldn’t process orders. I called up and was told that “moving a VPS you do it in its entirety and nothing can go wrong” and that VPS are unsupported and we should get our developer to go through the logs to find out what the problem was.
What? So I have to pay a developer to find out exactly what the problem is when you caused it through a hardware failure? Whatever happened to the consumer guarantees act? Fitness for purpose? So after jumping up and down (good exercise I guess, but bad for my blood pressure) I pretty much forced you to look into it yourselves – eventually you did and you resolved the issue, but like I said – only after some serious prodding from me. At the time of posting this letter our email is working, you’ve agreed that “there was an issue” (something to do with file permissions) and we’re up and running.
So what gives guys? I could have spent a few hundred dollars with a developer fixing your fault. I could have had a website that was out for several hours with no proactive advice. I could have lost significant sales while our email was mysteriously disappearing. Twice in 24 hours you swore black and blue that the separate issues we experienced were not in any way caused by you – both times your contention was proved wrong.
You see it’s all about putting things right.. you’ve got a Twitter account now, that’s great. But without a culture that is actually about making sure customers expectations are at a very minimum met, and, ideally, exceeded you’re just chirping into a great big void.
I hope some things change.