Why is Backup so Hard?
I’ve been thinking about a question for the past few months that I’m sure many of you have wondered about while sweating over the restore of a mailbox for a director or a lost line-of-business virtual machine. Why is backup and restore so difficult?
Thinking Out Loud
You might be able to guess from the nature of some of my recent posts that I’ve been doing a lot of work with Azure backup and disaster recovery solutions over the last 18 months. I’ve delivered a lot of presentations on the subjects to technical and sales staffs, and I’ve been challenging them to consider new ways to implement old requirements. Some see the need for change, others bring with them what I call “same old IT.”
I’ve not put my thoughts into one place before, so I thought that I’d write what I’ve been thinking down for Petri and see what the readers think. In this article, I’m going to share my opinions on why I think that backup, which should be nothing more than just a glorified file copy, is overly complicated, unreliable, a business and job risk, and usually downright sucks. I’d love to hear what you think; so don’t be shy — please post what you think in the comments below.
We Won’t Get Fooled Again
I’ve encountered many kinds of IT staff since I started working in IT many moons ago. Some enjoyed a loud discussion, while others would hide when there was anything more vigorous than a whisper. Some were self-educators that were keen to do better, and others were clock-punchers who just wanted to serve their time. Some were keen to listen, and some were cynical of everything. I admired many and learned from them. But a few… they were the sort that really made me want to scream. These were the people that afflicted the “same old IT” on the business.
You know the person I’m talking about — when offered an alternative, something that’s better, they’ll respond with a line that goes along the lines of:
- I just buy what my boss tells me to.
- We’ve always bought this yellow-box-backup product, and we always will.
I can feel the rage building already. Here’s the kind of response my angry gut wants to go with:
- Aren’t you employed to be an IT expert? Are you not supposed to provide feedback when something just doesn’t work and there’s a better alternative? Wouldn’t your employers value that?
- If you were sick and a treatment wasn’t curing you, would you give up, or would you seek a better alternative?
- You are what is wrong with IT.
There is plenty of blame to go around here:
- The business hired a CIO/IT manager that allows this behavior. Good IT people question things and are problem solvers. If I’m not being questioned, then I’m scared.
- The CIO/IT manager has hired lambs or service providers that won’t do their complete job. If I hire skills, then I expect those experts to give me advice. Do you want an accountant that just pays the taxes, or do you want one that finds legal ways to minimize your payments?
- The IT staff person isn’t doing their complete job, and to be honest, this is the person that is to blame for much of what’s wrong in IT.
OK; maybe the rage is leaking out a little! Until a business deals with “same old IT” in a tangible way, then there will be no improvements and further conversation is a waste. To quote George R. R. Martin: “Words are wind.”
Dirty Deeds Done Dirt Cheap
Many IT decisions are incorrectly made based on the sticker price of a product. You can get backup product X for $75 less than product Y, so go for X. Backup is a technology that is there to save and protect a business. So do you want to opt for what’s cheap or what works?
I should note that there are some extremely good backup products are very affordable, so don’t misread this text.
Start considering the total cost of a backup solution and not just the sticker cost. How much will it cost to buy the backup media and run it for 30 days, 1 year, 10 years or however long you need to retain backups for? What does the software cost and how much will vendor support cost? What time will be spent on fixing backups every day? And what will be the cost of losing a document, an email, or a virtual machine when the backup product continues to be unreliable? Consider all of those genuine costs when you’re trying to shave a few pennies from a purchase.
Letting Go of Archaic Technologies
Do you really need tape backup? Tape drives are unreliable and tapes are expensive. Off-site storage with tape drives are slow and are not going to be terribly reliable when there’s a wide-scale disaster outside. You don’t use 8-track or cassette tapes to listen to music because they age badly and are unreliable, but you want to protect your business on something similar? I’m sorry, but I’m confused, because I bet most of your music collection currently sits on something that wasn’t invented in 1951.
Disk storage has never been so cheap. You can use commodity hardware solutions, such as Storage Spaces, or aggregate big and cheap SATA disks with RAID 5. This can give you large amounts of storage for short-term retention.
No More Cloudy Days
At first mention of the cloud, many questions will arise. But the cloud has many answers, too:
- OPEX: Switching from CAPEX to OPEX allows a business to embrace drip-feed-payment solutions that otherwise requires either a bank loan or salaries not to be paid or stock not to be acquired in order to fund them.
- Affordability: See the comments above about comparing sticker prices. The cloud offers really cheap storage.
- Reliability: A cloud solution should have a lot of customers, constantly hammering it, instead of your one-off deployment in the computer room or data center. And when things go wrong, the vendor can patch the problem to prevent it from happening again.
Cloud and backup are not new bedfellows. Online backup has been around for over a decade. What has changed is that the big three clouds that have developed, which are offering solutions that evolve more quickly and are a lot more affordable. You can use cloud features in many ways, including:
- Disk-Cloud backup: Remove backup servers from your office, assuming that you still have one, and back up machines directly to the cloud.
- Disk-Disk-Cloud backup: Back up your services locally to more reliable disk storage for short-term retention. Use the cloud’s cheap storage for long-term retention by automatically forwarding selected data to a secure cloud backup vault.
Can we fix it?
Yes, we can! If you’re willing to change, then things can improve. I see four generations of backup product:
- Mainframe and Server: These are the products that existed before 2005, and before the rise of virtualization. The products focused on backing up legacy platforms to tape. When challenged by newer solutions, they have tried to McGuyver in disk-disk backup or cloud solutions, but they rarely work, or work well, or work for long. These are the vendors that prevent innovation. These are the products that don’t really support Live Migration or vMotion. These are the vendors that a lot of businesses have started to dump, and these are the products that cause the most headaches in backup and preconceptions on what a backup solution is or should offer.
- Virtualization: The rise of vSphere and Hyper-V created a new generation of startup software companies that realized that they could do backup better than the old guard. They realized that if virtualization is the norm, and improvements to virtualization platforms are frequent, then why are old server/mainframe backup solutions preventing customers from taking advantage of what they own or want to own. Disk-disk backup is the norm, and support for new versions of virtualization is something that takes days or weeks, not years.
- Online backup: A huge number of vendors launched online backup solutions over the last decade or so. Some of these solutions are well-tried, reliable, and trusted. But some, particularly those launched by legacy hardware vendors, come and go like ships in the night. A common factor is that these mature solutions are built on legacy storage, so they are not necessarily as affordable as they could be.
- Cloud: The big three vendors, Microsoft, AWS, and Google, acquire and deploy at scales that we ‘little’ IT people cannot fathom. They use commodity hardware that is acquired at huge discounts, aggregate it with software-defined storage systems (there are no SANs in the big-3 clouds), and they can sell storage at a rate that cannot be beat by online backup competition.
Choosing a Backup Solution
Right now, I believe there are two correct choices when choosing backup solutions:
- Those that were designed from the ground up for virtualization, with an emphasis on disk-disk backup.
- Backup solutions that are designed to leverage the cheap long-term storage of the big three clouds, subject to regulatory requirements.
Again, feel free to let us know your thoughts in the article comments below. I’d love to hear what you think on this topic.
Related article: