Preview of WORM Storage Added To Azure

Posted on July 6, 2018 by Aidan Finn in Microsoft Azure with

USB Type-C connector (Image Credit: thunderbolttechnology.net)

This post will discuss and demonstrate the new write once-read many (WORM) blob storage feature that Microsoft recently launched in preview.

 

 

More Cheaper Storage

Many organizations around the world have a legal, regulatory, or compliance requirement to retain data in an immutable form. You might have, for example, been on the phone with call center and told that your call “might” be recorded for training purposes. In a past life in the finance world, the IT department that I was a part of maintained a storage system for digitally recording paper contracts on laser discs. Normal disk storage was not accepted by courts as proof of an unmodified contract but WORM storage with write-only PDF scans was.

As you can imagine, phone calls, contracts, and the many kinds of data that pharmaceutical, banking, legal, and many more organizations can generate is huge. Keeping that data on-premises is a big expense that can impact profits, but more likely impacts the bottom line of clients.

Cloud storage is much more cost-effective and Azure blob storage is a very cost-effective way to store data in the cloud; blob storage is low cost and moving blobs (files) to cool or archive tiers can bring down the cost even more! The problem with blob storage is that it has not been capable of WORM storage and blobs (file) can be deleted before they should be … until now.

Immutable Blob Storage

A new policy for a blob (file) storage container (folder) called Immutable Blob Storage allows you to place a WORM policy on all blobs in that container; in other words, we can allow files to be uploaded to a folder in affordable Azure storage and once the files are uploaded, they become read-only for a certain number of days that we define. We can retain files for between 1 and 146,000 days. That’s 400 years! The policy can be locked but be careful with this. That policy becomes read-only too! Even though a file becomes read-only (or immutable), it can be moved between the various tiers of blob storage to suit usage demand versus cost-effective long-term retention needs.

This means that if you can write an application that can store files in blob storage, you can use this new immutable storage policy to secure the contained blobs for legal retention.

Enabling Immutable Storage

You must use a General Purpose v2 (GPv2) storage account for immutable blob storage. The benefit will be the availability of blob tiering which can greatly reduce the cost of cold data. Have a look at the preview of Azure blob storage lifecycle management. Create a container in blob storage to be used as your WORM storage location.

 

A blob container in an Azure GPv2 storage account [Image Credit: Aidan Finn]

A Blob Container in an Azure GPv2 Storage Account [Image Credit: Aidan Finn]

Browse into the container and click Access Policy. You can storage access and immutable blob storage policies here. Click Add Policy under Immutable Blob Storage.

A dialog will appear, allowing you to create a policy for blobs that are stored in this container. How this works depends on the Policy Type that you select.

  • Time-based retention: You can retain blobs in a read-only state for between 1 day and 146,000 days.
  • Legal hold: All data in a legal-hold container are retained indefinitely in a read-only state. You must configure between 1 and 10 tags. which are used as identifier strings such as a solicitor’s/lawyer’s case ID.
Creating an Immutable Blob Storage policy in Azure [Image Credit: Aidan Finn]

Creating an Immutable Blob Storage Policy in Azure [Image Credit: Aidan Finn]

The type of policy also affects how the policy can be deleted:

  • Time-based: You can delete a policy by selecting the Delete action. However, a policy can become permanently read-only if you select the Lock action.
  • Legal hold: Once the tags, or case IDs if you think of them that way, are deleted, the policy is deleted. There is no Lock action.
Locking an Azure Immutable Blob Storage policy [Image Credit: Aidan Finn]

Locking an Azure Immutable Blob Storage Policy [Image Credit: Aidan Finn]

Note that you can also place a Lock on a storage account to prevent accidental deletion and restrict access permissions to the storage account resource, resource group, or even the subscription if you wish while granting an application whatever rights it needs to write data to the container. Combine this with archived auditing from Activity Log and you have a pretty controlled storage system!

Regular Actions

Once your immutable blob storage policy is enabled, you can store/upload files as blobs in the container. Once the files are written, they are read-only and modification-proof until they either age-out or the policy is lifted. You can still change the blob tiers, but you cannot delete the blobs. You can try (I know you will) but the task will fail.

Opinion

Once again, Microsoft has identified a market where very large amounts of expensive on-premises storage are traditionally used and with a simple enough (for us to use) change, Azure has an alternative. There are some limitations. I can imagine SaaS vendors that operate in the legal, financial, and pharmaceutical documentation worlds jumping on this feature and offering customers a better alternative than has been available in the past.

Tagged with , , , , , , ,