On January 29, half of Microsoft 365 customers were surprisingly deleted from the Azure SQL databases (DBs) that utilize custom KeyVault keys for TDE (Transparent Data Encryption).
After MS Azure users had taken to social media to express their outrage, an exclusive report published by The Register (an independent news and views for the tech community) had reported that during a network infrastructure event “some internal code accidently dropped these databases during Azure’s portal wobble yesterday, forcing Microsoft to restore customer data from a five minutes ago snapshot. That means transactions, product orders, and other updates to the data stores during that five-minute window were lost.” The full report can be found here: “Forget snowmageddon, it’s dropageddon in Azure SQL world: Microsoft accidently deletes customer DBs.”
To summarise in short:
The exclusive report informs us that this was caused by “a CentruryLink DNS snafu locked half of Microsoft 365 users out of their cloud accounts. An automated process designed to trigger when custom keys have been removed from the KeyVault. A result of this rogue script running is an obvious, ill-fated and inconvenient cause of the ‘accidental deletion’ of the SQL databases.
In conclusion, TDE is intended to protect the Microsoft Azure database against external malicious activity, it is clear that from this unfortunate mishap, it has been its own worst enemy.
On the contrary, a big part of cloud computing is all about finding solutions to problems, Microsoft Azure’s solution to its problem is BYOK (Bring Your Own Key) support, allowing users to encrypt DEK (Databases Encryption Key) with an asymmetric key called TDE Protector. Microsoft notify customers of the following:
“Note! If the Azure AD identity is accidentally deleted or the server’s permissions are revoked using the KeyVault’s access policy, the server loses access to the key vault, and TDE encrypted databases are dropped within 24 hours.” And “Note! If TDE encrypted SQL databases lose access to the key vault because they cannot bypass the firewall, the databases are dropped within 24 hours.” This clearly tells us that although, the solution is viable, it doesn’t mean that all well ends well.