I want to set up a database in a high durability set-up on Azure. I've previously relied on DB-as-a-service offerings, but can't do that in this case, so I'd like your feedback on the plan below. Is this enough to ensure reliable storage of data?

  1. An Azure Web App takes in metric data from the web, does some minor processing and sampling, and sends the data in batches to VM2.
  2. VM2 runs the Clickhouse database, and stores data on an Azure Managed Disk
  3. Some periodical job takes snapshots of the disk and stores them to cold storage

My understanding of managed disks is that they are enough to ensure reliable storage of data, that they should take away any concerns of data loss due to hardware failure. Is this correct?

Another concern is data loss due to human error, i.e. accidentally running "DROP TABLE xx" on the wrong data. I think storing periodical backups takes away this concern (i.e. allows for recovery to the last backup). Do you agree?

The recovery plan is that if VM2 fails, some monitoring process catches this and spins up a new VM2 instance attached to the same managed disk. The Web App similarly restarts if it fails.

I understand that this setup isn't high-availability, if a VM fails there will be some window of time before it is able to store new data. This is acceptable to me. But I want to ensure that data that gets stored will not be lost, i.e. durably stored with very high probability. Is this enough to ensure that? Do you see any problems?

  • (I think ClickHouse relies on replication to protect against hardware failure. Backup and restore, which might involve some loss of data, protects against human errors. See Data Backup). Paraphrasing a little, you mention high durability, high reliability, and high availability. Those three terms don't mean quite the same thing. You might want to research the differences, and edit your question to clarify. Apr 1, 2021 at 12:39
  • You're right, I said "high reliability" where I should have said "high durability". Edited. When I said "high-availability" is something I'm OK with not having, that was intentional. I realize Clickhouse has replication built in. I'm asking if an Azure Managed Disk works as a substitute for replicating the database, for the purpose of ensuring high durability. Apr 1, 2021 at 12:52

2 Answers 2


I'm asking if an Azure Managed Disk works as a substitute for replicating the database, for the purpose of ensuring high durability.

I don't think so. A managed disk offers high availability by eliminating the disk as a single point of failure. But managed disks don't know anything about the resources that are writing to the disk.

Database administrators don't rely on filesystem backups to protect databases from human error; filesystem backups alone can't account for concurrent database use and such. DBAs do rely on backups, just not filesystem backups.

By the same token, managed disks expose a highly available filesystem. As a DBA, I don't think I'd rely on managed disks to replace replication for the same reason I wouldn't use a filesystem backup to replace a proper database backup.

  • I get your objection to using filesystem backups, I don't intend to do this, but rather to use clickhouse's built-in features for backups. But let's leave backups aside. I don't see how this objection applies to managed disks as a substitute for replication. Is there any specific scenario where this could cause problems? Apr 1, 2021 at 15:23
  • I can't name a specific scenario off the top of my head. My assessment is based on the more general fact that filesystems are not database-aware. Database replication is database-aware. Apr 1, 2021 at 16:51
  • After thinking about this, I think that the only scenario is if the managed disk somehow doesn't behave like a normal disk in its behavior towards the OS. As I understand it, the managed disk should behave like any other disk in the eyes of the OS, so as long as that is true it doesn't seem like it could be a problem (as far as I can see). One objection would if this would turn out to be less performant per dollar compared to having multiple VMs with local disks. Apr 1, 2021 at 18:54
  • @ServableSoup: It occurs to me that you might want to ask this question on dba.stackexchange.com before you commit to one or the other. Apr 4, 2021 at 9:32
  • That is an excellent suggestion, that's probably the best forum for this. Will post it there, thanks. Apr 5, 2021 at 11:39

Define what you mean by high durability. What type of data loss problems do you expect it to protect you against?

Database replication enables logical replication independent of the underlying block structure. Say some OS administrator did a silly thing and deleted database files. That would get captured in the next disk snapshot. But if you had a replicated database, file deletions aren't replicated. The secondary could be made the primary relatively quickly.

No storage system is perfect. There is a low, but non-zero, chance a disk will lose data. Perhaps faulty hardware or software in the storage. Maybe a natural disaster destroys multiple availability zones in a region. A database replica in another region is hopefully located in a distant city, and entirely independent of what happens to the primary's storage. Different storage hardware, different power grid, different physical risks.

Replicating another copy is expensive. You may find one backed up copy to be sufficient durability for your needs.

  • By durability I mean that data written doesn't get lost. Azure managed disks are backed by a triple-redundant copies of the data, that can be geo-redundant if you pay for it. MS advertises these disks for use in databases, and advertise "an industry-leading ZERO% annualized failure rate". To me it seems like a good fit, so I'm looking to hear specific reasons why its not. Apr 1, 2021 at 18:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.