Hard drive reliability when constantly handling lots of reads/writes/deletes

Question

In addition to storing lots of photos, my web application has the need to constantly read, write, and delete a lot of temporary files which it uses for various functionality. Some of these temp files can be very small, others may be quite large (1 GB +). I am running on Linux.

Would this constant reading/writing/deleting increase the risk of the drive failing? In order to ensure the actual photos remain safe, I'm tempted to store the photos on one drive and set up another drive for the temp files... but is that overkill?

Thank you, Brian

Answer 1

Using the hard disk will most certainly increase the risk that it breaks, compared to not using it...

However, since the drive can break anyway, I recommend you come up with some backup strategy for the actual data you are interested in. Backup to a second drive, or using RAID are the common strategies; depending on the risk you might want to backup to a different location perhaps. With proper backup, there is no need to have a scratch disk, IMO.

Answer 2

You may want to consider storing the these files on a tmpfs mount (with a good sized swap area on a raid-backed block device). This make all the small temp file both fast (in that they don't go to disk) and avoid wear on the disk.

If you are consider with the reliability of your system at all you'll have some type of UPS setup, so you can flush the temp files to avoid any data loss.

Answer 3

You'll want to setup a RAID array so that if you lose one drive your system stays online until that drive can be replaced. Beyond that you'll also want some sort of backup solution in place so that if you lose the RAID array you still have a copy of the data (all be it a little bit old).

Answer 4

If you're worried about the photos being kept safe, you need a good backup scheme. Whether it's creating a mirror (rsync?) to an external drive or setting up a backup tape scheme that runs nightly, that should be enough to keep the data safe. It's also assurance that the data is actually backed up as opposed to available (which is what RAID does). If you screw up the photo or corrupt it, RAID will happily ruin any chance of recovery.

Will constant use kill the drive faster? I assume it'll wear the drive a little bit, but the drive failure you usually encounter comes from the fact that the platters are spinning constantly. That's whether you're using the drive or not; as long as there's power, unless power saving mode is entered on the disk or the computer is asleep, it's spinning and wearing on the drive. The real kicker is that if you let a drive sit for a couple years it may not start spinning up again, so you still lose data.

This still kind of depends on what you're doing. I mean, if you have a 64 bit OS and a turtletits of RAM, you can create a multi-gigabyte temporary drive in RAM and use that for swapping. You could also use a separate drive, but that would be for speed, not necessarily because it would kill a drive (asynch I/O in that case). But just having your own drive, if fast enough for your needs, for temp files will work fine.

What it boils down to is needing a good backup scheme. I think the drive wear thing is just overthinking it a bit.