effective file updates under Linux

Question

In a high performance setting (many concurrent updates) under Linux, which is the most effective method to update a 30k file on disk:

1. simply update the respective file
2. delete the old file and save the new one

I am mostly concerned about disk access time but also the processor load may be a factor here.

Answer 1

The disk subsystem and filesystem you're using will have a huge impact here. In fact there are so many different possible results that you probably should benchmark it. However :

• keep in mind that real, synchronous IOs are limited to about 100 IOPS for a SATA drive, 200 for a SAS drive, and vary wildly from 10 IOPS to 10000 with SSDs. Multiply the number of IOPS by the number of data drives.
• modern filesystems will cluster write operations together. Proper filesystem choice and detailed tuning will change results by a factor of 10 to 100.
• modern storage controllers can cache writes. Proper write-back cache settings will once again change results by a factor of 10 to 1000.

So proper hardware (real RAID controller with WB cache, SSD), proper software (modern filesystem, ext3 is absolutely out of the question here, I'd go with xfs but ext4 is an option) and proper tuning (test various kernel IO scheduler, IO size, etc. settings) will have an immense impact.

Answer 2

If you are updating it a LOT then the files contents being stale is not likely to be an issue for you. If so, stick it on tmpfs, truncate the file on update and rewrite into it. That would be the cheapest method as it is not likely to use disk at all.

The next closest is to truncate/write onto a filesystem that has the noatime mount option set and journalling turned off. But again its risky if you crash out as you might possibly lose the data.

After that its noatime again with journalling turned on.

Remember, linux buffers writes and syncs to disk at determined intervals so you wont normally 'feel' the impact of a write from an I/O standpoint (unless its very heavy writing but thats tunable). You can alter the conditions to sync to disk too so you can have the write buffer fill up for quie a long time before syncing to disk.

If your after doing something really clever.. use fallocate to preallocate space for the file which would be its maximum possible value. Then, mmap open the file to read it directly to memory. Then rewriting will be near instantaneous (but lossy if you had power loss). You can then control when to flush back to the disk with the msync call.

Answer 3

Assumming a FLAT file, and not a database file, Pros and Cons on either method really:

1. If you simply overwrite the contents of the file in-place, then you certainly prevent the reallocation steps. So you might save a little time there. However, the placement of the pieces may not be optimal.

2. You might get a more optimal placement of data depending on the fragmentation of the disk. But it will be a little big slower as space must be allocated and if you are running in a more secure setting there will be time necessary in zeroing out the block before allocation.