11

When I run btrfs filesystem balance, does this implicitly defragment files? I could imagine that balance simply reallocates each file extent separately, preserving the existing fragmentation.

There is an FAQ entry, 'What does "balance" do?', which is unclear on this point:

btrfs filesystem balance is an operation which simply takes all of the data and metadata on the filesystem, and re-writes it in a different place on the disks, passing it through the allocator algorithm on the way. It was originally designed for multi-device filesystems, to spread data more evenly across the devices (i.e. to "balance" their usage). This is particularly useful when adding new devices to a nearly-full filesystem.

Due to the way that balance works, it also has some useful side-effects:

  • If there is a lot of allocated but unused data or metadata chunks, a balance may reclaim some of that allocated space. This is the main reason for running a balance on a single-device filesystem.
  • On a filesystem with damaged replication (e.g. a RAID-1 FS with a dead and removed disk), it will force the FS to rebuild the missing copy of the data on one of the currently active devices, restoring the RAID-1 capability of the filesystem.

5 Answers 5

18

TL;DR

Btrfs' defrag feature is specific to fixing fragmentation in folder metadata and file contents, whilst the balance feature was created to "balance" (hence the name) the amount of data shared between drives whenever a drive is added or removed. While they do have some theoretical overlap in what they do, they are not directly related, thus the documentation does not link the two features.

A verbose explanation is below. Note of course that the long answer is in the hope that it will help others who don't have the full context of what these features are for.


Chunk Allocation

An important concept with btrfs is chunk allocation. When you write data to btrfs, it writes that data into a "current" chunk, typically 1GB in size1. If the "current" chunk becomes full, it allocates a new chunk. If an existing chunk is emptied, its storage space is made available for re-allocation when a new chunk is needed.

If the filesystem is using more than one drive with the "dup", "single", or "raid1" storage profiles, the chunk allocator always prefers putting the next new chunk on the drive(s) with the most free space available. This ensures, generally, that drives are used equally.


How Balance Does its Thing

The balance feature works by taking existing data chunks and re-writing them into the "current" chunk. When an existing chunk is emptied in this way, it is automatically made available to the allocator. If the existing chunk being emptied was not full to start with (perhaps old data in the chunk was deleted), the net result is the freeing up of diskspace since the newer chunk is "more tightly packed" with relevant data.

This is the part which could, in theory, be used as part of a de-fragmentation strategy, which I feel is the reason that many people assume it already does. However, of course, the balance feature was built with a specific purpose in mind, thus why it does not look at the file content. It only checks whether or not the data it is taking out of the existing chunks is still relevant2 before copying that data to the new chunk.

Where does the Balance part come in?

When you add a new drive to the filesystem, the allocator will at first tend toward writing all new data to the new drive, mainly because it has more free space available than the existing drives. By re-writing all chunks, all initially-balanced chunks are written only to the new drive. Once it has equalised, the rest of the data will be equally re-allocated between the drives, so it is now balanced.

Typical Balance Scenario:

I have 2x 500GB drives with 240GB used on each; I add another 500GB drive:

  • drive a: 240GB used
  • drive b: 240GB used
  • drive c: 0GB used

I start a balance of all the data. About one quarter through the balance, I'm likely to see a situation similar to the following:

  • drive a: 180GB used
  • drive b: 180GB used
  • drive c: 120GB used

At about the one-third mark, it appears to be balanced:

  • drive a: 160GB used
  • drive b: 160GB used
  • drive c: 160GB used

You can of course stop the balance operation at this point, though there are reasons (good and bad) why you might want to let it finish3. If you let it finish, the overall "balance" won't change after this point, though all data will be re-written to new chunks.


How Fragmentation Happens in btrfs

Btrfs is a CoW (Copy on Write) filesystem, which means that data is never over-written4. If you have an existing 100MB file and over-write a 1MB portion of the file, that 1MB portion is not written over the existing data on the drive. Instead it is written elsewhere in the "current" chunk. Btrfs keeps track of where these "fragments" of new data are stored. This is most useful for maintaining snapshots of the data as it means the old data is preserved by default. Because SSDs, in a very similar way, also never overwrite data, this CoW mechanism lends itself well to allowing SSDs to maintain their lifespan and performance.

Where Defrag Comes In

Regardless of the advantages, some files are over-written very often (typically database files), so end up having hundreds of these fragments. With SSDs, there is little performance penalty in the short term. But with spindle drives, the performance penalty is severe.

One solution is to use btrfs' defrag feature. The defrag operation re-writes the file content in the current chunk in the logical order of its current state. For our example 100MB file it would reduce the fragments into one large 100MB set of data instead of hundreds of separate tiny pieces.

An alternative solution would be to use the "nocow" feature specifically for files such as this. The nocow feature causes the file to be overwritten in place. Beware that there are caveats to nocow5 6.


Summary Again

  • The balance looks at chunks and stripes - and is not actually aware of file content except for whether or not data in those chunks is still relevant.

  • The defrag operation looks at folder data and individual file content and re-writes the data in as contiguous a fashion as possible. There are downsides if you are using snapshots5.


Notes:

  1. Though chunks are typically 1GB in size, they can be bigger or smaller. When using raid types, chunks are typically striped across multiple drives in 1GB multiples. For example, 5 drives with raid0 typically results in a 5GB stripe consisting of 1GB chunks being written to each drive.

  2. Btrfs uses "references" to file content. When part of a file is overwritten, the live filesystem "references" the location where that data was written. A snapshot however might still "reference" the old location. If there is no snapshot - or the old snapshot is deleted, there will be no more "ref"erences left that refer to the original overwritten content. This content is then considered irrelevant and will not be copied with the other relevant data in a balance operation.

  3. At this point, assuming storage is using the simple "single" profile7, the first 160GB balanced would all be moved to the new drive - but also at this point, it still has about 320GB left to balance. The rest would be balanced equally across the drives. With spindles, ideally you would want to balance only 160 chunks before having btrfs re-balance all 3 drives for a better "spread" of the data. With SSDs, attempting to maintain an even "spread" of data gets very complicated, likely pointless, and far more likely very bad for the SSD lifespan.

  4. The exception is the "nocow" feature.

  5. If there are snapshots, defragmenting the "live" file causes the snapshots and "live" file to refer to divergent data locations on the disk, causing the data to be duplicated and thus take up extra diskspace. When a general-purpose de-duplication feature becomes available, this will not be as much of a problem.

  6. Using nocow means btrfs does not maintain checksums for the file-content.

  7. With most of the raid-type storage profiles (raid1 is the exception), "spread" across the drives is moot as the stripes are typically written across all drives anyway.

2
  • Wow, great answer. I see there is a serious shortage of BTRFS user-relevant information, in books or the like (rather unlike ZFS) which seems to contribute to it's continuing poor reputation. Have you got a blog or more good stuff like this? Jan 11, 2018 at 2:44
  • 1
    Thanks! I really should bring in some more up-to-date content there. :-| Time has been sorely lacking: dogma.swiftspirit.co.za
    – zaTricky
    Jan 11, 2018 at 13:27
6

Maybe looking at the source code of the command might help

Prefer btrfs balance start

'btrfs filesystem balance' command is deprecated, please use 'btrfs balance start' command instead.

And then on the command string

"btrfs [filesystem] balance start [options] <path>",
"Balance chunks across the devices",
"Balance and/or convert (change allocation profile of) chunks that",
"passed all filters in a comma-separated list of filters for a",
"particular chunk type.  If filter list is not given balance all",
"chunks of that type.  In case none of the -d, -m or -s options is",
"given balance all chunks in a filesystem."

I might give it a second look but I can't see any references to defrag on the structs nor the ioctl() calls. So there's no explicit defrag.

All it does is copy from one place to other and using the default allocator in the process. Taken from here

Depending on the purpose allocation and on allocation mode, algorithm either directly searches for a continuous extent of freespace in each suitable allocation group (a group in btrfs corresponds to a chunk described above

So depending on the allocation mode, free space on the device, and so on you can say that btrfs will allocate in such a way that defragmenting won't be necessary. Which you might consider a form of implicit defragmentation.

HTH

3

Balance works at the chunk level; chunks are how Btrfs implements raid redundancy. It doesn't do anything at the Btree level and doesn't defragment.

0

In case you use media with high access latency, framentation always counts, regardless of the filesystem used. A seek stays a seek, pediod.

2
  • 3
    Unless you're accessing the data from SSD drive then it means nothing at all.
    – hookenz
    Dec 30, 2013 at 2:26
  • 1
    That doesn't answer the question. Mar 19, 2017 at 19:04
-2

Defragmentation is overrated. Sure, on a FAT16, it makes a real difference, but not on anything modern, in most cases. Effectively, rebalance will be improving the organization of your file system, and files will be less fragmented.

2
  • 7
    Fragmentation isn't really an issue for ext2/3/4, xfs, jfs, etc., but it can be a significant problem for btrfs. See btrfs.wiki.kernel.org/index.php/Gotchas which says "Files with a lot of random writes can become heavily fragmented (10000+ extents) causing trashing on HDDs and excessive multi-second spikes of CPU load on systems with an SSD or large amount a RAM." That's not an exaggeration, even for common use cases (files downloaded with bittorrent, sqlite databases, etc.).
    – nemequ
    Jul 26, 2014 at 22:04
  • 2
    Defragmentation can make a large different even with more modern file systems especially once the drive starts to get full on a conventional HDD. Some file systems handle it better than others and some types of files fair worse than others. Slack space, impossible to optimise scenarios, read/write caches, readahead, application optimisation and so on tend to hide a lot of this. For the most part people don't have to worry about it and should only worry about if they actually have a serious problem that might be caused by fragmentation.
    – jgmjgm
    Aug 22, 2016 at 19:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .