eCryptfs headers errors

Question

I'm getting the following error on a server where a partition is encrypted thru ecryptfs.

[3440851.003561] Valid eCryptfs headers not found in file header region or xattr region, inode 22545087
[3440830.026081] Valid eCryptfs headers not found in file header region or xattr region, inode 22553905

After unmounting the encrypted partition and the ext4 partition below, I have performed a fsck that gave me the following result:

fsck from util-linux 2.20.1
e2fsck 1.42.9 (4-Feb-2014)
/dev/sda3: clean, 65092/72302592 files, 54219978/289200384 blocks

I am a bit clueless to understand what happen. We're using the same setup on several instances, and we observe this on only one of them.

The solution may be to change the underlying disks! But I would like to understand what happen, in order to eventually detect and prevent that kind of incident.

Answer 1

I have seen this happen when the system was not shut down cleanly. In particular I have seen it happen when the encrypted data was stored on a USB device where the connection between host and USB device was a bit unreliable. But I believe that other non-clean shutdowns while files are being written to could cause it as well.

Searching by inode as suggested in the answer by Giovanni can indeed be used to find the problematic file. Since ecryptfs preserves the inode numbers of the underlying file system, that command can be used to find both the encrypted and unencrypted path to the file.

Searching for the file on the underlying file system this way is significantly faster than searching through the ecryptfs file system. My measurements on one system showed a slowdown by a factor of 8 between the two with cold caches and a factor of 350 difference with hot caches.

Because of that overhead I suggest you first find the encrypted file on the underlying file system. For example in the default configuration on an Ubuntu system this command could be used:

find /home/.ecryptfs -inum 22545087

This should find the path to the encrypted file, which includes the name of the home directory in which it was found. Then when searching for the unencrypted file name, you can immediately limit the search to just one home directory:

find /home/username -inum 22545087

If the user has so many files that this is too slow, you can take advantage of the inode numbers to look up one directory level at a time. For instance if the encrypted file name was

/home/.ecryptfs/username/.Private/ECRYPTFS_FNEK_ENCRYPTED.AAAAAA/ECRYPTFS_FNEK_ENCRYPTED.BBBBBB/ECRYPTFS_FNEK_ENCRYPTED.CCCCCC

You can first run

ls -i /home/.ecryptfs/username/.Private/ECRYPTFS_FNEK_ENCRYPTED.AAAAAA

This will give you the inode number of the outermost directory. You can then look up the unencrypted version of that directory name:

ls -i /home/username | grep $INODE_NUMBER_FROM_LS

You can repeat this for each level in the directory hierarchy to get the unencrypted path without having to use as much CPU time as required to decrypt every single file name in that home directory.

Answer 2

Discover which file is causing this via:

find / -inum <inode number>

You will probably find a truncated file, and that's the reason why ecryptfs is raising that warning.

Try to read the file with cat, and rm it to fix the warning.