Fast linux file creation

Question

I want to create an empty (may not be zeroed) file in ext4 file system. I can do this with:

dd if=/dev/zero of=file count=xxxx

But it takes a while if x is around 0.5T. I suppose there is a way to to create such file fast. It may contain gibberish it does not matter.

Answer 1

You could create a sparse file, but may not have the results you want depending on why you are doing this:

dd if=/dev/zero of=sparse_file bs=1 count=1 seek=512M

ls -l sparse_file        
-rw-rw-r-- 1 kbrandt kbrandt 536870913 2010-03-03 08:30 sparse_file

The problem is that if a program doesn't know how to handle sparse files, when do you something like copy the file, it will fill it with zeros. For example, see this previous serverfault post. You can see the difference with du as well:

[kbrandt@kbrandt-opadmin: /bigfiles] du sparse_file
12  sparse_file
[kbrandt@kbrandt-opadmin: /bigfiles] du sparse_file --apparent-size
524289  sparse_file
[kbrandt@kbrandt-opadmin: /bigfiles] #cat is not sparse aware, watch what happens
[kbrandt@kbrandt-opadmin: /bigfiles] cat sparse_file > sparse_file2 
[kbrandt@kbrandt-opadmin: /bigfiles] du sparse_file2
524808  sparse_file2

Lastly, if you do want to actually put in the zeros, if you set a higher bs it might be a little faster like bs=4096. Here is a link to someone who actually did some benchmarks with dd block sizes.

Answer 2

Would a sparse file suffice?

% dd if=/dev/zero of=sparsefile bs=1k seek=500M count=1
1+0 records in
1+0 records out
1024 bytes transferred in 0.000224 secs (4569114 bytes/sec)

% ls -lh sparsefile
-rw-------  1 james  james   500G  3 Mar 13:30 sparsefile

Answer 3

Use:

truncate -s 500G filename

It is really fast if your filesystem supports sparse files.

File contents are zeroed, always. Linux won't allow you to reuse disk sectors without zeroing them since this is a security risk. For sparse files, sectors are only allocated when they are first written to.