6

I have a 64 GB Linux disk image with ~50 GB of unused space across the partitions. The file is sparse, so it only takes ~14 GB on disk.

But if I dd the image, it writes the full 64 GB, which takes quite a while.

Is there any way I can do the equivalent of dd if=os.img of=/dev/sdb with this image, without having to write 50 GB of zeros?

Is there any tool that is smart enough to do this i.e. an imaging tool that has an awareness of the EXT4 filesystem?

4

dd can handle this. You need to add conv=sparse to the command line.

From the man page:

              try to seek rather than write the output for NUL input blocks
5
  • 1
    I should have mentioned that I did consider conv=sparse but don't think it works in this case. I could be wrong. When dd is creating a file inside a filesystem it can safely seek because EXT4 supports sparse files (i.e. it will treat that seeked-over space as zeros). But when it's writing to a disk device, it's going to seek over random parts of the disk device which could be zeros or junk data, causing corruption. It seems like I would have to dd if=/dev/zero of=/dev/device first to ensure I have all zeros, which would defeat the purpose. Seems like an FS-aware tool is req. – hackerattacker2019 Sep 4 '19 at 20:20
  • @hackerattacker2019 What are you writing the image to? – Michael Hampton Sep 4 '19 at 21:58
  • I'm writing the image to a block device (an SSD) e.g. /dev/sdb – hackerattacker2019 Sep 4 '19 at 22:13
  • 1
    If it's really an SSD then it doesn't matter because you'll have trimmed the device first. – Michael Hampton Sep 4 '19 at 22:14
  • I am zeroing the drive with ATA security erase, it does read all zeros in my tests. So this does seem like it should work but I'm slightly uneasy ;-) Thanks! – hackerattacker2019 Sep 6 '19 at 17:02
2

I would consider using mksquashfs to create a read-only copy instead of making a dd of whole volume. This additionally compresses and de-duplicates the data.

Please note that you have to evaluate - it's usability depends on your specific use case.

1

It is awkward since you have a whole disk image but you could:

  1. use your partitioning program of choice to list the partitions in os.img and create them on /dev/sdb
  2. restore each partition with e2image. e.g., e2image -aro 1048576 os.img /dev/sdb1

This does not handle the MBR if there is one. /bin/dd if=os.img of=/dev/sdb bs=512 count=1 will do that.

Note: the e2image man page at linux.die.net is out of date. From a bionic install:

SYNOPSIS
   e2image [ -r|Q ] [ -f ] device image-file
   e2image -I device image-file
   e2image -ra [ -cfnp ] [ -o src_offset ] [ -O dest_offset ] src_fs [ dest_fs ]
1
  • The e2image utility does seem like exactly what I'm looking for. Will have to give this a try. Thank you! – hackerattacker2019 Sep 6 '19 at 17:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.