How to zero fill a virtual disk's free space on windows for better compression?

Question

How to zero fill a virtual disk's free space on windows for better compression?

I would like a simple open source tool (or at least free) for that. It should probably write an as big as possible file full of 0and erase it afterwards. Only one pass (this is not for security reasons but for compression, we are backing up virtual machines).

Should run from inside windows and not from a disk.

On Linux I do it like this (as a user):

cd
mkdir wipe
sudo sfill -f -l -l -z ./wipe/

Edit 1: I decided to use sdelete from the accepted answer. I had a look at the sdelete's help:

C:\WINDOWS\system32>sdelete /?

SDelete - Secure Delete v1.51
Copyright (C) 1999-2005 Mark Russinovich
Sysinternals - www.sysinternals.com

usage: sdelete [-p passes] [-s] [-q] <file or directory>
       sdelete [-p passes] [-z|-c] [drive letter]
   -c         Zero free space (good for virtual disk optimization)
   -p passes  Specifies number of overwrite passes (default is 1)
   -q         Don't print errors (Quiet)
   -s         Recurse subdirectories
   -z         Clean free space

This is an old version. I used the -c switch from the 2nd invocation and this was quite fast (syntax only valid for older versions before V1.6):

c:\>sdelete -c c: (OUTDATED!)

I have the impression this does what I want. The sdelete tool is easy to use and easy to get.

Edit 2: As scottbb pointed out in his answer below, there was a September 2011 change to the tool (version 1.6) The -c and -z options have changed meanings. The correct usage from 1.6 onwards is

c:\>sdelete -z c:

Edit 3: There is a 2.0 version of sdelete and sdelete64. They appear to be buggy when zeroing. It will appear to be stuck at 100% for extremely long times. Some people have reported 10 - 40 times longer. The older version 1.61 does not have this issue. See https://social.technet.microsoft.com/Forums/en-US/2ffb2539-34ba-4378-aa8a-941d243f117e/sdelete-hangs-at-100?forum=miscutils

Edit 4: Now there's the issue of dynamically allocated virtual disc space. If you have a 100GB disk that is not full and uses only 30GB on the host, zero filling should not increase dramatically the size of the disc because that contradicts the purpose of dynamic allocation. There is an answer for Oracle VM VirtualBox https://superuser.com/q/907196/44402 - but on other stacks like VMWare, Xen, XenServer, etc., this needs to be answered separately.

Answer 1

On windows use the sysinternals tool sdelete to zero out all the empty space. The command you want would look like this sdelete -z c:.

Usage: sdelete [-p passes] [-s] [-q]  ...  
sdelete [-p passes] [-z|-c] [drive letter] ...  
-a         Remove Read-Only attribute.  
-c         Clean free space.  
-p passes  Specifies number of overwrite passes (default is 1).  
-q         Don't print errors (Quiet).  
-s or -r   Recurse subdirectories.  
-z         Zero free space (good for virtual disk optimization).

For Linux I suggest you use zerofree.

Answer 2

Windows already has a built-in command line tool that does this: CIPHER. See its /W option.

So the complete command would be: cipher /w:c: (replace c: with actual drive letter)

(it writes zeroes to the unused space, then 0xff, so you have to watch it and stop it with ctrl-c when it begins to write 0xff - not ideal, but it is free and already available in Windows since XP at least)

Answer 3

fsutil isn't open source, but is does ship with Windows and therefore doesn't cost anything extra. I used it to zero the free space on an empty WD 250 GB 7200 RPM drive (F:). Here's what I ran from the command line:

fsutil volume diskfree f:

Which showed this report:

Total # of free bytes        : 249899469856
Total # of bytes             : 249997291520
Total # of avail free bytes  : 249899469856

I used Total # of avail free bytes in the following commands:

fsutil file createnew F:\clear 249899469856
fsutil file setvaliddata F:\clear 249899469856
fsutil file setzerodata offset=0 length=249899469856 F:\clear
del f:\clear

It took about 4 hours to write 250GB of zeros.

Answer 4

SDelete will not zero out the space for compaction of a virtual disk. SDelete starts by zeroing free space, but then fills it with 0xff and then random bytes See How to prepare a Virtual Server 2005 virtual hard disk file to send to Microsoft Product Support Services For how to set up a VHD for compaction

Precompact.exe can be difficult to find for some reason. I found a seperate downloadable copy here. However if you have windows virtual PC you can also get it from %programfiles(x86)%\Windows Virtual PC\Integration Components\Precompact.iso

Answer 5

I too had been looking for a way of using Cipher to only write 0x00 then exit when starting to write 0xFF to the free space. This will allow the maximum free space to be compressed. I have come up with the following basic PowerShell job

replace d:\ with the required drive

Function CipherFreeSpace
{
    $cipherjob = @()
    $Job = start-job -ScriptBlock {cipher /w:d:\ }
    while ($cipherjob -notcontains "Writing 0xFF")
    {
        Write-host "." -nonewline 
        Start-Sleep 2
        $cipherjob += $job | Receive-Job 
    }
    $Job | Stop-Job | Remove-Job -Force
}
CipherFreeSpace

Regards Simon

Answer 6

If you are using VMware, a much better way is to use the tools built into VMware to reclaim any unused space on the virtual drive.

I used this to reduce the size of the virtual disk from 28 gigabytes down to 13 gigabytes. Your mileage may vary.

When I tried using sdelete -z c:, it actually increased the size of the virtual drives to 30 gigabytes, which was its maximum size.

See the VMware website and Compact a Virtual Hard Disk, which is under Using VMware Workstation > Configuring and Managing Devices > Configuring and Maintaining Virtual Hard Disks.

Update

Sometimes, the GUI tools are not quite as reliable as the command line tools. I have run into issues where the GUI tools fail, but the command line tools work just fine. Here are the options for VMware Desktop 10:

C:\Program Files (x86)\VMware\VMware Workstation>vmware-vdiskmanager -h
Diskname or some other argument is missing.
VMware Virtual Disk Manager - build 1895310.
Usage: vmware-vdiskmanager.exe OPTIONS <disk-name> | <mount-point>
Offline disk manipulation utility
  Operations, only one may be specified at a time:
     -c                   : create disk.  Additional creation options must
                            be specified.  Only local virtual disks can be
                            created.
     -d                   : defragment the specified virtual disk. Only
                            local virtual disks may be defragmented.
     -k                   : shrink the specified virtual disk. Only local
                            virtual disks may be shrunk.
     -n <source-disk>     : rename the specified virtual disk; need to
                            specify destination disk-name. Only local virtual
                            disks may be renamed.
     -p                   : prepare the mounted virtual disk specified by
                            the mount point for shrinking.
     -r <source-disk>     : convert the specified disk; need to specify
                            destination disk-type.  For local destination disks
                            the disk type must be specified.
     -x <new-capacity>    : expand the disk to the specified capacity. Only
                            local virtual disks may be expanded.
     -R                   : check a sparse virtual disk for consistency and attempt
                            to repair any errors.
     -e                   : check for disk chain consistency.
     -D                   : make disk deletable.  This should only be used on disks
                            that have been copied from another product.

Here is an example of shrinking a disk. This command line worked, in spite of the same command from the GUI failing:

C:\Program Files (x86)\VMware\VMware Workstation>vmware-vdiskmanager.exe -k
 "H:\VMware\WindowsXP.vmdk"
  VixDiskLib: Invalid configuration file parameter.  Failed to read configuration file.
  Shrink: 100% done.

The error message starting with VizDiskLib is new to VMware Desktop 10 and can safely be ignored.

Answer 7

Google-up Eraser 5.8.8 32-bit Works nicely even on Windows XP SP3!

Answer 8

Mit,

I found that Simon's powershell script did exactly what you need, without the versioning problems of sdelete. (switches doing different things in different versions)

It had one problem, that cipher kept running, but I fixed that.

I further modified the script to cleanup the temp folder:

You will need to replace F:\ with your drive letter in both places.

Function CipherFreeSpace
{
    $cipherjob = @()
    $Job = start-job -ScriptBlock {cipher /w:F:\ }
    while ($cipherjob -notcontains "Writing 0xFF")
    {
        Write-host "." -nonewline 
        Start-Sleep 2
        $cipherjob += $job | Receive-Job 
    }
    Stop-Process -processname cipher
    Start-Sleep -s 3
    Remove-Item -recurse F:\EFSTMPWP
    $Job | Stop-Job | Remove-Job -Force
}
CipherFreeSpace

Answer 9

Why third party tools, when Windows already includes all you'll ever need, for "free" and as fast a it can ever be:

1) format X: /fs:XXXX /p:#, where X is the drive letter; XXXX is the file system,Yeither and # is the number of passes;

2) diskpart clean command: http://technet.microsoft.com/library/cc766465.aspx, http://support.microsoft.com/kb/300415;

Btw, you don't need more than a single pass, either - a claim otherwise is either outdated info for obsolete disk red/write strategies, people pushing products, paranoia driven, or bs passed on as "expert advice".

Edit: nvm, missed the free space part...