1

Imagine the physical volume was resize with:

pvresize /dev/sda2 

And the logical volumne was resized with:

lvresize -l +100%FREE system/home

But this command was not done:

resize2fs /dev/mapper/system-home

Since the last command was forgotten, there is unused space.

How to calculate the unused bytes?

Background: I want to write a check-script which checks if on a server the resize of the filesystem was forgotten.

BTW: I don't trust shell scripts.

3
  • Except EXT4 there're bunch of different FSes. Why limiting to a single one?
    – poige
    Aug 20, 2019 at 13:32
  • @poige yes, you are right. Above question uses resize2fs just as an example. I guess calculating the unused space is independent from the file system type, since it gets done on a lower level.
    – guettli
    Aug 20, 2019 at 13:56
  • as I explained in serverfault.com/a/979994/67675 it's pretty simple, being FS-independent at the same time. Moreover, it's furthermore LVM-independent…
    – poige
    Aug 20, 2019 at 14:02

7 Answers 7

6

The default size of the entire block device, plus online growth, makes the size check unnecessary. Run resize2fs on all of your file systems.

(If you have a use case that requires the size, please edit your question.)

Prevent forgetting the file system again by adding lvresize --resizefs to your scripts or shell aliases.


Ideally the lack of space will keep appearing on capacity planning reports. Hopefully it won't get so full as to trigger alerts about almost filling up.

1
  • Nevertheless, I am curious and I would like to calculate the size of the unused area.
    – guettli
    Aug 14, 2019 at 6:26
4
+25

Step 1 : identify filesystem boundaries

dumpe2fs -h /dev/mapper/system-home | grep -E "^Block count|^Block size"

you will get total filesystem size with those 2 numbers, example below:

Block count:              98304
Block size:               1024    (which is 1k)
FS size = 98304 = **96MB**

Step 2 : identify block device boundaries

a) identify the dm associated with your logical volume readlink -f /dev/mapper/system-home which gives you /dev/dm-XX

b) identify the /dev/dm-XX boundaries cat /sys/block/dm-XX/size which gives you the total size in (512 bytes) blocks. If the size is 300000 blocks, then you have a bit less than 150MB on the block device

150-96 makes ~54MB of free allocatable filesystem space

1
  • @mebius99 thanks for improvements, and sorry for my english ;-)
    – averon
    Aug 20, 2019 at 14:29
2

I solve the problem of the resize with a script that find all block devices involved and prints the resize commands them accordingly.

https://github.com/mircea-vutcovici/scripts/blob/master/vol_resize.sh

In a similar way you can change it to run the commands that show the block device sizes and the file system size. Just taking a look in the script you will realize that the answer is a bit more complex.

E.g. Imagine a layout like:

  • Multipath for a LUN
  • partition table
  • lvm
  • dmcrypt
  • zfs
  • zfs zvol

How to find the size:

  • block devices: blockdev --getsz sdX, grep . /sys/block/*/size
  • lvm: pvs
  • ext2, 3, 4: dumpe2fs /dev/sdX|egrep '^Block (count|size):'
  • zfs: zpool status, zpool list, zfs list
  • btrfs:
  • xfs: xfs_info
  • device mapper: dmsetup table
  • mounted file systems: df
1

Others have listed tools that could be used to answer your question, but I wanted to produce an actual working implementation that you could use. So, I wrote a bash script to do exactly what you are asking: reporting how much free space is at the end of the LV not yet absorbed by the FS (such as happens when someone forgets to grow the FS itself).

I should note that since determining the size of the filesystem is dependent on that filesystem's tools, you have to personalize it for which filesystems you are using for your LVM disks. I personally use both XFS and EXT4, so the script below implements both of them. Specifically, I had to use tune2fs for EXT4 and xfs_info for XFS.

lvfree.sh

#!/bin/bash

# Header
printf '%-20s %-7s %8s %8s %8s\n' Mountpoint FS_Type LV_Size FS_Size LV_Free

lsblk -nlbo TYPE,FSTYPE,MOUNTPOINT,PATH,SIZE | grep -E '^lvm +(ext4|xfs)' | while read TYPE FSTYPE MOUNTPOINT PAATH SIZE; do
  if [[ $FSTYPE == 'ext4' ]] ; then
    FSSIZE=$(tune2fs -l $PAATH | grep -E '(Block count|Block size)' | awk 'BEGIN{x=1}{x=x*$NF}END{print x}') 
  elif [[ $FSTYPE == 'xfs' ]] ; then
    FSSIZE=$(xfs_info $MOUNTPOINT | grep ^data | /bin/tr = ' ' | awk '{print $3*$5}')
  fi
  LVFREE=$((SIZE - FSSIZE))
  printf '%-20s %-7s %8d %8d %8d\n' $MOUNTPOINT $FSTYPE $((SIZE/1024/1024)) $((FSSIZE/1024/1024)) $((LVFREE/1024/1024))
done

Testing it out

First, I extend a couple of my LVs, but forget to extend the FS

[root@xps joshua]# lvextend -L +1G /dev/vg1/home
 Size of logical volume vg1/home changed from 90.00 GiB (23040 extents) to 91.00 GiB (23296 extents).
 Logical volume vg1/home successfully resized.

[root@xps ~]# lvextend -L +256M /dev/vg1/usr
 Size of logical volume vg1/usr changed from 25.00 GiB (6400 extents) to 25.25 GiB (6464 extents).
 Logical volume vg1/usr successfully resized.

Now I run the script and see the free space in the LV

[root@xps ~]# ./lvfree.sh 
Mountpoint           FS_Type  LV_Size  FS_Size  LV_Free
/                    ext4        5120     5120        0
/var                 ext4       24576    24576        0
/var/log             ext4       20480    20480        0
/usr                 ext4       25856    25600      256
/home                xfs        93184    92160     1024
/tmp                 ext4       10240    10240        0
/opt                 ext4        5120     5120        0

Now, I remember to grow the XFS and EXT4 filesystems:

[root@xps ~]# xfs_growfs /home
meta-data=/dev/mapper/vg1-home   isize=256    agcount=8, agsize=3276800 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=0        finobt=0, sparse=0, rmapbt=0
         =                       reflink=0
data     =                       bsize=4096   blocks=23592960, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=65536  ascii-ci=0, ftype=0
log      =internal log           bsize=4096   blocks=6400, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 23592960 to 23855104

[root@xps ~]# resize2fs /dev/mapper/vg1-usr 
resize2fs 1.45.3 (14-Jul-2019)
Filesystem at /dev/mapper/vg1-usr is mounted on /usr; on-line resizing required
old_desc_blocks = 2, new_desc_blocks = 2
The filesystem on /dev/mapper/vg1-usr is now 6619136 (4k) blocks long.

And finally, I run my new script to see that the new space has been incorporated into the filesystems

[root@xps ~]# ./lvfree.sh 
Mountpoint           FS_Type  LV_Size  FS_Size  LV_Free
/                    ext4        5120     5120        0
/var                 ext4       24576    24576        0
/var/log             ext4       20480    20480        0
/usr                 ext4       25856    25856        0
/home                xfs        93184    93184        0
/tmp                 ext4       10240    10240        0
/opt                 ext4        5120     5120        0

Footnote: PAATH in my script is not a typo. I had to not use the variable name PATH since it is has a reserved meaning in shell scripting.

1

Simply, using features:

crtdev=/dev/sda9
crtdev=/dev/Vol-1/Root
  1. Device size (real):

    { read devsz;read devss;}< <(blockdev --getsz --getss $crtdev)
    devsize=$((devsz*devss))
    

    Nota: You could use lsblk to list devices with sizes:

    lsblk -no KNAME,NAME,SIZE
    

    This could be usefull in a script... See further!

  2. File system size (current)

    1. for ext2|ext3|ext4:

      while IFS=$':\t\r\n' read fld val;do
          case $fld in
               Block\ size)  fsbs=$val;;
               Block\ count) fssz=$val;;
           esac
      done < <(dumpe2fs 2>&1 -h $crtdev)
      fssize=$((fssz*fsbs))
      
    2. for xfs:

      XFS info won't execute on mounted devices, you have to use mont-point instead:

      mntpnt=/mnt
      while IFS=$' =,\t\r\n' read -a lne;do
          [ "$lne" = "data" ] &&
              fsbs=${lne[2]} fssz=${lne[4]}
      done < <( xfs_info $mntpnt )
      fssize=$((fssz*fsbs))
      
    3. for ntfs:

      Things seem not as easy for this ... As I don't use them a lot, even if I've already used ntfsresize in the past. There is something:

      read fsinfo < <(file -bs $1)
      fssz=${fsinfo#*, sectors } fssz=${fssz%%,*}
      fssize=$(( 512 * ( fssz + 1 ) ))
      

      Nota: I hate this: I could not explain + 1 and some of my test work with + 8 instead...

Then finally

printf "Dev: %d - FS: %d = Unused: %d\n" $devsize $fssize $(( devsize - fssize ))

Script using lsblk for scanning all devices:

This script, run without argument, will print one line by device having diff non null. Run with any (dummy) argument, will print one line for each xfs or ext[234] device.

As this use with efficient bashism and limited forks to lshw, dumpe2fs and/or xfs_info only, this script present a very small resources footprint.

!! To be run as root or with sudo !!

#!/bin/bash
shopt -s extglob
declare -A KNAMES='()' # Prevent double check
infolog() {
    # Entering this with $kname, $size, $fs, $name, $fsbs and $fssz.
    fssize=$((fssz*fsbs))
    (( $# | size-fssize ))&& # Only if diffsize > 0 or if argument was submited
        printf "%-25s %-6s %15u %15u %14u\n" \
               ${NAME##*/} $FSTYPE $SIZE $fssize $((SIZE-fssize))
}
while read line; do  eval "$line"
    [ "${KNAMES[${KNAME##*/}]}" ] || case $FSTYPE in 
        ext* ) while IFS=$':\t\r\n' read fld val;do
                case $fld in
                    Block\ size)  fsbs=$val;;
                    Block\ count) fssz=$val;;
                esac
            done < <(dumpe2fs 2>&1 -h $KNAME)
            infolog $@ ;;
        xfs ) check=$KNAME
            [ "$MOUNTPOINT" ] && check="$MOUNTPOINT"
            while IFS=$' =,\t\r\n' read -a lne;do
                [ "$lne" = "data" ] &&
                    fsbs=${lne[2]} fssz=${lne[4]}
            done < <( xfs_info "$check" )
            infolog $@ ;;
        ntfs )
            read fsinfo < <(file -Lbs $KNAME)
            fssz=${fsinfo#*, sectors } fssz=${fssz%%,*}
            (( fssz += 1, fsbs=512 ))
            infolog $@ ;;
    esac
    KNAMES[${KNAME##*/}]="done"
done < <(lsblk -bnpPo KNAME,NAME,MOUNTPOINT,FSTYPE,SIZE)

( This is a little shrinked to prevent SO's scrollbars;)

Sample run (without argument):

./unused_bytes_by_dev.sh
Vol--I-XDATAS             xfs        48318382080     42949672960     5368709120
sdd1                      ext4        1010774016       175472640      835301376

With one argument:

./unused_bytes_by_dev.sh dummy
md0                       ext4         246349824       246349824              0
Vol--I-ROOT               ext4        1610612736      1610612736              0
Vol--V-USR                ext4       31138512896     31138512896              0
Vol--I-VAR                ext4       22548578304     22548578304              0
Vol--I-HOME               ext4       34359738368     34359738368              0
Vol--I-XDATAS             xfs        48318382080     42949672960     5368709120
Vol--I-Win                ntfs       42949672960     42949672960              0
sdd1                      ext4        1010774016       175472640      835301376
0

df gives you total size as seen by FS, with -k — in KB.

sudo blockdev --getsize64 gives you pure block device size — in bytes.

It's pretty simple to convert bytes to KBs and compare if there's any significant difference between those two.

-2

To get unused bytes, you should get partition table, and take a look at boundaries of partitions and boundaries of filesystem.

There are multiple ways how to do that, so I am going to name a few of them

fdisk

This needs some calculation. You can see how many sectors there are on the disk, and you can see number of sector at the beginnig and at the end of every partition, so you can calculate if there is any free space left.

# fdisk -l /dev/nvme0n1
Disk /dev/nvme0n1: 477 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: THNSN5512GPUK NVMe TOSHIBA 512GB        
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: B9E14CDC-1817-4552-9AD0-967A6FC9C4CE

Device            Start        End   Sectors  Size Type
/dev/nvme0n1p1     2048     411647    409600  200M EFI System
/dev/nvme0n1p2   411648    2508799   2097152    1G Linux filesystem
/dev/nvme0n1p3  2508800   18782207  16273408  7.8G Linux filesystem
/dev/nvme0n1p4 18782208 1000214527 981432320  468G Linux filesystem

parted

You can invoke parted to print free space in the partition table

# parted /dev/nvme0n1 print free
Model: THNSN5512GPUK NVMe TOSHIBA 512GB (nvme)
Disk /dev/nvme0n1: 512GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name                  Flags
        17.4kB  1049kB  1031kB  Free Space
 1      1049kB  211MB   210MB   fat16        EFI System Partition  boot, esp
 2      211MB   1285MB  1074MB  ext4
 3      1285MB  9616MB  8332MB
 4      9616MB  512GB   502GB
        512GB   512GB   335kB   Free Space
1
  • guettli was talking about discrepancies between block device size and file system size. And not about the unalocated disk space on the partition table. Aug 20, 2019 at 12:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .