What is Fragmentation?

When a filesystem (FS) is created on a Unix operating system, it allocates contiguous space to the FS. When a file is created, sequential units are allocated so that the seek operation will be quick. In the case of a fragmented filesystem, there are many non-contiguous units in the file system.

For example: If there are 5 files on the FS name - a,b,c,d and e, and you delete a file 'e' in the middle of the FS (meaning that it's not the last file in the FS, you can verify by inode of the file) which has a size of 2MB, there will be free space available for 2MB in the FS. If you create a new file of size 5MB, it's not going to use the 2MB which was previously allocated. Instead, it will try to assign the contiguous space of 5MB and there will be a free space of 2MB in the middle of the FS.

When these types of non-contiguous space increases, fragmentation occurs.  We will be creating a FS, see how to make the FS fragmented, and then we will defrag it.



1. A VxFS filesystem was created and mounted at /datavol.  We use fsadm to record the segmentation statistics for it.

# fsadm -F vxfs -E /datavol
 Extent Fragmentation Report
        Total    Average      Average     Total
       Files    File Blks    # Extents   Free Blks
           0           0           0     4175784
   blocks used for indirects: 0
   % Free blocks in extents smaller than 64 blks: 0.00
   % Free blocks in extents smaller than  8 blks: 0.00
   % blks allocated to extents 64 blks or larger: 0.00
   Free Extents By Size
          1:          0            2:          0            4:          2
          8:          2           16:          1           32:          2
         64:          1          128:          2          256:          2
        512:          0         1024:          1         2048:          0
       4096:          1         8192:          1        16384:          0
      32768:          1        65536:          1       131072:          1
     262144:          1       524288:          1      1048576:          1
    2097152:          1      4194304:          0      8388608:          0
   16777216:          0     33554432:          0     67108864:          0
  134217728:          0    268435456:          0    536870912:          0
 1073741824:          0   2147483648:          0


2. Now we will create some big files and delete them so that there will be free space in the middle of the file system.

# dd if=/dev/zero of=/datavol/temp
^C2350465+0 records in
2350465+0 records out

# dd if=/dev/zero of=/datavol/temp2
^C4858753+0 records in
4858753+0 records out


# dd if=/dev/zero of=/datavol/temp3
write: No space left on device
1117841+0 records in
1117841+0 records out

- Deleting the files so that there is a gap in between the two files:

eg: File name Inode
f1 812
f2 813
f3 814 

Delete the file f2 so that there is a gap between f1 and f3.

# rm -rf /datavol/X11
# rm -rf /datavol/cacao

3. With fsadm, we can now see the fragmentation statistics for the file system:

# fsadm -F vxfs -E /datavol/
 Extent Fragmentation Report
        Total    Average      Average     Total
       Files    File Blks    # Extents   Free Blks
         165        3163           2         117
   blocks used for indirects: 2
   % Free blocks in extents smaller than 64 blks: 100.00 <<<<
   % Free blocks in extents smaller than  8 blks: 17.95 <<<<
   % blks allocated to extents 64 blks or larger: 99.84 <<<<
   Free Extents By Size
          1:          5            2:          2            4:          3
          8:          2           16:          3           32:          1
         64:          0          128:          0          256:          0
        512:          0         1024:          0         2048:          0
       4096:          0         8192:          0        16384:          0
      32768:          0        65536:          0       131072:          0
     262144:          0       524288:          0      1048576:          0
    2097152:          0      4194304:          0      8388608:          0
   16777216:          0     33554432:          0     67108864:          0
  134217728:          0    268435456:          0    536870912:          0
 1073741824:          0   2147483648:          0

Note: To see if the FS is fragmented, one or more of the following conditions should be met:

% Free blocks in extents smaller than 64 blks: 100.00 =>100/100 >.5 OR
% Free blocks in extents smaller than  8 blks: 17.95 =>17.95/100 >.05 OR
% blks allocated to extents 64 blks or larger: 99.84 =>99.84/1000 <.05

 

 An unfragmented file system has the following characteristics:

• - Less than 1 percent of free space in extents of less than 8 blocks in length
• - Less than 5 percent of free space in extents of less than 64 blocks in length
• - More than 5 percent of the total file system size available as free extents in lengths of 64 or more blocks

A badly fragmented file system has one or more of the following characteristics:

• - Greater than 5 percent of free space in extents of less than 8 blocks in length
• - More than 50 percent of free space in extents of less than 64 blocks in length
• - Less than 5 percent of the total file system size is available as free extents in lengths of 64 or more blocks

3. Now we are going to Defrag it:

# fsadm -F vxfs -de /datavol

We will check the defragmentation again:-

# fsadm -F vxfs -E /datavol

 Extent Fragmentation Report
       Total    Average      Average     Total
       Files    File Blks    # Extents   Free Blks
         167        2707           1         518
   blocks used for indirects: 4
   % Free blocks in extents smaller than 64 blks: 38.22 <<<<<
   % Free blocks in extents smaller than  8 blks: 5.79 <<<<<
   % blks allocated to extents 64 blks or larger: 99.95 <<<<<
   Free Extents By Size
          1:          4            2:          5            4:          4
          8:          3           16:          3           32:          3
         64:          1          128:          2          256:          0
        512:          0         1024:          0         2048:          0
       4096:          0         8192:          0        16384:          0
      32768:          0        65536:          0       131072:          0
     262144:          0       524288:          0      1048576:          0
    2097152:          0      4194304:          0      8388608:          0
   16777216:          0     33554432:          0     67108864:          0
  134217728:          0    268435456:          0    536870912:          0
 1073741824:          0   2147483648:          0



Note the specific lines starting "% Free blocks" in the - as you can see FS is defragmented.  

Defragmentation:

During defragmentation fsadm reads the memory map from the mmap syscall and then tries to do the unmap of the memory by the munmap sys call. Once the unmapping is done it resyncs the file descriptor cache with the FS by using the fdsync routine. There is no physical movement of the data that takes place. To make sure that the newly organized space by fsadm is available to the next file allocation unit we need to do the following:
 

This will ensure that all the free space is now available at the end of the FS, and if any file is created, the FS will allocate the space from the end of the FS.