Fresh vs. rotten ext3

Did you ever hear sentences like “Linux/Unix filesystems are superior, to stuff like NTFS, let alone FAT32 – you don’t even need a defragmentation tool.”?

That statement may be technically correct, since fragmentation is really rare with ext3 – but what about spatial locality of reference – files that are often accessed at nearly the same time being spread over the whole disk, thus causing long access times due to head positioning?
There is (AFAIK) no way to (programmatically) optimise / sort a Linux filesystem in a way that for example all init scripts, binaries, libraries, etc. reside in nearby sectors on a harddisk. Those tools do exist in the Windows world (built into those 3rd party defragmenters).

The only way to do this optimisation is “by hand”: make a backup, mkfs and restore the backup – but that’s something you only do when you have a lot of time, or when I/O finally got painfully slow.

One factor that probably plays an important role, is, that I usually only have one partition (+swap) that contains everything – etc, usr, usr/portage, var, home, …
With such a setup, after only a few months of updating the system, downloading stuff from the net, copying pictures from the DSLR to the harddisk, updating the system again, etc. will lead to all kinds of files being well spread throughout the whole disk. Imagine some init scripts and binaries they call still being at the “beginning” of the disk, some others, due to updates, at the “end”. And that’s what makes I/O slow… see for yourself:

From loading the kernel to KDM being ready for login on a

• fresh ext3 – 32 seconds
• rotten ext3 – 57 seconds

(sorry for the poor video quality)

That is an ~80% performance decrease.

How I got the videos:

1. Capture startup of my system stored on a months old ext3 partition
2. Backup (with dar)
3. mkfs.ext3
4. Restore
5. Capture startup of my system on a freshly made ext3 partition
6. Trim the videos so that frame 1 is the first frame on which kernel output is visible and the last frame is the first frame on which KDM shows the input widgets

Hoping that this situation will improve with ext4 – I heard online defragmentation will be possible at some point, and that probably also makes “sorting” the filesystem possible.