Filesystems Benchmarked
Linux Filesystem Performance Tests
ByChoosing the right filesystem for a particular job can be a difficult task. We tested seven candidates and found some interesting results to make an administrator’s choice easier.
Admins building a new server or setting up a new storage device have to make important decisions on the basis of predicted usage. Does the server need to write millions of files quickly to keep the backup window manageable, or does it need to deliver multiple-gigabyte virtualization images within seconds?
The choice of filesystem can make a huge difference in such cases, so Linux Magazine put seven filesystems through numerous tests. The results of the benchmarks in this article could help readers choose the most appropriate filesystem for the task at hand.
The tests were performed on a Transtec Calleo appliance (see the “Test Hardware” box) with eight fast disks in a RAID level 0 array with a stripe size of 64KB. The RAID is divided into an SSD array and an HDD array with identical partitions on both arrays, thus also taking partition alignment, which is relevant for the hard disk’s performance, into account. The partition, therefore, starts at LBA address 2048.
Operating and Filesystems
The operating system used in the test was SUSE 12.1 with the latest updates and kernel 3.1.10. For the comparative tests with kernel 3.3.6 (from 12-5-2012 ), my choice fell on the largely identical rolling release, also based on SUSE 12.1, named openSUSE Tumbleweed.
The seven most widely used filesystems under Linux were the focus:
- ext2: The oldest ext filesystem, without journaling, and correspondingly with little protection against data loss if worst comes to worst.
- ext3: The long-term standard in many distributions.
- ext4: The latest ext standard since kernel 2.6.28.
- Btrfs: The newcomer, with properties similar to ZFS.
- XFS: A journaling filesystem developed by SGI.
- ReiserFS: The filesystem originally initiated by Hans Reiser.
- ZFS: A filesystem from the Sun universe that is considered by many experts to be the most advanced.
Note that ZFS, however, is not a competitor here, because it only recently became natively available on Linux – thanks to the “ZFS on Linux” project. Developer Paolo Pantò recently added it to the software index of the SUSE Build System.
Nevertheless, long-term historic data, which is very important for admins, still doesn’t exist. This makes ZFS potentially dangerous, in spite of all the advantages and its good reputation, because version 0.6 of the code, for example, still cannot offer format assurances.
If you are looking to compare filesystem speeds, you should not set too much store by absolute figures – that is, the net megabytes per second (MBps) in the graphics or the run times in the benchmarks – because the values depend too greatly on the specific hardware in use.
A direct comparison between the candidates is much more definitive, especially for sequential or random read and write operations. Making these measurements is the task of IOzone; Figures 1-4 show the most important results. The complete set of raw data and the scripts are available online.
![](/var/linux_magazin/storage/images/media/linux-magazine-eng-us/images/abb1_eng/542203-1-eng-US/abb1_eng_mainstory1.jpg)
![](/var/linux_magazin/storage/images/media/linux-magazine-eng-us/images/abb2_eng/542206-1-eng-US/abb2_eng_mainstory1.jpg)
![](/var/linux_magazin/storage/images/media/linux-magazine-eng-us/images/abb3_eng/542209-1-eng-US/abb3_eng_mainstory1.jpg)
![](/var/linux_magazin/storage/images/media/linux-magazine-eng-us/images/abb4_eng/542212-1-eng-US/abb4_eng_mainstory1.jpg)
All Buffers Off!
To ensure that no unwanted optimizations by buffers or caches on the test system distorted the results, adding different levels of variance depending on the filesystem, I disabled both the RAID controller cache and all the HDD caches. Additionally, before each test, I ensured that the Linux kernel itself had no chance to perform any optimizations by serializing I/O operations through buffers and caches. To do this, I ran sync and doubly deleted the page cache, inodes, and dentries (with echo 3 > /proc/sys/vm/drop_caches).
The benchmark script prepared each filesystem for the benchmark run with the appropriate mkfs tool (on ZFS, this was zpool) without further options. Also, the countless possibilities for optimization were ruled out in the tests (e.g., btrfs -o SSD can increase the throughput in individual cases by up to 30 percent).
In addition to the IOzone tests, I performed a long series of my own benchmarks, including mkdir, touch, echo, cat, dd, rm, and rmdir in long loops with meaningful values.
Ranking and Kernel Test
If you simply assign points on an equal footing for placement in each of the many individual disciplines, you receive interesting results: XFS scores 56 points ahead of ReiserFS and ext4, followed by Btrfs and ext2. If you consider only the more atomic tests from Bash scripts, XFS is still in the lead ahead of Btrfs and ReiserFS.
In the IOzone benchmark results, which are based exclusively on data throughput, however, ext3 wins by a large margin (32 points), with ReiserFS (26), XFS (25), and ext4 (24) lagging a bit behind.
As Figures 5 and 6 show, migrating to the latest kernel doesn’t always mean benefits.
![](/var/linux_magazin/storage/images/media/linux-magazine-eng-us/images/abb5_eng/542215-1-eng-US/abb5_eng_mainstory1.jpg)
![](/var/linux_magazin/storage/images/media/linux-magazine-eng-us/images/abb6_eng/542218-1-eng-US/abb6_eng_mainstory1.jpg)
In a test with kernel 3.3.6 (current when this issue went to press) on SUSE Tumbleweed, performance in almost all filesystems declined compared with kernel 3.1.10. The ext filesystems only benefited in the random read discipline. In all other disciplines, the filesystems performed more poorly with kernel 3.3.6 than kernel 3.1.10.
Ext-ra Special: XFS
XFS is obviously still a good choice despite its age. On SSDs and HDDs, it delivers fast atomic actions and stable values in the IOzone benchmark. Larger files seem to be a problem. Another interesting result is that XFS seems to have improved on SSDs between kernels 3.1.10 and 3.3.6.
Taking the silver medal, ext3 impresses in the IOzone benchmark. In contrast, the filesystem is not so good at handling atomic file system operations. Ext4 was clearly better, but, unfortunately, at the expense of throughput, which explains why ext4 only took bronze in the overall standings. The legacy ext2 impresses with its performance in random write and is one of only a few to benefit from the new kernel 3.3.6.
ReiserFS performed very well on average, achieving stable values in the atomic filesystem operations. However, its sequential write performance is disconcerting: It was several hundred percent behind all the other candidates, and, although this is not reflected directly in the points score, it could very well be a problem in everyday life.
ZFS on Linux is merely another tail-ender; however, it is the undisputed leader on both kernel versions when it comes to sequential writing of large blocks in large files. In contrast, Btrfs was not only the filesystem to suffered the most on the new kernel (up to 40 percent less throughput), it proved to be a disappointment overall. The young system only set high standards in sequential reading from HDDs.
SSD or HDD?
Regardless of filesystem, the difference between SSD and HDD throughput rates for writing and, even more so, in random tests is significant. This finding confirms the recommendation to use SSDs for systems with high random read/write traffic, regardless of the filesystem because they do not suffer from the mechanical latencies that slow down hard drives.
In the test scenario here, just two SSDs was all it took to outperform a RAID array with six disks. The throughput in random write operations was, in some cases, almost three times that of the hard disks. That said, however, traditional disks in a RAID were able to match SSD performance in terms of sequential transfer rates.
Info
[1] Transtech Lynx Calleo Application Server
The Author
Michael Kromer is Head of Professional Services, Germany, with Zarafa. Among other things, he is responsible for Zafara’s large-scale implementations and architectures. In his free time, Michael is involved in various open source projects (e.g., the Linux kernel, OpenNX, Asterisk, VirtualBox, ISC BIND, and openSUSE) and all kinds of virtualization technologies and processor architectures.
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
![Learn More](https://www.linux-magazine.com/var/linux_magazin/storage/images/media/linux-magazine-eng-us/images/misc/learn-more/834592-1-eng-US/Learn-More_medium.png)
News
-
OpenMandriva Lx 24.07 Released
If you’re into rolling release Linux distributions, OpenMandriva ROME has a new snapshot with a new kernel.
-
Kernel 6.10 Available for General Usage
Linus Torvalds has released the 6.10 kernel and it includes significant performance increases for Intel Core hybrid systems and more.
-
TUXEDO Computers Releases InfinityBook Pro 14 Gen9 Laptop
Sporting either AMD or Intel CPUs, the TUXEDO InfinityBook Pro 14 is an extremely compact, lightweight, sturdy powerhouse.
-
Google Extends Support for Linux Kernels Used for Android
Because the LTS Linux kernel releases are so important to Android, Google has decided to extend the support period beyond that offered by the kernel development team.
-
Linux Mint 22 Stable Delayed
If you're anxious about getting your hands on the stable release of Linux Mint 22, it looks as if you're going to have to wait a bit longer.
-
Nitrux 3.5.1 Available for Install
The latest version of the immutable, systemd-free distribution includes an updated kernel and NVIDIA driver.
-
Debian 12.6 Released with Plenty of Bug Fixes and Updates
The sixth update to Debian "Bookworm" is all about security mitigations and making adjustments for some "serious problems."
-
Canonical Offers 12-Year LTS for Open Source Docker Images
Canonical is expanding its LTS offering to reach beyond the DEB packages with a new distro-less Docker image.
-
Plasma Desktop 6.1 Released with Several Enhancements
If you're a fan of Plasma Desktop, you should be excited about this new point release.
-
SUSE Offers CentOS 7 Support with Liberty Linux Lite
SUSE's Liberty Linux support offering now includes CentOS 7, which means businesses won't be forced to migrate those servers for some time.