IO benchmarks refer to the computerized test for measuring various properties of an IO device under multiple conditions. Various features include Read/Write speed, IOPS, seek time, etc and various conditions refer to random/sequential reads/writes, block size used by the filesystem, buffer cache size, etc. There are multiple IO benchmarks available online, but the top 3 opensource IO benchmarks, in my view, include FIO, IOZONE, and BONNIE++. There are several pros and cons for each of them, which we will be discussing in this blog post.
Fio (Documentation, Repository)
FIO is an I/O tool meant to be used both for benchmark and stress/hardware verification. It has support for 13 different types of I/O engines (sync, mmap, libaio, posixaio, SG v3, Omega Replica Watches splice, null, network, syslet, guasi, solarisaio, and more), I/O priorities (for newer Linux kernels), rate I/O, forked or threaded jobs, and much more. It supports Linux, FreeBSD, NetBSD, OS X, OpenSolaris, AIX, HP-UX, and Windows.
Example: sudo fio –name=randwrite –ioengine=libaio –iodepth=1 –rw=randwrite –bs=4k –direct=0 –size=512M –numjobs=2 –runtime=240 –group_reporting
Bonnie++
Greatly improved disk I/O benchmark based on the code for Bonnie by Tim Bray. Bonnie++, rewritten in C++ by Russell Coker (russell@coker.com.au), adds the facility to test more than 2Gb of storage on 32-bit machines, and tests for file create(), stat(), unlink() operations.
Example: bonnie++ -d /tmp -s 4G -n 0 -m TEST -f -b -u udit
IOzone (Documentation, Repository)
The benchmark generates and measures a variety of file operations. IOzone is useful for performing broad filesystem analysis of a vendor’s computer platform. It can also be used for NFS performance testing and the latest version now supports cluster testing as well. IOzone had been ported to numerous UNIX OSes and Windows NT.
Example: iozone -e -I -a -s 100M -r 4k -r 512k -r 16M -R -i 0 -i 1 -i 2 -f /dev/sdda0s1
Comparison
| IOZONE | FIO | BONNIE++ | |
|---|---|---|---|
| Activeness in development |
High | Very High | Moderate |
| I/O buffer types available |
direct, buffered | direct, atomic, buffered | Direct, Buffered |
| IO R/W available | Read, write, random read, random write, reread, rewrite, strided read/write, Random Mix, Backwards read, Fread, Fwrite, Freread, Frewrite |
Read, write, random read/write trim, random trim, readwrite, random readwrite, trimwrite |
Read/write, random read/write |
| Special type of IO | Mmap, Async I/O | sync, psync, vsync, pvsync, pvsync2, libaio, posixaio, solarisaio, windosaio, mmap, sg, net, netsplice, cpuio, guasi, rdma, falloc, ftruncate, e4defrag, rados, rbd, gfapi, gfapi_async, and lot more. |
None |
| Types of random datasets generated for random R/W |
Uniform | Uniform, gaussian, pareto, ziph, zoned, absolute zoned |
uniform |
| Random number generating engine supported |
Fisher yates algo | tausworthe, lfsr, tausworthe64 | – |
| Flexibility | Quite customizable | ‘Flexibility’ is in its name! It’s Highly configurable, with number of available modes. |
mostly automatic. Very less configurable |
| License type | customized | GPLv2 | GPLv2 |
| Benchmark type | Filesystem benchmark that can benchmark Storage devices too |
Specifically built for benchmarking storage device, but can be used for filesystem benchmarking too |
Filesystem benchmark |
| Statistics available as output |
Amount of I/O generated in KBps | Max. Min. Avg latency with standard deviation, Average bandwidth, IOPS , amount of I/O generated in KBps, average queue depth |
Amount of I/O generated in MBps available in CSV format |
| Others | Stonewalling to synchronize threads | Pattern verification available; user configurable error handling, clock settings etc; |
– |
Conclusion
Although it’s pretty clear from the above comparison that Fio fits perfectly for benchmarking storage devices, IOzone and Bonnie++ are still used for applications that don’t require too many features, just simple read/write. quick benchmarking. Further, for any naive user, getting grip on IOzone is much much easier than struggling with complex features of Fio.