8

I am planning to test a filesystem. What all should i consider to do. or how to accomplish the same. Are there are any tools to test a filesystem or any reliable references?

Purpose: Planning to shift from ext3 to EFS.

Improve this question
1

6 Answers 6

Reset to default
5

My first answer would also be bonnie++, a quick search also reveals Iozone3. Both are available as Debian packages. If you do simpler tests (cp, mv, rm), you should consider clearing the cache kept by the kernel with echo 1 > /proc/sys/vm/drop_caches before each test otherwise it might cause differences that are not representative of the filesystem itself.

Improve this answer
3
  • 1
    used bonnie++ for our requirements. proved beneficial. Also used rugg.
    – Viky
    Jun 30, 2009 at 8:45
  • Actually, I recommend sync && echo 3 > /proc/sys/vm/drop_caches instead to flush dirty pages first and then free pagecache, dentries and inodes afterwards (because drop_caches will only drop non-dirty data). The number is a bit mask: 1 == pagecache 2 == dentries and inodes 3 == pagescache + dentires and inodes
    – knweiss
    Jul 5, 2009 at 19:54
  • 1
    While I understand that to make tests comparable you want the same starting point (empty caches/buffers), it begs the question how representative is that of the real world scenarios? In the real world do you clean caches before every batch of work, or are you trying to leverage all the caching, buffering, preloading and whatever else you can conjure up? Filesystem testing is an interesting conundrum, because the two main principles of testing are at odds--repeatability and representation of real world conditions are orthogonal. So you want to run two set of tests: synthetic, and real world.
    – Marcin
    Jul 29, 2009 at 13:13
3

Here are two more options, I've found them really useful for filesystem benchmarking.

1) Postmark - it's a benchmark which emulates real-world usage of a filesystem accessed by a busy mail server.

Download: http://www.freshports.org/benchmarks/postmark/

When testing, make sure you've compared different options of ext3 logging - they may be quite different in terms of performance (writeback logging gave best result in my Postmark experiments).

2) Filebench - another great benchmark, giving you even more flexibility. If you have a specific reason for considering a move from ext3, you'll probably like filebench - it has many different workloads so that you can only concentrate on the performance bottleneck which interests you (file create/delete operations or sequential read/write vs random ones, to give you a few examples).

You have to google for it as I'm not allowed any more hyperlinks yet - the project itself is hosted on SourceForge, and a really nice quick start guide is found on OpenSolaris.org website.

One final note: be sure to spread the load across as many disks (spindles) as possible for the most accurate results. It also makes sense to allocate a few disks for benchmarking specifically, and newfs them before each run of your benchmark.

Improve this answer
3

Don't just test for speed, also consider reliability. Try to, e.g., power down disks on a busy filesystem and see what's left.

The quality of the repair and recovery tools available is also important, and very hard to test discretely. Block structure may e.g. inhibit tools that try to salvage data in raw mode from a unsalvageable filesystem.

For more hints on testing a filesystem under very harsh beatings you may be interesting what the ZFS guys did: One Two

Improve this answer
1
  • +1. This is one of the things I had on my list to check.
    – Viky
    Jun 4, 2009 at 9:43
1

We use Rugg to test the file systems of our servers. It's Python based and easy to use. It works by creating a large file which it subdivides, then performs various operations on in sequence or parallel.

Edit: Here is an example script that will test /data (a partition) repeatedly until the script is killed:

#!/bin/bash

(
while true
do
  cd /data
  rm -rf *
  rugg 'zone 97%, subdivide 16 | blank | ensure blank | fill same text, ensure same'
done
) | tee /tmp/rugg.log

exit 0
Improve this answer
1
  • +1. Checked rugg documentation. looks promising. Will try this along with bonie++ suggested by other members.
    – Viky
    Jun 5, 2009 at 3:33
1

Some more:

  • fio which is similar to filebench mentioned above: "fio is an I/O tool meant to be used both for benchmark and stress/hardware verification. It has support for 13 different types of I/O engines (sync, mmap, libaio, posixaio, SG v3, splice, null, network, syslet, guasi, solarisaio, and more), I/O priorities (for newer Linux kernels), rate I/O, forked or threaded jobs, and much more. It can work on block devices as well as files. fio accepts job descriptions in a simple-to-understand text format. Several example job files are included. fio displays all sorts of I/O performance information. It supports Linux, FreeBSD, and OpenSolaris."
  • File System Exerciser (fsx) originally from Apple but nowadays there are variants for various OSes.
Improve this answer
0

While bonnie++ is primarily focused on bench-marking performance it does work above the filesystem. It performs some tests that give different results per-filesystem. I suspect you are probably interested in the change in performance related to EFS as well, so you may want to try that at least as one of your tests.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .