Could you please help me to understand 'direct-io-mode' in GlusterFS.

I'm getting better read results with direct I/O mode disabled - is it mean that in disabled mode it's writing/reading from cache - is it a system cache or GlusterFS cache? What's the recommended method? I'm using 4 nodes with distributed-replicated volume for web serving:

Write: dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc
echo 3 > /proc/sys/vm/drop_caches
Read: dd if=tempfile of=/dev/null bs=1M count=1024
Read-Read: dd if=tempfile of=/dev/null bs=1M count=1024

Write           Read            Re-Read
67.5 MB/s       94.9 MB/s       94.6 MB/s   direct-io-mode=enabled
65.8 MB/s       230 MB/s        226 MB/s    direct-io-mode=disabled

after a long search i've found this article. it's not glusterfs specific, but it may help to understand better: in short, direct i/o mode disable the file buffer implemented by the os for that mount point.

after reading it i've come to this conclusion:

Enabling or disabling may give you better transfer rate depending on which application is using your glusterfs volume: if you are using applications or databases that handle by themselves caches you are going to have some boost enabling it ( because of the missing overhead of a second buffer ). if you are just doing file accesses probably disabling it will increase your performances because of less network access due to local file cache.

  • 3
    What is direct-io-mode set to by default if you don't pass it in? Apr 6 '18 at 8:15

You enable direct-io-mode by"mount -t glusterfs XXX:/testvol -o direct-io-mode=enable mountpoint"??If so, the file was not cached in GlusterFS client kernel,in other words,the cache is system cache.But,the file was cached in GlusterFS server kernel.

  • 1
    Welcome to serverfault! Please take the guided tour. You have a question about how the direct-io-mode was enabled; it should be in a comment to the question, not an answer. I don't see that the rest of your post is an answer to the question. To answer questions usefully, it would be better to concentrate on recent unanswered questions than three-year-old questions with an accepted answer.
    – Law29
    Jun 19 '16 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.