Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

The partition is in RAID1 using software RAID.

    Command line used: iozone -s 190g -r 300k -O -b test_file
    OPS Mode. Output is in operations per second.
                                                            random  random    bkwd   record   stride                                   
          KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
   199229440     300     240     234      254      253      58     127     100     6539       91      234      234     254      254

so for 300K blocks random read using one thread was 58 ops/second of about 17MB/s.

But when I tried 5 threads each reading 300K blocks, unexpectedly for me it worked faster, although I'd expect more random access:

    OPS Mode. Output is in operations per second.
    Command line used: iozone -s 80g -r 300k -O -l5 -u 5 -F test_file1 test_file2 test_file3 test_file4 test_file5

    Children see throughput for 5 random readers    =      69.27 ops/sec
    Parent sees throughput for 5 random readers     =      69.27 ops/sec
    Min throughput per process                      =      13.81 ops/sec 
    Max throughput per process                      =      13.89 ops/sec
    Avg throughput per process                      =      13.85 ops/sec

Does anyone knows what reading/caching policies in Linux OS or hard drives makes 5 thread reading random 300K blocks faster than 1 thread reading random 300K blocks?

share|improve this question
    
The subject says 300k files (unspecified size) and the question says 300k blocks. Did you do the reads to a single large file (which had at least 300k blocks)? Or something else? Can you provide more specific detail of the files that were created for the test, and the size of the entire potential surface? –  Matthew Ife Apr 13 at 15:50
    
it's reading blocks of 300K. iozone creates files and read blocks of specific size, it is a HD benchmark software. –  Mladen Adamovic Apr 13 at 17:50
    
How come you have different output for two tests? The first test suggests you were doing random writes yet the second test does not indicate that? –  Matthew Ife Apr 13 at 22:17
    
I've been able to duplicate this on my own machine using fio. What kernel version are you using? –  Matthew Ife Apr 13 at 23:15
    
3.2.0-60-generic I think the underlying reason might be RAID1 with partition, with 2+ threads one thread reads from one drive, another from another drive (or something like that theoretically). –  Mladen Adamovic Apr 14 at 21:51

2 Answers 2

What is surprising?

1 thread means command, wait, read, loop. There is a wait element in - the latency.

Multiple threads eliminate that and - allow the discs to be more optimized thanks to cueueing of multiple commands. That you get more IOPS with 3 threads is not surpsiing at all.

share|improve this answer
    
This might be correct. fread and read has a parameter which specifies file size, so hard drive controller can know for each block which clusters it has to read. When having multiple threads reading, it can benefit from optimizing the head movement between each blocks. –  Mladen Adamovic Apr 16 at 7:08
    
Even if not. There is a thing called NCQ (SATA) or TCQ (SAS) - drives get multiple commands. Multiple threads thus can be handled in a more optimal fashion. Stnadard Admin 101 knowledge. –  TomTom Apr 16 at 7:33

It's a raid1, so writes go to both disks, but reads can be satisfied with one, so with more threads, I'd expect better performance. But it might be best with threads equal to the number of disks in the raid 1 (or the raid 1 part of a raid 10).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.