Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Whenever I read a large file from my Samba server I get around 40 MB/sec transfer rate. If I read the very same file again, the speed is all of a sudden bumped up to 70 MB/sec.

Why do I not get 70 MB/sec the first time around? The disks easily perform up to 95 MB/sec. Does Samba have to cache the entire file in RAM to get performance or something?

When writing files to the server I get 95+ MB/sec every time, quite the difference.

I have tried setting some socket options (TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=65535 SO_RCVBUF=65535), but they don't seem to do much to help.

So, any tips on how to improve read performance in Samba?

Note: The disks are 2x1TB Samsung Spinpoint F1 7200 RPM configured in software RAID 1.

Update: It seems the client had as much, if not more, to do with this issue as the server. I was using my trusty old Total Commander, which seems to be too old these days. When I copy files using Windows Explorer instead, even uncached files are read pretty fast. Thanks for your help.

share|improve this question
    
What version of Samba are you using? –  3dinfluence Aug 24 '09 at 19:13
    
At first I was using 3.2.5 but I tried upgrading to 3.3.4, made no difference in speed. The client is a Windows 7 machine, I wonder if this could be affecting my results due to some kind of incompability between the two. –  weazl Aug 24 '09 at 19:40
add comment

4 Answers

up vote 1 down vote accepted

Till now all answer are more related to the discs than to the RAID configuration. Maybe question 19 and 4 of this guide can help you: Software RAID HOWTO.

Another thing is the network side. Do you have TOE enabled on your NIC?

And the last thing: Did you check that your bottleneck is not on the client side? It could be that your FTP client is keeping more data in the RAM than the CIFS service does. And that's maybe one reason why FTP is faster.

share|improve this answer
    
As it turns out, it was the client software in this case. –  weazl Aug 24 '09 at 22:24
add comment

First off those TCP socket options were meant for 2.4 kernels and on the Samba mailing list the developers have repeatedly said that they make no sense on 2.6 kernels.

Beyond that something isn't right with your numbers here. There is no way that 2 SATA drives in a RAID1 (mirror) configuration are going give you write speeds of 95MB/s and I highly doubt you'll see read speeds that high either. Except maybe on the very outside track of the drive. How are you bench marking your RAID volume? Keep in mind that dd isn't a filesystem benchmark.

Gigabit speeds also can eat up a significant amount of CPU cycles if you're NICs are consumer grade. So if you have a slower cpu in the system don't rule out cpu as a bottle neck.

Also keep in mind that the disks and cpu in the Server and Client here both have to be able to sustain the speeds you're trying to achieve so don't just look at the server as the source of the bottle neck as it is just as likely to be on the client side.

share|improve this answer
    
I've benchmarked the RAID array using ´hdparm -t /dev/md1´, I'd be happy to try other suggestions. Also, the client is more than twice as fast CPU-wise. The server has a Core 2 Duo 1,83 GHz CPU whereas the client has a Core 2 Quad 3,2 GHz CPU. The NICs could be an issue, but they wouldn't explain why the second read is so much faster. –  weazl Aug 24 '09 at 14:09
    
The second read being much faster definitely points to the bottle neck being in the disk subsystem. Both the server and client should be fast enough cpu wise to deal with the GigE speeds. I think the difference in speed that you're seeing between FTP and CIFS is just the over head associated with CIFS...it's a pretty chatty protocal. I would try Bonnie+ or IOZone to get a better benchmark of your storage and filesystem though. At least then you'll have a real world baseline. –  3dinfluence Aug 24 '09 at 14:20
    
pastebin.com/m4f624a9a If I read these results correctly, I get about 74 MB/sec sequential read which would be in line with that I see when transfering files using FTP. –  weazl Aug 24 '09 at 17:37
    
Yeah looks like 74MB/sec read and 65MB/sec write. Not too bad for a couple of SATA drives in RAID1. I'm not sure what more to tell you other than your expectations are pretty high for such a small amount of hardware. Cifs has a much higher overhead, and features, compared to ftp. I'm not saying that with some tweaking you can't get a couple more MB/sec out of Samba. You can try the kernel tunables that SaveTheRbtz suggested. I've had good luck with these in the past but the servers were serving files to dozens of clients. I'm not sure how much difference you'll see with a single client. –  3dinfluence Aug 24 '09 at 19:10
    
Another thing you can try if your switch supports it is enabling Jumbo frames. You'll also need to enable this on the server and clients. This can lower the TCP overhead involved but there are a lot of devices that do not play well with Jumbo Frames so this may not be a solution but is certainly something that you can experiment with. –  3dinfluence Aug 24 '09 at 19:12
add comment

1.Try to increase readahead

# /sbin/blockdev --getra /dev/sdb
256
# /sbin/blockdev --setra 16384 /dev/sdb

2.Try to change IO Scheduler, and find one with best performance on your workload

# cat /sys/block/hda/queue/scheduler
noop [anticipatory] deadline cfq
echo deadline > /sys/block/hda/queue/scheduler
share|improve this answer
    
Thanks for the tips, didn't seem to do much difference though. =/ –  weazl Aug 24 '09 at 19:02
    
Hmm.. the only thing that i can now suggest you is to play with aio read size = XXXX, read size = XXXX and use sendfile = Yes parameters in smb.conf –  SaveTheRbtz Aug 24 '09 at 20:04
add comment

Linux disk caching is the reason for the second read being faster, not to mention any H/W disk caching that may be involved.

To improve read performance: Use disk striping, increase RAM, go to the next level of hardware interface to your disks (replace SATA with SAS or FC4), use faster disks (15K RPM instead of 10K), add RAM to your RAID cache.

70MB/s is going to be expensive to improve upon, but you should be able to improve first-read speed some.

share|improve this answer
    
I doubt the disks are the main issue here, when using FTP as a control test I get solid read speeds of around 70-75 MB/sec from uncached files. –  weazl Aug 24 '09 at 13:04
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.