Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We are running website which is currently serving 3-5 million page views. Our site is file sharing site and so it contains 250,000 files and few thousand symbolic links.

The hard disk is a 1500GB SATA disk.

Using hdparm we came to know that our hard disk speed has been reduced to 15-20 mbps, which was 80 mbps.

So now we want to run fsck to fix the disk problem.

I have 2 questions:

  • Will fsck will solve this issue?
  • How much time will fsck take to complete (just we want to calculate the downtime which we are going to have)?

Thanks.

share|improve this question
    
What filesystem are you using? Is this a RAID array, or a simple SATA disk? –  Kyle Smith Feb 21 '12 at 16:30
    
simple SATA disk and file system is ext3 –  khizar ansari Feb 21 '12 at 16:32
6  
Why are you running a production web server on a single disk? Servers with single disks aren't servers-- they're ticking time bombs. –  Evan Anderson Feb 21 '12 at 16:45
2  
It sounds like that disk is dying. Migrate your data (to a real RAID array with a hardware battery-backed controller) immediately. –  Joel E Salas Feb 21 '12 at 17:05
add comment

3 Answers

up vote 2 down vote accepted

The speed degradation is to be expected as the number of files being accessed simultaneously increases. Hard disk drives do not like to be accessed in parallel: every time the read/write head needs to switch cylinders you lose several milliseconds. Even if two files are on the same cylinder, or even the same track, you may still have to wait a rotation to move from one to another. If you measure drive performance in megabits per second, expect that to drop exponentially as parallel access increases.

fsck will not help with this: it only repairs damage to the directory structure, it does not perform any optimization.

The ideal solution would be switching to solid-state storage since that does not have any of the physical limitations of spinning platters. But that's probably cost-prohibitive.

The next best would be to use a RAID optimized for parallel access. Keep in mind that RAIDs can be configured for many different performance profiles, so you will need to take some time to learn the settings of any given RAID hardware and drivers.

You may be able to reduce the problem using aggressive filesystem caching. If your system has sufficient RAM, linux should be doing this fairly well already. Run a program like top to see how much free RAM there is. But if the most commonly used files do not fit in RAM (or any RAM you are likely to acquire), this won't really help.

A poor-man's work-around would be to split your files across several different physical hard-drives (not just different partitions on the same drive). That's not really a long-term scaleable solution and would end up costing you more than a decent RAID. But it might be a quick fix if you have drives lying around.

For any solution involving hard disk drives, make sure they have a fast rotation speed and low seek latency.

I have written an article with some general background on hard-drive performance here:

UNIX Tips - Filesystems

share|improve this answer
    
I don't see his hdparm benchmark having much to do with "parallel access". It sounds more, to me, like he's got a failing disk. It was faster in the past and now it's not. Probably because it's relocating sectors. –  Evan Anderson Feb 21 '12 at 16:44
    
That is certainly a possibility, although I would think relocation on that scale would produce some I/O errors. Based on the very slow baseline of 80 megabits per second, I was assuming the test was run on an active system. So... are there I/O errors in the system log, how were the hdparm tests performed, and were the results in "megabits" or "megabytes" per second? –  Seth Noble Feb 21 '12 at 17:51
add comment
  1. No (fsck can fix corrupted filesystem metadata, not a broken disk, nor is it a defragmentation tool).
  2. Depends on the filesystem. With ext3, excruciatingly long, I'd reserve several hours. More modern filesystems such as ext4 or xfs can easily be an order of magnitude faster.
share|improve this answer
add comment

I would expect 5 hours for the fsck to complete.

I would instead consider (that means: testing, testing and testing) a migration to reiserfs.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.