4

I'm just wondering how folks handle ongoing file system stability when using a Windows Server as a file server without taking the system offline to perform chkdsk /f or chkdsk /r? Obviously, one doesn't really want a file server to be unavailable...and file servers now have so much storage that it could take days to run a chkdsk...so how are you protecting data from corruption?

1
  • Days to run chkdsk? Just how many petabytes of storage does your server have? Mar 12, 2011 at 18:51

4 Answers 4

0

Microsoft has published prescriptive guidance for improving the performance and minimizing downtime when running checkdisk:

NTFS Chkdsk Best Practices and Performance
https://www.microsoft.com/downloads/en/details.aspx?FamilyID=35a658cb-5dc7-4c46-b54c-8f3089ac097a

Of particular note:

  • Volume size has no effect on performance.

  • For volumes with large numbers of files (hundreds of millions/billions), the performance increase of utilizing more memory for chkdsk is dramatic.

  • Windows 2008 R2 chkdsk is between two - five times the performance of Windows 2008. Windows 2003 was so bad, they were probably too embarrassed to publish the statistics.

  • You should proactively check if the volume(s) are dirty before a scheduled restart. This can help mitigate the effect of unexpected multi-hour startup delays.

Not in the document, but highly recommended: using a multi-purpose server for file serving hundreds of millions of files increases that probability that a crash may occur, and a volume will be marked dirty. Measures should be taken to ensure that a crash would not occur. An example would be not using the file server as a print server (printer drivers have a long notorious history in blue screen land). Another example would be "file archiving software". A backup power source with extended runtime is highly recommended.

5

In my opinion chkdsk is not a tool for performing preventive maintenance. If you're having to run chkdsk on a regular basis to correct problems then you have an underlying problem that needs to be solved.

3
  • What do you use for preventative maintenance then...? Mar 12, 2011 at 15:55
  • I use Defrag to keep the disk as tidy as possible but there isn't a tool that prevents disk problems or data corruption. Chkdsk is not a preventive maintenence tool, it's a tool to fix problems that already exist. Chkdsk doesn't prevent disk problems or data corruption. Even tools such as those mentioned by shadowman12 don't prevent problems, they correct problems. There isn't a tool that can prevent a disk block from going bad or a file from becoming corrupt, they can only act on and remediate a problem that exists, not prevent them.
    – joeqwerty
    Mar 12, 2011 at 16:07
  • Even technologies like S.M.A.R.T don't prevent problems. They monitor, predict, report, and in some cases act on problems, but none of them can prevent problems.
    – joeqwerty
    Mar 12, 2011 at 16:12
5

I maintained file-servers with around 7TB of general user data. That 7TB was built up mostly of office-type files, so we're talking millions. I don't have an exact number because it takes so long to get, but somewhere between 7-12 million files in the various file-systems on our Server 2008 fail-over cluster.

We never run chkdsk except to fix problems, and we never defrag.

NTFS is now self-healing enough that we run into problems very, very rarely. When we do get into problems it's generally due to a fault in the storage-system infrastructure in some way; spontaneous fibre-channel array controller reboot, FC switch panic-and-reboot, that kind of thing. Yanking the power out of the back of the server is eminently surviveable.

In fact, we recently survived a catastrophic UPS failure. The entire room dropped hard, simultaneously. NTFS recovered with nary a peep, and no need to run chkdsk.

About defrag... our FC disk array has 48 drives in it, and as it is an HP EVA the stripes are randomly distributed across the spindles. This means that even largely sequential accesses are actually random as far as the drives are concerned, which further means that a significantly sequential file-system performs minimally better than a significantly fragmented one. Therefore, routine defrags do very little to help for a lot of I/O overhead.

As for preventative maintenance, NTFS is now automated enough to do nearly all of that by itself. Once in a while I'll run chkdsk in read-only mode to see if running it in full mode is worth it. So far on our cluster it has yet to be needed. Even on our 2TB, 4 million file LUN it runs in less than a day.


That said, there are some architectural decisions you can make that can help reduce the eventual need for an offline chkdsk and make it go faster if you ever need to do one:

  • Set the cache policy on your RAID/SAN controllers to not cache writes. However, this is why battery-backed cache exists, so the performance hit this will cause does not need to be taken. But this is the top thing to do to prevent an offline chkdsk.
  • Keep your LUNs smaller. File-count matters more than size. A 6TB LUN full of Ghost images will check a lot faster than a 512GB LUN full of 6KB files.
  • Maintain adequate free-space. A good rule of thumb based on entirely subjective criteria is no less than 15% free at any time.
  • If your data allows, use a block-size larger than the default 4KB block-size for NTFS. After doing some statistics on my files, I've found I can use 16KB blocks for most of my filesystems. Larger blocks mean fewer blocks to check, and also allow the storage subsystem to take better advantage of read-ahead. Yes, itty bitty files consume more space, but on our volumes it only added about 4% to total size.
1
  • "Yanking the power out of the back of the server is eminently surviveable." This is true if you're not adding/removing/changing files significantly and if you have a little bit of luck. I was forced to run a chkdsk because power went out during heavy write operations.
    – Phil
    Aug 18, 2011 at 16:36
2

In the previous where I worked we used Tripwire. For more information you can take a look here:Tripwire File Integrity Manager

Here you will also find an overview of solutions in the market for File Integrity checking:File integrity checkers

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .