Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Recently I was at a local user group meeting where the presenter noted that the maximum throughput of the NTFS IO stack was 1 GBps. He substantiated his claim by simultaneously copying two large files from the same logical volume to different logical volumes (i.e. [a] is the source, [b] is destination 1 and [c] is destination 2) and noting the transfer rates hovering around 500 MBps. He repeated this test a few times and noted that the underlying storage subsystem was flash (to make sure we didn't suspect slow storage).

I've been trying to verify this assertion but cannot find anything documented. I suspect that I'm searching for the wrong search terms ("1GBps NTFS throughput", "NTFS throughput maximum"). I'm interested in whether or not the IO stack is actually limited to 1GBps throughput.

EDIT

To clarify: I do not believe the presenter intended to imply that NTFS was intentionally limited (and I'm sorry if I implied that as well). I think it was implied that it was a function of the design of the filesystem.

share|improve this question
    
1GB/s is pretty fast even for flash drive(s) –  TheFiddlerWins Aug 19 '13 at 17:46
2  
@TheFiddlerWins 1 GB/s != 1 Gbps –  Kermit Aug 19 '13 at 18:00
1  
I agree, but his question says "... the maximum throughput of the NTFS IO stack was 1 GBps", as far as I know B=bytes and b=bits –  TheFiddlerWins Aug 19 '13 at 19:32
1  
Simple experiment -- copy the file across RAM disks? Not difficult to get 16 * 2 = 32GB of RAM these days. –  kizzx2 Aug 20 '13 at 5:21
1  
JamesRyan - using FTP as the transfer mechanism does not suddenly change the filesystem. Don't confuse SMB with NTFS. –  mfinni Aug 20 '13 at 13:51
show 5 more comments

7 Answers

up vote 35 down vote accepted

Even assuming you meant GBps and not Gbps...

I am unaware of any filesystem that has an actual throughput limit. Filesystems are simply structures around how to store and retrieve files. They use metadata, structure, naming conventions, security conventions, etc. but the actual throughput limitations are defined by the underlying hardware itself (typically a combination of lots of hardware involved).

Comparing different filesystems and how they affect performance of the underlying hardware can be done, but again that isn't a limitation directly imposed by the filesystem but more of a "variable" in the overall performance of the system.

Choosing to deploy one filesystem over another is typically related to what the underlying OS is, what the server/application is going to be, what the underlying hardware is, and soft factors such as the admin's areas of expertise and familiarity.

==================================================================================

TECHNICAL RESOURCES AND CITATIONS


Optimizing NTFS

NTFS Performance Factors

You determine many of the factors that affect an NTFS volumes' performance. You choose important elements such as an NTFS volume's type (e.g., SCSI, or IDE), speed (e.g., the disks' rpm speed), and the number of disks the volume contains. In addition to these important components, the following factors significantly influence an NTFS volume's performance:

  • The cluster and allocation unit size
  • The location and fragmentation level of frequently accessed files, such as the Master File Table (MFT), directories, special files containing NTFS metadata, the paging file, and commonly used user data files
  • Whether you create the NTFS volume from scratch or convert it from an existing FAT volume
  • Whether the volume uses NTFS compression
  • Whether you disable unnecessary NTFS behaviors

Using faster disks and more drives in multidisk volumes is an obvious way to improve performance. The other performance improvement methods are more obscure and relate to the details of an NTFS volume's configuration.


Scalability and Performance in Modern File Systems

Unfortunately, it is impossible to do direct performance comparisons of the file systems under discussion since they are not all available on the same platform. Further, since available data is necessarily from differing hardware platforms, it is difficult to distinguish the performance characteristics of the file system from that of the hardware platform on which it is running.


NTFS Optimization

New white paper providing guidance for sizing NTFS volumes

What's new in NTFS

Configuring NTFS file system for performance

http://superuser.com/questions/411720/how-does-ntfs-compression-affect-performance

Best practices for NTFS compression in Windows

share|improve this answer
9  
Even still, I could give a symposium with only 802.11g connected on all devices and swear the throughput limit of NTFS was <54Mbps by demonstrating over and over a copy between the devices. –  TheCleaner Aug 19 '13 at 17:47
1  
Could be, but saying "NTFS has a hard limit" vs. "NTFS is slower than ext4 on <x> hardware" is a big difference. He may have mispoke, you might have misinterpreted, regardless...there you go. –  TheCleaner Aug 19 '13 at 17:54
7  
Again, NTFS the filesystem won't have any such limitation, but a given NTFS driver might. –  mfinni Aug 19 '13 at 18:03
1  
Don't think of it as a limit but an logical overhead. That also includes the driver as a "set" limit would have to be a definition value defined in the code of the driver. However I understand you thought process <- (@mfinni)... hard limits are defined in throughput on disk ability to process Read/Write IO and also the technical limitations of the transport medium. –  AngryWombat Aug 19 '13 at 23:53
1  
Nice adds, @TheCleaner –  mfinni Aug 20 '13 at 13:52
show 4 more comments

I very much doubt there is a data transfer bottleneck related to a filesystem, because filesystems don't dictate implementation details that would hard-limit performance. A given driver for a filesystem on a particular config of hardware will have bottlenecks of course.

share|improve this answer
    
I didn't think it was intentionally limited, but thought that, perhaps, it was a limitation of the design –  swasheck Aug 19 '13 at 17:50
    
Thanks for the focus down from "filesystem" to "driver." –  swasheck Aug 19 '13 at 18:16
5  
You can't increase the speed of a book - you can increase the speed of the reader and things the reader depends on. –  mfinni Aug 19 '13 at 18:19
    
Limits and bottle necks are two different things... as a File System can cause overhead it is theoretically possible to create a bottle-neck but will not define a hard set limit which I believe was the intent meant in this post. –  AngryWombat Aug 20 '13 at 0:00
add comment

I would be very surprised if this was true. Let's look at everything that can slow down a filesystem:

  • The physical media (disk, ssd)
  • Connection to this media (sas, sata, fcal)
  • Fragmentation
  • Bad locking algorithms or other code issues
  • CPU and memory speed

The most common limiting factor is your physical media. Rotating rust is SLOW. Take for instance this really new disk which has a maximum speed of 6 Gbps (that's Gbps, not GBps!). Of course using a raid 1 setup will speed this up. Of course you'll never achieve this, as seeks kill your performance. So let's use an SSD you say? Oh look at that, 6Gb again.

Then there's the connection: sas (the fastest local storage) goes up to 6 Gbps, FC goes up to 16Gbps though.

Are you sure your demo was using such high end, state of the art, hardware?

If you are: interesting! You may have hit case 3 and your filesystem needs some optimizing. Or more likely your drivers and application are eating up your CPU (5). If neither of those are though, you may have hit upon an actual performance issue in NTFS, please report it to microsoft.

And even then: this is not an artificial limit, put in place to make your life more miserable. Filesystems don't intentionally limit transfer speeds, but are limited by whatever your hardware can give you.

share|improve this answer
add comment

I don't think there is a maximum. But I know it's more than 1 GB/s because the people at Samsung did 2121.29 MB/s read and 2000.195 MB/s write with their 2009 rig with 24 SSD drives http://www.youtube.com/watch?v=96dWOEa4Djs

They think they hit that limit because this was the combined total hardware bandwidth of the controller cards the SSDs were plugged in to.

Also this page http://blog.laptopmag.com/faster-than-an-ssd-how-to-turn-extra-memory-into-a-ram-disk shows a RAM disk formatted with NTFS getting 5 to 7 GB/s. Try it yourself with one of the RAMdisk titles listed on http://en.wikipedia.org/wiki/List_of_RAM_drive_software

share|improve this answer
add comment

The only logical way to compare filesystem limits would be to do so across systems where the constant was the filesystem and the variables were the other factors such as devices, connections, etc. Using one system to compare transfer speeds over several iterations prove only that the particular system was limited, not that the filesystem was limited.

share|improve this answer
add comment

There's no need to theorize whether or not there's a 1 GBps limit to NTFS--modern SSDs already surpass this. The test bench is a Windows desktop.

enter image description here

share|improve this answer
add comment

There is no built - in throughput limit in NTFS. The only constraint on speed is the performance characteristics of the underlying hardware.

share|improve this answer
    
I didn't think it was intentionally limited, but thought that, perhaps, it was a limitation of the design. –  swasheck Aug 19 '13 at 17:49
6  
@swasheck I don't think it's possible to design a filesystem that won't transfer data twice as fast if you've got a processor twice as fast and can read the disk twice as fast and can seek twice as fast. Even the most inefficient possible design can be made faster by making everything it uses faster. –  Random832 Aug 19 '13 at 18:16
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.