Primarily questions about disk storage, sometimes questions about application storage or cloud storage. Always questions about storing data.

learn more… | top users | synonyms

109
votes
5answers
10k views

Are SSD drives as reliable as mechanical drives (2013)?

SSD drives have been around for several years now. But the issue of reliability still comes up. I guess this is a follow up from this question posted 4 years ago, and last updated in 2011. It's now ...
101
votes
2answers
13k views

What are the different widely used RAID levels and when should I consider them?

This is a Canonical Question about RAID levels. What are: the RAID levels typically used (including the RAID-Z family)? deployments are they commonly found in? benefits and pitfalls of each? ...
75
votes
6answers
13k views

Why is Enterprise Storage so expensive?

This is a Canonical Question about the Cost of Enterprise Storage. See also the following question: What's the best way to explain storage issues to developers and other users ...
53
votes
1answer
755 views

Fibre channel long distance woes

I need a fresh pair of eyes. We're using a 15km fibre optic line across which fibrechannel and 10GbE is multiplexed (passive optical CWDM). For FC we have long distance lasers suitable up to 40km ...
52
votes
13answers
74k views

Windows 2008: WinSXS directory growing uncontrollably, blocking server

I run a (remotely hosted) virtual Server with Windows 2008 Server for a client. Initially, it had 10 GB of space. During the course of a few weeks - during which nothing was done on the machine ...
47
votes
8answers
2k views

What are the advantages of tape drives?

IBM still develop and sell tape drives today. The capacity of them seems to be on a par with today's hard drives, but the search time and transfer rate are both significantly lower than that of hard ...
43
votes
7answers
3k views

Probability of Blade Chassis Failure

In my organisation we are thinking about buying blade servers - instead of rack servers. Of course technology vendors also make them sound very nice. A concern, that I read very often in different ...
38
votes
8answers
8k views

Is it safe to use consumer MLC SSDs in a server?

We (and by we I mean Jeff) are looking into the possibility of using Consumer MLC SSD disks in our backup data center. We want to try to keep costs down and usable space up - so the Intel X25-E's ...
34
votes
8answers
2k views

What's the best way to explain storage issues to developers and other users

When server storage gets low developers all start to moan, "I can get a 1 TB drive at Walmart for 100 bucks, what's the problem". How can the complexities of storage be explained to developers so ...
32
votes
11answers
21k views

ZFS vs XFS

We're considering building a ~16TB storage server. At the moment, we're considering both ZFS and XFS as filesystem. What are the advantages, disadvantages? What do we have to look for? Is there a ...
32
votes
20answers
13k views

Storing a million images in the filesystem

I have a project that will generate a huge number of images. Around 1,000,000 for start. They are not large images so I will store them all on one machine at start. How do you recommended on ...
29
votes
3answers
4k views

Tuning iSCSI storage

This is a Canonical Question about iSCSI we can use as a reference. iSCSI is a protocol that puts SCSI commands as payload into TCP network packets. As such, it is subject to a different set of ...
27
votes
2answers
5k views

Why are enterprise SAS disk enclosures seemingly so expensive?

I will begin by stating that I do not believe this is a duplicate of Why is business-class storage so expensive?. My question is specifically about SAS drive enclosures, and justifying their expense. ...
26
votes
2answers
2k views

What is a Storage Area Network, and which benefits does it have over different storage solutions?

I'm proposing this to be a canonical question about enterprise-level Storage Area Networks. What is a Storage Area Network (SAN), and how does it work? How is it different from a Network Attached ...
25
votes
10answers
20k views

Cheapest way to connect 20-24 Sata II HDDs in a budget storage server? [closed]

I need to assemble a high density storage server for as cheap as possible. It's been a while for me and the last systems I integrated didn't even have Sata yet... During my Research I of course ...
24
votes
5answers
2k views

Do I need to RAID Fusion-io cards?

Can I run reliably with a single Fusion-io card installed in a server, or do I need to deploy two cards in a software RAID setup? Fusion-io isn't very clear (almost misleading) on the topic when ...
24
votes
11answers
10k views

Does an unplugged hard drive used for data archival deteriorate?

If I were to archive data on a hard drive, unplug it, and set it on a (not dusty, temperature-controlled) shelf somewhere, would that drive deteriorate much? How does the data retention of an ...
24
votes
1answer
3k views

How should I set up my Fibre Channel (FC) network?

How do FC switches work, and how should I configure mine?
23
votes
12answers
3k views

Storing and backing up 10 million files on Linux

I run a website where about 10 million files (book covers) are stored in 3 levels of subdirectories, ranging [0-f]: 0/0/0/ 0/0/1/ ... f/f/f/ This leads to around 2400 files per directory, which is ...
22
votes
5answers
26k views

What is the difference between SAN, NAS and DAS?

What is the difference between SAN, NAS and DAS?
21
votes
7answers
2k views

Best way to test new HDD's for a cheap storage server

I want to build a storage server and bought 10 x 2TB WD RED's. The HDD's just arrived. Is there any tool you guys use to check for bad drives or to best defend against infant mortality before copying ...
21
votes
8answers
7k views

What kind of storage do people actually use for VMware ESX servers?

VMware and many network evangelists try to tell you that sophisticated (=expensive) fiber SANs are the "only" storage option for VMware ESX and ESXi servers. Well, yes, of course. Using a SAN is fast, ...
20
votes
4answers
14k views

How to move files between two S3 buckets with minimum cost?

I have millions of files in a Amazon S3 bucket and I'd like to move these files to other buckets and folders with minimum cost or no cost if possible. All buckets are in the same zone. How could I ...
20
votes
8answers
533 views

Server vendor that allows 3rd party disks

As noted here, Dell is no longer allowing 3rd party disks to be used with their latest servers. As in, they don't work period. Which means that if you buy one of these boxes and want to upgrade the ...
19
votes
5answers
2k views

BBWC: in theory a good idea but has one ever saved your data?

I'm familiar with what a BBWC (Battery-backed write cache) is intended to do - and previously used them in my servers even with good UPS. There are obvously failures it does not provide protection ...
19
votes
3answers
7k views

Linux - real-world hardware RAID controller tuning (scsi and cciss)

Most of the Linux systems I manage feature hardware RAID controllers (mostly HP Smart Array). They're all running RHEL or CentOS. I'm looking for real-world tunables to help optimize performance for ...
18
votes
2answers
2k views

ZFS Data Loss Scenarios

I'm looking toward building a largish ZFS Pool (150TB+), and I'd like to hear people experiences about data loss scenarios due to failed hardware, in particular, distinguishing between instances where ...
17
votes
7answers
708 views

Is there still a place for tape storage?

We've backed up our data on LTO tapes for years and it's a real comfort to know we have everything on tape. A sister project and one of our data providers have both moved to 100% disk storage because ...
16
votes
7answers
4k views

RAID-5: Two disks failed simultaneously?

We have a Dell PowerEdge T410 server running CentOS, with a RAID-5 array containing 5 Seagate Barracuda 3 TB SATA disks. Yesterday the system crashed (I don't know how exactly and I don't have any ...
16
votes
8answers
536k views

How to format as FAT32 from Windows 7/Vista

What is the best way to format a USB drive with FAT32 (for Mac compatibility) from within Windows 7/Vista? I ask because the Disk Management only lets you pick exFAT (because the disk is over 32GB I ...
16
votes
10answers
24k views

What brand(s) of hard disk has the lowest failure rates? [closed]

Wiki for information on which storage manufacturer(s)'s drives have the longest life / lowest failure rate? Everyone has a favorite, but if you can refer to a (hopefully recent and somewhat ...
16
votes
2answers
439 views

How can I safely use storage thin provisioning?

I have storage that allows me to thin provision my volumes presented to the clients. Is this safe? What are the best practices?
16
votes
10answers
9k views

Distributed Storage Filesystem - Which one/Is there a ready to use product?

With Hadoop and CouchDB all over in Blogs and related news what's a distributed-fault-tolerant storage (engine) that actually works. CouchDB doesn't actually have any distribution features built-in, ...
16
votes
3answers
3k views

How effective is LSI CacheCade SSD storage tiering?

LSI offers their CacheCade storage tiering technology, which allows SSD devices to be used as read and write caches to augment traditional RAID arrays. Other vendors have adopted similar ...
15
votes
1answer
7k views

Hosting a ZFS server as a virtual guest

I'm still new to ZFS. I've been using Nexenta but I'm thinking of switching to OpenIndiana or Solaris 11 Express. Right now, I'm at a point of considering virtualizing the ZFS server as a guest within ...
14
votes
8answers
2k views

Do snapshots + RAID count as a good on-site backup solution?

The two main reasons I can think of for taking backups seems to be taken care of when I use both snapshots and RAID together with btrfs. (By RAID here, I mean RAID1 or 10) Accidental deletion of ...
14
votes
12answers
9k views

Are there no downsides to NetApp SAN solutions other than price? [closed]

We have pretty much decided on a NetApp solution for our first SAN. Given that, I've been tasked with finding as much reason not to go with NetApp as I can. We like to do this A) so we know what we're ...
14
votes
6answers
2k views

Cost effective, long term archival of video and image data? ~50 TB

My lab is in the process of setting up a small server that holds data (mostly video and image data, plus a few documents) for the project our group is working on at a moment in time. Historically, ...
14
votes
5answers
966 views

Choosing a SAN technology for 100s of VM Web Servers

The Problem We have an issue with performance on an existing platform, so I'm turning to the hive mind for a second opinion on this. The performance issue so far relates to IOPS rather than ...
14
votes
5answers
887 views

Is my new Oracle DBA talking sense

When we setup FC LUNs for our MSSQL boxes we rarely have to present them with more than about 8 distinct LUNs of various kinds (Quorum, MSDTC, TempDB, Data, Logs, Backup and a few others). We have a ...
14
votes
11answers
4k views

S3-compatible Object Storage systems [closed]

Has anyone come across a near 100% Amazon S3 RESTful API compatible object storage system? What I am after is a layer that sit on top of any (preferably a POSIX) filesystem that provides Amazon S3 ...
13
votes
3answers
2k views

Economical way to get many harddrives into rack mount?

Please bear with me as being a bit of a newcomer at 19" rack-mounted equipment. I've thought a fair bit lately about the best way of getting 4x or 6x of 2.5" hard drives into my rack and are ...
13
votes
3answers
2k views

150 TB and growing, but how to grow?

My group currently has two largish storage servers, both NAS running debian linux. The first is an all-in-one 24-disk (SATA) server that is several years old. We have two hardware RAIDS set up on it ...
12
votes
4answers
616 views

Hooking up many different external HDs simultaneously

I need a large amount of external storage for an upcoming project. I'm planning on purchasing 10 2TB external drives. I need them all hooked up to a single machine at the same time. What issues ...
12
votes
7answers
677 views

How do storage IOPS change in response to disk capacity?

All other things being equal, how would a storage array's IOPS performance change if one used larger disks. For example, take an array with 10 X 100GB disks. Measure IOPS for sequential 256kb block ...
12
votes
8answers
760 views

Why buy costly hard drives from storage server vendors like Dell/HP/Sun?

I'm just browsing around at storage servers like the Dell MD1000/MD3000 and the Sun J4200, and although the storage server pricing seems reasonable (approx $3000-$4000 AUD) the hard drives that you ...
12
votes
8answers
1k views

Datacenter Power requirements. A=W/V… I must be doing something stupid

Okay, our datacenter doesn't charge us based on our "useage" just on how many Amps we have delivered to our rack. I'm planning on putting in a new rack this upcoming year for some server expansion, ...
12
votes
2answers
822 views

Safety of write cache on SATA drives with barriers

I've been reading lately about write caching, NCQ, firmware bugs, barriers, etc regarding SATA drives, and I'm not sure what's the best setting that would make my data safe in case of a power failure. ...
11
votes
4answers
783 views

Disk Speed Performance in a SAN

We are currently looking into buying a SAN, and have some concerns over the speed of disks we are going to put into the array. We would normally have gone for 15k disks over 10k every time, but one of ...
11
votes
3answers
5k views

Tuning Linux disk caching behaviour for maximum throughput

I'm running into a maximum throughput issue here and need some advice on which way to tune my knobs. We're running a 10Gbit fileserver for backup distribution. It's a two disk S-ATA2 setup on an LSI ...