2

I am setting up a 16TB Storage server (which will eventually double in capacity) to store media. This server needs to be read from by two windows server 2008 clients which host our companies applications. I've heard of iSCSI but it seems from what I've read that iScsi treats the target server as if it was directly connected hard drive thus making it very hard to share with two clients as they would randomly write and read data which would mess things up.

So, what software / hardware combination can I use to get high read speeds on the windows side? Right now I am considering 2 dual gigabit ethernet nics with TOE and NFS but I don't want to purchase the nics until I'm sure that it will be fast enough.

Thanks

3
  • You need to further specify the requirements. Do the two application servers need access to the same set of files? What kind of performance numbers do you need to get? What's the budget around? Aug 2, 2009 at 18:33
  • Why do you need two front end servers? This complicates the situation a lot.
    – Posipiet
    Aug 2, 2009 at 18:42
  • The two front end servers do different things but its cheaper to have a lot of storage in one box that it is to load up each box with its own storage.
    – edude05
    Sep 20, 2009 at 14:09

4 Answers 4

3

Ok, so let me get this straight. You have direct-attached storage on a server but you want to share files on it with two other Windows 2008 servers? Why not use Samba and be done with it? Samba basically turns your Solaris server into a domain-member file server. Just install Samba, set up your file shares, and let the Windows 2008 servers access the files from there.

If you are wanting to do some kind of shared-cluster solution, you'll need to use a clustering filesystem. Yes, iSCSI is meant to provide block-level sharing. However, with a clustering filesystem, you would use something like iSCSI as a layer beneath it to share physical disk space.

I guess the issue is that you need to clarify what the 2008 servers will use the Solaris install for. This would at least give an idea about what kind of sharing is involved.

2
  • This answer is pretty old, and admittedly I wrote alot of this in a hurry. I forgot to mention that Samba 3 is not cluster-aware and you'll want to look at Samba 4 if you intend to use file sharing on a clustered filesystem, but also intend to attach clients to both servers and the data needs to be in a consistent state. If you just want fail-over to the 2nd server, then this setup will work just fine. If you need the "true clustered" solution, you'll want to uncheck this as an answer (as much as that hurts) and keep looking. Perhaps OpenAFS might suit your needs instead? Sep 21, 2009 at 16:30
  • 2
    FWIW OpenSolaris comes with a native CIFS implementation integrated in ZFS.
    – knweiss
    Oct 7, 2009 at 20:12
6

Starting out: Determine a price for the data. The cost of losing the data, or loss of access per day.

Usage: Determine how many individual read and write accesses you will receive in peak times. Also determine the data rates for reading and writing.

Network: With this amount of data, you should look into 10Gb Ethernet. Do the math how long it will take to copy the data there.

Raid: On Raids of this size, automatic rebuild is not a sane idea. The rebuilds take so long, and the chances of finding a defect on an additional disc are big enough, that it is better to avoid the automatics. Go for Raid 10, or Raid 6 at least.

Sharing: If you do need backup for the thing, it is probably best to mirror the storage. Look into something with realtime replication. A NAS or SAN is probably the best way to go. Or you look into clustering your MS Servers.

Experience: Build a mockup. You are planning a complicated setup. Try to break it. You will be astounded how easy it is to put this offline if not properly planned.

2

That's not a small amount of storage. You're talking about at least 10 disks (2 TB drives, RAID 5 + 1 hot spare) and probably 20, maybe more. And you say that it will double over the lifetime of the box. I wont say that you can't find a server to hold all those disks but you will have very few choices and will probably end up with a couple of external drive trays in any case. Rolling your own single server solution for this is certainly possible but you have to ask yourself if you are going to be comfortable with all of that data hanging off something with limited redundancy and minimal support.

You should take a look at pricing for entry level iSCSI SAN\NAS storage systems to get an idea what they will cost and see if they can deliver what you need. At the very least you will get a ballpark number for what a shrink wrapped commercial solution costs.

You are correct that iSCSI wont allow you to share between two Windows servers in an Active\Active sense but you could use iSCSI to present the storage to one of the servers and have that server share out the volume(s) so the second server could access them with SMB. You could also opt to just share out directly using SMB - pretty much all NAS boxes will do this too - and just connect both servers that way.

In terms of your nic choices my answer would be that if the data availability is important you should be considering at least two on the storage server in any case, maybe more with that amount of storage. And at least two per target server, with redundant switches in the mix as well to ensure that you have no single points of failure regardless of the performance requirements.

If you find that you need more performance than single GigE links can provide (ie bandwidth over about 80-100Megabyte/sec or random IOPS (4K blocks) >~8000) be sure to check that whatever solution you do select does actually support some mechanism to make use of multiple nics per connection (multi-pathing/channel bonding/link aggregation etc) that Windows 2008 and your choice of network hardware also support.

1

Just one small thing: if you go for speed and availability, forget RAID 5. Go with mirroring, 16TB is not so expensive to reach.

You'll get spread reading (up to double throughput) and when a disk fails, only part has to be rebuild with a least inexpensive operation (no special checksums have to be calculated).

Plus, up to half of the drives can go dead in best worst case™.

Otherwise I've been experimenting at home with about 360GB of mirrored SATA, Solaris on Intel and OS X client and Gbit ethernet. All normal hardware, no server components; set up jumbo frames and got to 58MB/s with SMB, very close to integrated iMac disk.

NFS should be even faster, but not on every platform, the one on OSX is not very good, don't know for windows.

Also, for tuning you have to know what kind of load you'll do to pick right drives (small number of bigger or larger number of small drives), buffers, packet sizes, ...

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .