Note: Some of you may ask, why not just use a cheap hardware solution. The solution is not so cheap in the country it has to be installed in, with as much as 100% import tax on high end technology. It's much cheaper to get a couple of new components in and use an existing server than to try to get a new server imported in.
Note 2: Please excuse the confusing FC and iSCSI, I see them as the same thing from an OS perspective. So either solution works.
I intend to build myself an "as cheap as possible" SAN with Fibre Channel. I'm less concerned about HA at this point, but may need to do HA down the road. The idea is that I need a decent iSCSI solution for VMWare vSphere to run virtual machines from.
I've been running vSphere from local storage, and some applications (like SQL Server) just work better with native drives rather than running on VFS.
My thought was that I could take a server I alrady have and plop in an FC HBA then run some software that will allow me to treat the drives as iSCSI targets (there are several choices there). And, if I use standard SATA drives, or possibly even SAS "near line" drives i can get a lot of storage for a lot less money, and can maybe get near 15K SAS drives running in RAID5 by running the slower drives in RAID10 and still save money.
However, I do have a few questions that I hope someone can answer as i'm not an FC guru:
1) Do I need an FC Switch if i'm just running between two servers? Can I do Point-to-Point with any two FC HBA's? Or does the HBA have to support that mode explicity?
2) if the HBA's have 2 ports on them, can i run 2 cables and get 2x performance seamlessly (Teaming)? Or do I have to assign some LUN's to one port and some to another?
3) How much CPU load does handling SAN requests take (obviously depends on the OS, but in general)? Can I take a lower spec server and put in a decent SATA controller with room for drives and an FC HBA and not really worry about CPU load much (assuming it does nothing else)?
4) Is this even a good idea?
5) Would GigE make more sense with multiple NIC teaming? I mean, could I put in a couple of 4 or 8 port E1000's and approach a dual port 2 or 4Gb FC in terms of performance?