Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

First of, i'm a webdeveloper and my server experience lies in setting up FreeBSD servers for webserver.

I'm working on a project for at photographer, and i'm hired to develop a new online photo ordering system - where user of course can view their photos :) They have a massive need of storage, thus we have bought a HP G6 and 8x1TB SATA HDD. Our plan is to install VMWare ESX 4.0, running multiple virtual machines; FreeBSD 8 for webserver and some windows servers. Allready done that. Then mount one big storage to the BSD, and share it through Samba to the WinServers.

The raid is set up with an array of 2x 1TB to handle the VMs. And the rest is setup as 3 2x1TB to handle the photo-data. Thus 2.73TB for photo-data (the raids are 1+0).

Now if we add a datastore in the ESX and add the 3 LUNs we can get a datastore of 2.74TB. But i don't se how i can add this datastore direct to the VM. Only the BSD VM needs access to this.

Only way is to create a VirtualDisk, with a max of 2TB (8MB blocksize). This is because the datastore where we save the virtualdisk has a maximum filesize of 2TB. Then add it as a harddisk to the BSD VM.

In the 'Add Harddisk' pane for the VM, i see an option for Raw Disk Management. I think this is to access the datastore or the raid directly. Only problem is that its greyed out!

Can i access the datastorage directly from the BSD? Without creating and adding virtualdisk.

share|improve this question
    
We're now thinking of creating 1 big datastore, adding 3 virtualdisk, adding them as 3 extra harddrives to the BSD. Then within the BSD setting up a software raid (LVM/Vinum) creating a volumegroup/spannedvolume of the 3 harddrives.. any thoughts on that? –  Phliplip May 14 '10 at 0:20

5 Answers 5

This is a pretty good summary of your direct storage access options:

http://professionalvmware.com/2009/08/vmdirectpath-paravirtual-scsi-vsphere-vm-options-and-you/

Another options would be guest OS logical volumes or raiding to join multiple vmdk files together into one large volume.

share|improve this answer

Raw device mapping requires a hardware-compatible RAID card to operate, so the only option you have is to create a disk image ontop of the drives.

On another note, is there any particular reason you went for 3 logical drives? You can go RAID1 (or RAID1+0) with 6 drives, and it will mirror/stripe over all the disks.

You say ESX in your title, but do you actually mean ESXi? If you really do have ESX, there is a workaround for this here - but use at your own risk

share|improve this answer
    
The server console says 'VMware ESX 4.0.0 build-208167' And that the server is a HP ProLiant DL180 G6. We tried the big raid 1+0 for all 6 disks, and as i remember the ESX saw it with 0kb !?? –  Phliplip May 13 '10 at 23:34
    
Thank you for the link, definately the closest to a solution so far. Only thing that stops me is the "seems to work". –  Phliplip May 13 '10 at 23:46

Datastores are limited by LVM's 2TB limit per extent, you can have 32 of these per datastore for a max 64TB datastore. VMDKs are limited to 2TB each but can be bound within the VM (exact details are OS-specific). We try not to do this but have got the odd one or two that have been very stable. For >4TB VM volumes we test to go out from the VMs via iSCSI or NFS etc. to NetApps or similar.

share|improve this answer
    
By 'bound', do you mean something like Windows DFS? –  Phliplip May 13 '10 at 23:49
    
No, I mean how in windows disk manager you can create a logical disk from what windows sees as two or more physical disks. –  Chopper3 May 14 '10 at 7:30

Here is what I've done, which I think is what you are looking for, unless I'm misreading the question:

  1. Configure your RAID with whatever level you think is best.
  2. Set up as many partitions of <2TB as necessary to use all the space.
  3. In the ESX vSphere Client, add the first LUN as a new datastore under Host->Configuration->Storage.
  4. Select the datastore then choose properties, and add each remaining LUN as an extent by clicking Increase, creating a single logical volume as far as the host is concerned.
  5. Profit.

EDIT: Add screenshot: screenshot of VMware ESX Datastore larger than 2TB

Hope that helps. Good luck,

--jed

EDIT2 after reading comments:

@Philplip Ahhhh. I thought you wanted a datastore of greater than 2TB. But what you really want is a VirtualDisk of greater than 2TB. You are out of luck. I think your best bet is to add multiple VirtualDisks of 2TB each and then join them up in the guest VM (probably as a JBOD, since they are already raided on the host side).

share|improve this answer
    
Hi Jed, Thanks for your answer. Unfortunately we have allready tried this, and i can't figure out how to access this datastore directly. I can only add virtualdisk (a file on datastore) of 2TB, due to the maximum filesize of the datastore. –  Phliplip May 14 '10 at 0:49
    
How secure is the data if i use a software raid to span the 3 virtualdisk in a JBOD, or raid-0... how likely is a software raid to break? If the software raid-0 breaks i in trouble even though the physical raid is mirrored. I know that raid is no backup. –  Phliplip May 14 '10 at 1:38
up vote 0 down vote accepted

We ended up with creating 3x arrays of 2x 1TB (Raid 1). Then creating 3 datastores, 1 for each LUN - you could just add all 3 luns to 1 datastore.

Added 3 virtualdisk's of 0.9TB to each datastore. Added the 3 virtualdisk's to the FreeBSD VM as 3 seperate harddrives.

In freebsd we now have harddrive /dev/da0** witch is the initial harddrive where the OS, swap, /home resides. And then we have 3 harddrives /dev/da1, /dev/da2, /dev/da3

# zpool create storage /dev/da1 /dev/da2 /dev/3

Now we have /storage that is 2.7TB, and we can store data on it!

Additionally we have set up Samba to share the /storage on the network, so the win-boxes can access the data.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.