Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I use a distributed user-space filesystem (GlusterFS) and I would like to be sure GlusterFS processes will always have the computing power they need.

Each execution node of my grid have 2 CPU, with 4 cores per CPU and 2 threads per core (16 "processors" are seen by Linux).

My goal is to guarantee that GlusterFS processes have enough processing power to be reliable, responsive and fast. (There is no marketing here, just the dreams of a sysadmin ;-)

I consider two main points :

  • GlusterFS processes
  • I/O for data access (on local disks, or remote disks)

I thought about binding GlusterFS instances on a specific "processor".

I would like to be sure that :

  • No grid job will impact the kernel and the GlusterFS instances
  • Researchers jobs won't be affected by system processes (I'd like to reserve a pool of cores to job execution and be sure that no system process will use these CPUs)

But what about I/O ? As we handle a huge amount of data (several terabytes), we'll have a lot of interuptions.

How can I distribute these operations on my processors ? What are the "best practices" ?

Thanks for your comments!

share|improve this question

2 Answers 2

up vote 1 down vote accepted

Pinning a userspace to a specific processor sometimes can make it perform more consistently; but not the kernel. (check taskset)

if it were possible, you would be turning the userspace/kernel switchtme (already a measurable factor in an OS performance) into an inter-processor communication / synchronization issue. Many orders of magnitude worse.

edit: now that you've removed the 'pin the kernel' idea, it's a lot more reasonable. Yes, you can use taskset to start all grid processes and leave one or two CPUs free for GlusterFS. For an analogous example, in Xen systems is considered a 'best practice' to reserve one CPU for the Dom0, which handles all I/O.

share|improve this answer
    
Thanks for your input. What I'm afraid of is having the 16 "processors" used by grid jobs, leaving no cpu for GlusterFS and the I/O. Right now, I think I'll set a limit of 15 jobs per exec nodes, hoping that 1 processor will be fine for GlusterFS and the I/O. –  Philippe Mar 31 '10 at 14:33

That's why I don't want to use FUSE filesystems in production... I'm using PVFS2 instead.

share|improve this answer
    
Never heard of it. I'll take a look. Thanks! –  Philippe Apr 8 '10 at 16:55

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.