Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I am new to the whole distributed computing / cloud thing. But I had an idea at work for our multimedia stuff like movie encoding / cpu intensive things tasks (which sometimes take a few hours).

Is there a 'free' (linux?) way to go about using a Windows machine, and offsetting those cpu cycles for that task to say 10 servers that are generally idle (cpu wise)?

I'm just curious if there is a way to do this or am I just grasping at straws here. My thought is that a 'cloud' setup would achieve this, however like I stated initially, I am a total newbie when it comes to it.

This is just an idea, looking for some thoughts? Anyone achieve this?

share|improve this question

4 Answers 4

up vote 3 down vote accepted

Corporately we use grid computing for things similar to this. A great one to investigate is Condor.

share|improve this answer
    
Also check out Sun/Oracle Grid Engine (similar product), available as open source too at gridengine.sunsource.net –  m0dlx May 4 '10 at 15:03

are those 'cpu intensive things'...

  • developed in-house? then you'll have to split and distribute the job yourself. there are some nice libraries, but everything is very low level.
  • prebuilt software? then talk to the developer/provider. either it's supported or it's not
  • same process applied to lots of individual items (for example, processing thousands of images)? then you might be able to set a simple system out of a DB table holding the subtask status + some batch scripts that: pick a task, mark as doing, do it, return the result, mark the task as done, repeat until all tasks are done.
share|improve this answer

I think that if you are talking about render farms, there are some examples of such projects.

share|improve this answer
    
Not just render farms, but more like for large cpu intensive cycles, somehow 'funneling' that work through a server farm, but one that is not dedicated just for rendering. Not sure if I make sense... :P –  Jakub Apr 30 '10 at 17:12
    
sure it makes sense. you're thinking in terms of mosix and beowulf clusters! but i think that application supported distribution is better than a generic solution. –  sybreon May 1 '10 at 15:06

Ubuntu with Sun Grid Engine would work and is available in the repos. I think everyone's ideas are going to require you to wipe the host servers though, not sure if that's what you're looking for.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.