Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If you had some money to spend (approximately CAD180,000) on hardware and software to set up a data-archival system. We have a quote from Isilon Systems however before the money is spent we have to put the purchase through a tendering process. So, I am looking for ideas for the specifications of our ideal archival-server.

In the ideal world, the system should be able to archive and retrieve the many kinds of experimental data our lab collects. This data ranges from microscopy image stacks, to digitized electrical signals collected in wide variety of formats etc.

What software, hardware and hardware would you have in your ideal data-archiving server if you had about CAD180,000 to spend (or would I need to spend more)?

We currently collect data on the order of about a ~100GB a month and would want to hopefully archive it for a number of years.

share|improve this question

closed as not constructive by Chris S Jan 20 '13 at 14:26

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

How much data are you planning on storing? – womble Jan 20 '10 at 0:39
How long are you planning on keeping the data for? – James Jan 20 '10 at 0:43
@womble we probably would need to end up storing on the order of hundred's of TB of data. – Azim Jan 20 '10 at 1:24
@James, we would like to keep the data around for at least a few years. Our current, adhoc system does not work. – Azim Jan 20 '10 at 1:25
I would get you to give me the $180,000 as a "consultant" and then take a holiday to the Maldives – Mark Henderson Jan 20 '10 at 4:08

The real question to me is what do you want to do with the archived data? The requirements you given are really vague. The 2 big options that I think of are Documentum and Microsoft Office Sharepoint server (and you can even get solutions that combine the 2). In a very very simplified and generic way the basic difference between the 2 is how data is treated once it's archived.

Documentum is an admin managed, policy driven, document lifecycle manager at its core (eg document x is 6 months old so it's migrated to never never land, it contains Mr. Jones' notes). To get to those notes you (or usually an admin) retrieves the document. It's very efficient at storing docs.

Sharepoint on the other hand is not as space efficient at storing docs (it's much better in 2010) but Sharepoint's point of view is that docs contain data that can be reported on(with images you might track the metadata) and that users should be able to represent data in whatever view suits thier current need. So you might not even need to retrieve Mr Jones' notes because all you needed was his data that got aggregated into a sharepoint report (of course you can always go back to the source)

I don't think there is anything out of the box that will do exactly what you want so plan on budgeting some consulting time as well. Talk to the various departments and make sure you fully understand and have documented their requirements (saying you need to archive and retrieve data is not a great requirement list) although good to know size and document end of life are not boundaries either (except to know that you want san hardware to be supported for at least a "few" years). Really dig in to what the business is going to need to do with this data. It may turn out that you don't need any sort of document lifecycle management but simply hierarchical storage management (another reason to like compellent)

For san hardware I've found nothing that compare to compellent sans

share|improve this answer
Thank you for your answer. I agree the question is a bit vague and we need an idea what the work flow is going for the collection, dissemination and archiving of the experimental data that is collected in our lab. – Azim Jan 20 '10 at 16:38

Not the answer you're looking for? Browse other questions tagged or ask your own question.