Tell me more ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.
up vote 0 down vote favorite

I have an array of 6 million database records. Each record has 100 columns. Is there a formula to calculate the amount of memory that is being utilized by the server? Will this be exponential?

The processing involves writing the contents of these records to XML files. It won't be 6 million files, but it will be a significant number.

I need to know what kind of RAM to allocate for this process, before I kick it off.

If there's no formula, what steps should I take? Any insight would be helpful.

share|improve this question

1 Answer

up vote 1 down vote

There's really no way we could possibly answer this.

A program could be written that pulls 1 byte from the DB at a time and writes it to an XML file; so 640K should be enough.

At the other extreme the program could load all 600 million fields into memory then allocate all the space needed to write every single XML file; if the fields were all typical Integers you'd need something in the order of 6GB (600M * 4 Bytes * ~2.5 = 6GB); if they're all 10MB jpeg images more like 15PB (600M * 10MB * ~2.5 = 15PB).

share|improve this answer
I do believe that all 600 million fields are in memory. The dataTable object in .Net stores all records in memory, thus enabling you to do all the processing IN-MEMORY. Perhaps we might have to consider changing the datatable to use a dataReader which will stream data as required. – abhi Apr 28 '11 at 15:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.