I have Apache running on a quad-core Ubuntu server on a 384kbps ADSL. Files are uploaded by users via a web-form and processed by various Python programs running as CGI scripts. Some scripts are CPU intensive and run at 100% (on one core) for a few minutes; these email the results to the user so the HTTP session is not held open. Some scripts require larger files (total of a few MB) to be uploaded. At present, usage is very low, with a handful of hits per day and very few, if any, instances or more than user making use of the services at the same time. I will need to make these services available to a greater number of users in the medium-term.
I suspect that the infrastructure I have built does not lend itself easily to scaling. For example, one user has requested that I allow multiple files to be uploaded to the CPU-intensive program. This means that the machine will be busy for a longer period of time. If another user also uploaded multiple files to the same script, the machine may become very busy for an even longer period.
I know discussion-type questions are not permitted here, so I'd like to ask the following specific questions:
What strategies or approaches will I need to consider when making these services scalable -- that is, do I need to rethink the infrastructure completely?
If I made no changes and 10 people each uploaded 10 files to the CPU-intensive program, for example, would all 10 threads created by the CGI script just run happily (if slowly) over all 10 input files? Is it "safe" to have a server running at 100% CPU usage for an hour or two or three?