I have 400ish time consuming web extraction tasks that I want to run, but rather than just running them one after the other I want to run i.e. 6 at a time to speed the overall task time up. What I want to achieve ideally is to create the 400 tasks, select that I want 6 to be running concurrently at all time and then the scheduler software will make sure that 6 are always running, when one finishes the next one in the list is started. Is this possible?
If the above ideal solution is not available, I was thinking about creating 6 separate task lists, each with a 1/6 of the total list in. Then the tasks can be started all at the same time, and each separate task list will start the next in the list when the previous finishes.
Any ideas?