The reason this advice comes up is to avoid overloading one disk group. Consider the following set of simultaneous events:
- A DB connection runs a SELECT on a big table
- A second DB connection runs a large INSERT or UPDATE on a different table
- Several people all log in to the machine at once (SSH)
If your system was running on a single disk the head now needs to bounce around between where the table we're
SELECTing from is located, wherever
/var lives (to record the logins), the location of the Write-Ahead Log (to record the
UPDATE), the various locations in
/bin needed to start up the new user sessions, and eventually the tables that were
INSERTed into or
All that disk-head travel takes time - as the number of conflicting requests pile up the time required becomes measurable, and possible noticeable as a performance hit.
If each of those three items were happening to a separate disk group (
SELECTs to the data group,
UPDATEs to the WAL group, and OS stuff to the OS group) each of the three items above could theoretically happen in parallel (and the eventual flushing of the Write-Ahead-Log could be done when it's convenient for the data group).
The key is that this is based on a theoretically small number of spindles - a single disk, or a small RAID group of 2-4 drives.
As both you and ewwhite point out, at a certain point having more spindles mitigates the conflicting head travel requests, and eventually the gain eclipses the potential performance penalty for most workloads.
The key here is benchmarking your workload (as much as you practically can).
Also as has been pointed out SSDs invalidate the major underlying assumption (SSDs are constant-time access for any "sector" - there's no rotational or head-positioning delay), so you should bear that in mind if you're using SSDs in your design: You'll probably see more benefit from striping SSDs than you would from separating them to isolate activity based on the assumption that conflicting disk requests require more time to reposition mechanical heads and platters.