Short answer, they struggled!
Sometimes a bit of evolutional history is a good way to understand where we came from and where we are are right now......
A simple web server had to bind to an IP address.
So that really meant if you restricted yourself to one port (80) you could
only have one real domain per ip address (machine). However you could specify a directory of where content was , maybe a user $HOME dir.
File access was enforced by simply by user accounts permissions.
Your Unique User Identification (UUID) would be deemed enough to separate accounts.
Because the way the web server is architected, it doesn't really use the traditional way of user account privileges and permissions, the web server was generally running as root (in the worse cases) or as an under privilege user in the best cases. (www-data/nobody).
The best thing about a web server was that it could transfer files you wanted across a network to be rendered in a browser, the worse thing about a web server is that it could transfer files that you probably don't want too. (/etc/passwd).
Then came along the apache virtual host directive. This allowed the web server to identify what domain the client web browser wanted.
The invention of a Vhosts, so the web server could serve files dependant on the host header name, not the ip address of the server.
File transfer services such as FTP/SSH were linked via your username to an area that you had the permission you could write too... (these systems also had their own security problems).
PHP caught on at the same time, customer demand for being able to write active scrips mushroomed. They wanted to be able to run on the web server dynamically and they wanted it NOW!.
So, it was a matter of trying to secure a unix system, where everything is effectively running as the same UUID... can you see a problem starting to occurring?
This started the web server security arms race!!!
An attack would be discovered, and then a patch or way of dealing with it would be deployed... that usually meant more code in production... or restricting what the web server could do in terms of configuration.
Sometimes this would be bug in the actual code, sometimes configuration would allow for a flaw to be exploited. The worse case scenario was prevent your users from access some kind a feature they relied on!!
followsymlinks on apache why is it a security risk
So, the hosts deployed fix after fix, patch after patch. Like all security controls, you start to restrict by configuration, or patch code to 'do the right thing' in terms of security, you start to break compatibility, and start to come up with integration issues.
Add more arms race technology like SELINX, although you can create a secure web server, you break so much software, that it becomes useless... it's either works or becomes so hard to manage it becomes unmanageable. Now multiply that with a X amount of users on a the same machine... Every layer of security added could break existing PHP scripts of make debugging them extremely difficult.
You could get to a point where you would be ultra secure, but nothing would actually run..... ;-).
Allowing customers to upload their own scripts to the machine or a flaw where attacks could do the same, could lead to a comprise of that server, allow an attacker gain control not only of the that account, by escalate them to root privileges on that machine.
Even local privilege escalation bugs are problems, because when you running your a active scripting language like PHP, you are effectively local on the machine.
The bad news is , that all this is still with us.
The good news Docker and other containers / virtual machine technology only shift the problem along. However, you can use a much simpler configuration ,use less code in the container to do the same thing.
Also, configurations can be much much simpler, and can be actually managed effectively.
You can probably see why there now a shift away from large 'kitchen sink' web servers and 1000's of accounts on the same machine.