Packages provided by your distribution. With Debian/Ubuntu, you can do "apt-get install wordpress", etc. Source Debian Unstable repo if you need to. RPM-based distros will have something very similar. How these distribution-provided webapps are packaged varies, some packages may support virtual hosts and multiple installations, but many of them only support one instance running on the same server.
Hosting companies usually use Softaculous (or Fantasico, installatron) to do this. You can license and install Softaculous yourself. Your VPS hosting company might already have a reseller license for cPanel/Plesk + Softaculous, which you can install on your VPS for a small monthly fee. Virtual hosts are generally well-supported. http://www.softaculous.com/softaculous/
You might find Sofaculous AMPPS useful, it's a LAMP dev stack for Windows/OSX with Softaculous one-click installer built-in, so you can test apps easily on your desktop before installing them on your server. http://www.ampps.com/