I've installed CentOS, along with some applications such as Apache and PHP.
How do I remove the currently installed versions so that I can install the correct versions using yum?
Whatever version of each package you got from the installer, that's the only major version release of that package that's available for that version of CentOS, at least from CentOS itself; it follows the Redhat Enterprise Linux package set, which typically lags behind the latest and greatest for any given package by a few versions in order to provide a known-stable release.
That said, to remove a package you can use
Search the required package with the command:
rpm -qa | grep -i "package name"
and simply type the command yum remove as shown below:
yum remove "package name"