If you can wait, CentOS 5.6 should have native PHP53 since upstream RHEL 5.6 now ships PHP53.
If you can't wait then this might help you install the packages from the REMI repo:
yum remove mysql php* (if currently have PHP 5.1.x and/or mysql installed)
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-5.rpm
yum --enablerepo=remi install php
Getting the mysql commandline client on the box is tricky:
yum --enablerepo=remi --disablerepo=updates --disablerepo=base install mysql
Edit: If your MySQL server is on the same box this procedure has the unfortunate consequence of having you run the MySQL Server from the REMI repo. i.e. You're basically in dependency hell.