Currently, my PHP is on 5.3.3, how can I upgrade it? Also how can I upgrade anything? For example, if I want to upgrade phpMyAdmin as well?
Upgrade all packages:
If you want to upgrade just one package (e.g. php5):
For the package versions available on Debian take a look at:
If you want to install php5 5.4.4-2, that is only available on wheezy, you should add wheezy to your /etc/apt/sources.list:
To don't get any surprises, you should use apt pinning in order to prevent that your system from installing packages from wheezy, just create the file /etc/apt/preferences
So when that when you do apt-get install, if you don't specify -t wheezy it will by default install the package from squeeze.
Pinning is going to put you in dependency hell, as @Juan noticed. When I want the bleeding edge on my Debian systems I usually turn to http://www.dotdeb.org/
For proper instructions -> http://www.dotdeb.org/instructions/
Debian stable is designed to be really, really stable; thus using newer versions of packages is usually not supported, often tricky and/or a bad idea, and sometimes impossible. That said, there are ways to do it:
If no backport is available, and a simple sid backport is not possible, then you probably will not be able to get the package working on your system. You can consider upgrading to debian testing (which is not so bad when testing is under freeze, as Debian Wheezy is as of this writing).
This answer would not be complete without repeating the "simple sid backport" instructions given by the dpkg bot on irc.debian.org:
See also: http://wiki.debian.org/Backports