I have a publicly accessible page which just has
<?php phpinfo(); >
I use it for debugging purposes while we're in beta, but is there any harm in leaving it accessible when its a live site?
It would entirely depend on how confident you are about your PHP install. If you think it is rock solid, even if an attacker knows everything about your PHP install, then you could leave it in place.
But really, why would you leave this in place on a production system anyway? There may be exploits you are not aware of in your version of PHP - people may now or in future scan for your version of PHP, or particular options you have enabled, because they know how to carry out these exploits. So by keeping this up publically, you added yourself to their hitlist.
If you want to keep it up, you can put it in a password protected directory, or just switch it on when you need it. Given the small cost of these options, I wouldn't take the risk of keeping it public.