I just installed LAMP and Virtualmin on my Amazon EC2 instance. My domain name is mapped to the external IP and now everything is working well. But I feel like it is not very secure. When I visit my external IP, it just shows the same page as my domain. Am I supposed to hide it? And when I go to mydomain.com/phpmyadmin, the php page doesn't run and it's downloaded to my computer.
Why would you want hide it? The IP and your website are public information. There is no security to be gained by trying to hide either.
How did you install phpmyadmin?
This problem indicates that php scripts are not configured to execute in that virtual server.
If you installed using apt-get, it would definitely not be expected to work in a Virtualmin default configuration (though you'd get a 501 error instead of downloading the file), because Virtualmin sets up suexec for all virtual servers. Suexec is a security feature that causes scripts to be executed as the owner of the virtual server account rather than the Apache user. The phpmyadmin package from Debian/Ubuntu apt repositories runs from within the default document root, and would need to run as the apache user. So, you'll need to either disable suexec for the virtual server you want to run it under, setup a virtual server specifically for this install of phpmyadmin that has suexec disabled, or install phpmyadmin in the public_html directory of the virtual server you want it to run under and owned by that user. Suexec will not run applications not owned by the owner of the domain or outside of the suexec docroot (and your phpmyadmin is presumably owned by the apache user and outside of the suexec docroot).
But, your specific problem is that php scripts are not being executed in the location where phpmyadmin is installed.
As for security, there are three things that are the most common sources of exploited servers in the wild, and they aren't related to any of your questions:
These three things alone would prevent 99% of the exploited systems I've seen in my 15 years of system administration. Don't get distracted by security theater until you've accomplished those three things and have made them a part of your daily sysadmin routine. After that, you can worry all you want about other stuff (but it probably won't make a big difference in the security of your server to worry about a bunch of other stuff).
Edit the master php.ini to secure some values (e.g. in my virtualmin disable_functions was empty by default), you might even deny php.ini edits if you really want security and your users could live wothout that.
If you don't have any special needs for mails just completely disable mails and use google apps, before you are doing that change consider the limitation (e.g. http://www.google.com/support/a/bin/answer.py?answer=166852&&hl=en)
As quoted here - How can I prevent Virtualmin from storing passwords in cleartext? - it's now possible to hash passwords, consider that.
Please also take a look at the question Centos Dedicated Server Security
Just google for normal good security approaches (e.g. deny remote root ssh access)
For the phpmyadmin issue, it may not be installed correctly. The server prompts you to download the code if it is not configured to run it. You need to check your configuration, or you can try re-installing phpmyadmin.
To secure your web server like apache, you may do the following:
There should be more things to check/do.