The public DNS name (whether elastic IP address or not) is exactly the same as using the public IP address (elastic IP or not) with the one following important difference:
If you query the public DNS name from outside of EC2, it resolves to the public IP address. If you query the public DNS name from inside of EC2, it resolves to the private IP address.
You can use this trick with or without Elastic IP addresses. I recommend using Elastic IP addresses as it keeps the public DNS name the same even after stop/start or moving your service to another EC2 instance. Because of this, you can always use the public DNS name of the Elastic IP address and it will resolve to the internal IP address of the current instance to which the Elastic IP is associated.
You can extend this by using a CNAME DNS entry where you map your preferred hostname to the external DNS name of the Elastic IP.
Here's an article I wrote about using this feature to save money and speed up network performance with internal EC2 communication without having to keep track of the current internal IP address for each instance on all your other instances: http://alestic.com/2009/06/ec2-elastic-ip-internal
Other than this one difference, I agree that you might as well use the public IP address instead of the public DNS name because:
You save time by not doing a DNS lookup
You avoid any security risks that occasionally arise in the DNS protocol.
so I suppose, in reality, right there are two more differences...