You can do this, both sites need to have a valid SSL certificate. This way browsers won't give a security popup. If both sites exist at the same server however, both domains need to be hosted from different IP addresses.
A webserver looks at the "Host" header in the HTTP request to see which site it needs to serve. The SSL negotiation happens before the HTTP request is sent, so at that point the webserver can't tell which website it will display. It will always send the same certificate to the browser.
There are two ways to work around this:
- Have a wildcard certificate for *.example.com, so all subdomains can share the same certificate.
- Run each SSL site at a different IP address. This way, the webserver knows which SSL certificate it can send to the browser. - by inspecting the IP address which received the incoming connection.
Note it's perfectly possible to attach multiple IP addresses to the same network adapter, it's just that you need a second IP adress available in your IP address space.
update: nowadays, you can run multiple SSL sites at a singe IP. To enable this, configure SNI support at your webserver. Most modern browsers (except windows XP, and Android 2) support this.