If all the first host is doing is email sending, then setting
domain.com to the second host is no problem. If the first host is also receiving email then this is what you use MX records for. The
MX record will take care of routing the email while you point the main
A record to the web server. If the first host also has webservices or something for checking your email then I'd probably use a sub-domain for that such as
If you have different webservices on each server and want both under the
www.domain.com namespace then you probably want to setup a reverse proxy on one of the hosts.
Also it's no problem to create an
A record with both IP addresses. However this is probably not what you want for this situation. You would do that to do simple load balancing.