For a straightforward, command line managed, simple SMTP email server, I've used Postfix with very little problem.
For a slightly more complicated but very secure mail server you can look at Qmail.
Both once properly set up are fairly simple to run and fairly self-maintaining for the most part. The hard parts include getting logging set up, making sure your MX records are properly set up, and your relaying settings are properly configured. And of course backups.
Postfix is fairly nice to use in that it's easy to plug in things like ClamAV to scan for malware as well as block attachments that are executable and set limits on incoming file sizes. It's also fairly simple to plug in spam blocking and bayesian filtering.
Both of these mail servers are simple to manage from the command line.
Make sure you consider:
redundancy (RAID?)
backups
monitoring (check logs for red flags, maybe nagios for disk space and CPU usages?)
regular updates in case there are holes found that need to be plugged
filtering (attachments, viruses)
Those are the bits I can think of off the top of my head. Mail servers are among the more intensive things at times to administrate because they do take regular monitoring and maintenance, and usually if it goes down it's very visible to the users and they'll get very upset at outages. If you're not one comfortable with adding it to your server monitoring or maintenance rotation, you may want to outsource it or have someone else do it for you. It's not all that horrible necessarily but if you have strong reservations about doing it then it may not be a task you want to tend to. Ignored mail servers become nice targets for abuse and spammers.