So the job listing I answered for my current job actually listed the title as "IT Generalist." I work for a Managed Services Provider - basically I'm outsourced IT. We have specialties within what we do - I, for instance, do most of our Exchange work - but we all pick up the slack where it's needed. In larger organizations, such as my best friend just went to work for, there are teams that are responsible for specific areas of the infrastructure. He's on the web services team, maintaining the servers that drive the web application. There's a helpdesk team there, and also a more generic infrastructure team that deals with non-client facing systems. This works for them because they have 3000+ employees and a large IT staff.
There's definitely a place for specialization. That place, to me, is within a very large team. If you have 30 admins, you can afford to have 5 of them be the mailserver group. There's coverage if one is out sick or on vacation, and enough institutional knowledge that if one leaves, the infrastructure won't crash.
In a 3 person org, such as you were in, you HAVE to be more of a generalist. You may well be the best Exchange admin on earth, but when the fileserver goes down and Bob The Fileserver Guy is out on paternity leave, management is going to turn to you to fix it. So you need to have the knowledge to figure out what's going wrong, and make a stab at fixing it. Throwing your hands up in the air and saying "I'm the email guy" does not help your organization (or impress your boss).
If you genuinely love one aspect of IT - networking, Linux Samba administration, or whatever - great. Specialize in that. But know that you need to know how the rest of your environment works or your organization runs the risk of lack of coverage, and, more importantly to me at least, your troubleshooting will take much longer. Also - products change, concepts don't. If you love email and admin Exchange - great. Concentrate on the email aspects of things. The next version of Exchange may kill the product line, and then where will you be? I tend to view generalization as a hedge against unemployment a bit too.