We use Mac OS X Server extensively, and have it deeply intertwined with our Linux infrastructure. We use Open Directory to do all user authorization and authentication for 3 distributions of Linux, a windows box, a cisco VPN, and of course the mac clients. We use Xserves running AFP and NFS servers to serve home directories (from an iSCSI SAN) out to mac and linux clients.
Apple is opaque as a company, and that's really frustrating as an enterprise. We don't know what's in their product pipeline, and don't get very detailed information about what changes in OS updates. I think it was 10.5.2 when there was a widespread problem that caused our AFP servers to crash several times a day, and Apple Support's only comfort was We're working on a fix, but can't say when it will be released. That was really bad.
We mitigate these problems by having really good testing and change management procedures. We treat every Apple update like toxic material until we convince ourselves that it won't break our environment. We use radmind to deploy an image on all workstations, and can use it to rollback software updates quickly and precisely. We make good use of the community with sites like Server Fault, AFP548, Mac OS X Hints, and the like.
None of these qualms with Apple are specific to their server offerings. They are just as secretive and vague with their consumer-targeted stuff. I don't expect they will exit the enterprise market completely, as they continue to pile on new features to OS X Server that compliment iPhone (Push email, iCal Server, Address Book Server).
My hope is that they try to do less themselves, and open parts of their Server offerings to other companies (as they have with their fiber channel RAIDs) or the community (as they have with large portions of their software). Then I think we will be getting the best of both worlds.