Sometimes the appliance comes with specialized hardware that I'd have no possibility of duplicating, such as with the F5 BigIP hardware load-balancers. In those cases, the appliance is well worth the effort. The Cisco 6509 in our DC is a whonking big appliance with very specialized hardware, and I wouldn't dream of replacing that with software/hardware I can rip into.
Recently, appliance vendors have been offering VMWare images instead of hardware for those that just want to drop a binary blob in their ESX config instead of dealing with multiple physical servers. We have one of those right now. This makes it easier to scale up an appliance, since all you have to do is add resources it to it in the VIC.
Appliances, especially virtual appliances, are not a whole lot different than closed-source COTS. Their footprint is a lot bigger, and the debugging surfaces tend to be a LOT smaller, but not much different.
I don't mind them.
Except... each appliance is yet another discrete server that probably won't fit easily into our patching rotation and schedules. A lot of appliances really lag behind on certain OS patches, which can be a real problem. It is for this reason that I don't like using appliances unless there is no choice, or the platform for the appliance is one that isn't being actively exploited as is the case with Linux appliances.