My answer doesn't differ between virtualized and non-virtualized environments:
Images are useful tools, but they don't make good backups.
In a virtual environment you can do a base snapshot of the system and then store multiple delta snapshots (which would need to be replayed in order).
Eventually you have so many deltas that you need to do another base snapshot.
In a physical environment you can't really do proper snapshots without quiescing the system.
That's fancy-pants talk for "Outage Window".
There are non-outage techniques for snapshotting which produce usable results, but there are LOTS of caveats.
In either case above, you can't pick and choose what you're restoring: You need to restore the last snapshot (plus some deltas) to get the whole system at a specific point in time.
You can't just get back "the one file the CEO deleted this morning that is ABSOLUTELY CRITICAL for this afternoon's presenation" - you have to restore the whole system somewhere and then pluck that file out of it.
This basically means you still need traditional backups if you're using snapshots!
What would I recommend instead? Traditional backups.
(Yes, even on virtual machines. Treat them exactly like a physical host.)
Think about what is involved if you have a hardware failure in each candidate workflow:
- Network boot (or boot off CD) into snapshot recovery software.
- Restore the appropriate base snapshot, plus any incremental snapshots.
- If you have traditional backups, restore them from the snapshot point forward.
Traditional Backups (With Standard Images)
- Network boot (or boot off CD) into imaging software.
- Load standard image (including backup/restore software & needed apps).
- Configure the network as needed.
- Restore from backups.
So we're adding one more step (network configuration), but it's not a big step.
The time to recovery with standard images is pretty close between backups and snapshots, and not much longer with manual installation if your staff is reasonably competent.
How you handle off-site backups is your call - you can certainly go with traditional magnetic tape (or removable hard drives), and Iron Mountain provides tape service where they'll bring you big blue bins of media on a standard rotation.
You can also drop a storage system (NAS, mirrored SAN, computer running some software) at an off-site location, or use commercial solutions like Amazon Glacier or http://rsync.net
Note that you can (and should) use snapshots for one of the cases you described in your question: To have something to revert to in the event an upgrade or other major change goes wrong. Snapshots are great for that.
Prior to doing an upgrade or major change, snapshot critical systems and store the snapshots locally. If you need to revert, simply restore from the image you made.
In this case you're already in an outage window (hopefully, if you're doing it right) so you can shut the system down to take a cold snapshot, and it's a single point-in-time that you're interested in ("before we start making changes"), so none of the potential drawbacks of snapshotting apply.