Flashcache, for those who haven't seen it before, is a method for extending the Linux block-cache with a SSD drive. It's cheaper than running a server with a half TB of RAM just for caching.
Will this even work?
It should. The Linux block-cache works by caching accessed blocks, not files. So long as you're not giving the KVM machines direct access to the block devices (you're not), the Linux Block Cache will be in play. However, if you are giving KVM machines direct block-device access the answer there is less clear.
If you're using file-backed virtual-disks, it'll definitely work.
If you're using LV-backed virtual-disks, I don't know.
How much would I expect to increase performance?
That is something we can't answer. It depends on a variety of things. In the abstract, you'll get the best performance for sizing your SSD to be larger than the active-set of blocks. If you get perfect caching, your performance will be similar to running your entire system on SSDs. Which you'll effectively be doing.
How big does the SSD needs to be?
Finding out the exact size you need is something we can't help with. More is better, obviously, but finding the exact ratio between cache-SSD and primary storage it not a simple matter.
Complicating this are writes set to flush immediately, such as certain file-system operations and some database configurations. Those writes will only be briefly cached, and their performance will not be affected in any way by the presence or absence of flashcache.
What happens if the SSD dies?
The same thing happens when you tell Linux to drop-caches but with a twist. With drop-caches, any unflushed writes that are in the block-cache will get flushed to disk. What happens when the SSD disappears depends on the caching mode:
Writethrough: All writes are written to the cache and primary storage in parallel, so the chances of a sudden SSD loss causing errors on the VMs are very small.
Writearound: All writes are written to primary storage and only cached when read. No chance of errors in the VMs.
Writeback: All writes go to the Cache first, and are written to primary storage in the background. The most likely to cause errors in your VMs should the SSD fail, and I wouldn't use this mode in production.
How much faster would writeback be in comparison with writethrough and writearound?
Depends on how much writing you're doing. If your writes periodically saturate your primary storage, the performance increase could be rather significant. If you're mostly read with some write, you'll not likely notice improvements.
Also, writeback is a bad policy for what you're doing so don't use it.