As a reference, this page from the VM subsystem author will give you a good idea what is likely to happen;
Note that this is actually very difficult to answer in its current form (if your are talking about file backed caches) because its dependant on how hot the cache is, how long you cache items for and the 'heat' of each object in the cache.
Assuming the following:-
- The cache is backed by a 10G file and has been completely filled.
- 3% of the objects in the cache make up 90% of all hits.
- You have a concurrency of around 80 handled connections at any given time.
- You have not changed your VM policy from the default.
- Before the command the pagecache is nearly almost exclusively filled up with pages from your varnish cache.
- 500M of data is exclusively allocated to anonymous pages from other things running on the system.
As your varnish cache size is 10G it will never fit completely in memory, thus the following formula is relatively representative
- There is 3500M of page cache.
- The hottest 1750Mb of varnish cache is in the 'active' list and protected from pagecache eviction.
- The cooler 1750Mb of varnish cache is in the 'inactive' list and not protected from eviction.
- The coldest 6500Mb of varnish cache does not reside in memory and lives on the disk somewhere.
So the following is probably the result for all of your commands you run..
- This file is placed into memory and cached, but all new objects in the cache are sent to the 'inactive' list by default.
- This evicts 1750MB of your 'cooler/inactive' memory from the varnish cache and replaces it with the catted file.
- The kernel is now forced to write out 1750M of this data back to the disk (in the absolute worst case scenario).
- IO Wait and device utilization takes a hit because you are reading in a 2G file and writing out a 1750M file.
- 97% of inbound requests are unaffected by this because they want the hottest 1750Mb of varnish data which is in the active portion of page cache!
- 3% of unlucky clients want data in the cooler cache. These guys see a delay now because the disk utilization is already pretty high and they are queueing to fetch back pages into page cache again! Since the catted file is never re-read quickly enough page cache evicts those pages in favour of the 3% of clients wanting some cooler data.
This is the absolute worst case scenario. So, broadly speaking -- whats the impact?
There is no negligable impact for 97% of your served requests.
Of the 3% that are affected, expect a higher delay in those being served up -- probably 500 millisecond.
BUT of those 3% of unlucky requests around 2% of them anyway would have been slow because they wanted something out of the 6500Mb cache that was never in pagecache anyway! They however do suffer from the high disk utilization now.
So, in summary with my contrived and assuming example you'll see in broad terms about a 3% loss in efficiency. (100% efficient would be all objects for every request are served out of memory).
In 'normal' running in this contrived example performance would be about 98% efficient.
Not bad for a cache that doesn't fit into memory!