You can configure the total memory to be used for AppFabric caching on each cache host in a cluster using the Set-CacheHostConfig Powershell cmdlet. Use the CacheSize parameter (quoted in MB) e.g.
Set-CacheHostConfig -HostName MyCacheServer -CachePort 22233 -CacheSize 1024
Each cache host also has two memory thresholds, the low and high water marks, expressed as a percentage. I can't find it explicitly written down anywhere on MSDN, but I assume these are percentages of the cache size, rather than the percentage of total memory on the machine. Again, you can tune these settings with
As stated in Expiration and Eviction, once the low water mark is reached, expired objects are evicted from the cache (which to me raises the question of why they aren't evicted once they expire but that's for another day) on a Least-Recently-Used basis. If the high water mark is reached, unexpired items will be evicted until the low water mark is reached.