I want to run varnish cache in a docker container on AWS ECS. If the container dies and a new one takes its place the new one should have easy immediate access to the the cached data so it can continue working seamlessly. The best way I can see to do this is for varnish to persist the cache in an S3 bucket (while also keeping some data in memory for speed of course). I imagine I'm not the first to have this idea, and varnish has pluggable storage backends, so is there an existing library that does this? Google isn't helpful as all the results seem to be about putting varnish in front of files served by S3.
No, there's no S3 module currently. Additionally, the overhead of the s3 storage calls may negate some of the speed benefits Varnish will provide.
Using a simple oldschool rsync or cronjob to sync the file periodically to s3, and sync it back on container boot, might be a more performant solution to cache persistence across container death?