It seems to me like these are likely reasons for your stated combination of observations:
you deleted files that are still open, in such a situation the files are no longer visible in the filesystem but the space cannot be reclaimed until they are closed
you may want to look at eg lsof -n | grep DEL
there are a large number of files smaller than what you ...
docker stats reporting pages that are used for disk caching as well as used, which is a bit misleading.
If your project is sensitive for resource utilization, you should definitely use resource requests and limits parameters.
In order to protect your cluster running out of resources, you might also want to enable Resource Quotas
Once you enable it, you'...