0

As the question says. Is there a way to find out how many files can Ceph delete per 1 minute?

A little backstory: Image hosting website, running Ceph FS on 8TB WD RED 3.5 HDD's, around 51 TiB already used with 40408370 stored objects. All files are images. Currently, we are deleting 300 images per minute. Is it a lot, is it too little? Is there a way to find out the optimal number for deleting images?

Improve this question
1
  • If you’re actually reaching a performance bottleneck one work around redhat.com/en/blog/scaling-ceph-billion-objects-and-beyond For bulk delete operations, S3 object deletion API found to be significantly slower compared to Ceph Rados APIs. As such we recommend using either object expiration bucket lifecycles, or the radosgw-admin tool for bulk delete operations.
    – Bob
    Feb 25, 2021 at 10:09

1 Answer 1

Reset to default
0

The deletion process itself happens with a delay, and the bottleneck with current ceph versions is the performance of rocksDB (OSD metadata). IIRC it only can process 15 objects per second, not very much. Even with fast devices for rocksDB/WAL you could quickly run into severe performance issues on the ceph cluster. I've seen this several times and I believe the rocksDB improvements will land in Ceph Pacific. So my suggestion would be to monitor the ceph cluster while doing deletions and get an impression of the impact.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .