I haven't great hardware, but my requirements are low, I would like to start using ceph so as to abstract filesystem location and allow potential easy scaling to bigger hardware in an hypothetical future.

My actual hardware meets ceph hardware requirements except the ethernet bandwidth part between the hosts. Mine is 100 Mbit/s which is much lower than the 1Gbps expected in ceph, even from the minimal requirement.

Will I be able to use ceph in a very small smili-prod environnement (with limited number of clients) ?

FYI: My hardware is 2 or maybe 3 hosts having each 4 core Intel, 24Go RAM, 2x2To disks but 100Mpbs between them.

  • If my question is improper in any way, please take time to explain why. – vaab Nov 17 '12 at 16:14
  • Not improper. It's just very clear that 100Mbps falls below the minimum requirements of Ceph. And in 2012, obtaining some basic 1 Gigabit connectivity shouldn't be difficult. – ewwhite Nov 17 '12 at 16:22
  • Thanks for answering. If you are right that this is under the minimum requirement of Ceph, that I've read before posting, it's difficult to know for which type of production environment they are tailoring these requirements. I wanted to know how bad it'll get if I went down to 100Mbps. And this is not explained in the docs I've read. – vaab Nov 17 '12 at 16:27
  • Being quite new to these topics there are for sure some concept that are straightforward. Please bear this candid questions, I'm having several dedicated server between different provider, I don't think 100Mbps to internet is so rare. But I understand now that it seems totally off topic to have this type of scenario with ceph. – vaab Nov 17 '12 at 16:33

Your problem will be during rebalancing. I've once run a setup like that in lab and it topsized while I was doing some exercises. I think I changed my crush map and then all switches went to become a christmas light thing.

You will have a better testing experience running with VMs on the same host. If you don't have an alternative to 100mbit, at least build two networks (front and backend) and limit your test filesystems to be 5GB-ish.

Gigabit is also a limiting factor for ceph, so if you want to learn about how bottlenecks look and could be remedied, there is nothing wrong with trying on 100mbit. But I think you'll grow some grey hair during that.

  • Thanks for your answer. This is a clean answer, even if I would have preferred to hear the opposite conclusion. – vaab Nov 17 '12 at 16:35

For the application, even with a small environment, Ceph is still a cluster/scale-out solution. If you're talking about nodes that are local to each other, there's no excuse not to use Gigabit ethernet.

Either way, the hardware requirements ask for multiple 1 Gigabit (or greater) ports.

Not the answer you're looking for? Browse other questions tagged or ask your own question.