What I've tried:
- I have two email storage architectures. Old and new.
- courier-imapds on several (18+) 1TB-storage servers.
- If one of them show signs of running out of disk space, we migrate a few email accounts to another server.
- the servers don't have replicas. no backups either.
- dovecot2 on a single huge server with 16TB (SATA) storage and a few SSDs
- we store fresh mails on the SSDs and run a doveadm purge to move mails older than a day to the SATA disks
- there is an identical server which has a max-15min-old rsync backup from the primary server
- higher-ups/management wanted to pack in as much storage as possible per server in order to minimise the cost of SSDs per server
- the rsync'ing is done because GlusterFS wasn't replicating well under that high small/random-IO.
- scaling out was expected to be done with provisioning another pair of such huge servers
- on facing disk-crunch issues like in the old architecture, manual moving of email accounts would be done.
- I'm not convinced with the synchronously-replicated filesystem idea works well for heavy random/small-IO. GlusterFS isn't working for us yet, I'm not sure if there's another filesystem out there for this use case. The idea was to keep identical pairs and use DNS round-robin for email delivery and IMAP/POP3 access. And if one the servers went down for whatever reasons (planned/unplanned), we'd move the IP to the other server in the pair.
- In filesystems like Lustre, I get the advantage of a single namespace whereby I do not have to worry about manually migrating accounts around and updating MAILHOME paths and other metadata/data.
- What are the typical methods used to scale up/out with the traditional software (courier-imapd / dovecot)?
- Do traditional software that store on a locally mounted filesystem pose a roadblock to scale out with minimal "problems"? Does one have to re-write (parts of) these to work with an object-storage of some sort - such as OpenStack object storage?