I'm using a dual-head ZFS-backed NAS for high availability cluster shared storage, based on Nexenta's recommended architecture as seen here:
The disks in 1 JBOD will store the database files for a single 4 TB Postgres database and the disks in the other JBOD store 20 TB of large raw binary flat files (cluster results for large stellar object collision simulations). In other words, the JBOD backing the Postgres files will handle mainly random workloads while the JBOD backing the simulation results will handle mainly serial workloads. Both head nodes have 256 GB of memory and 16 cores. The cluster has about 200 cores each maintaining a Postgres session, so I expect about 200 concurrent sessions.
I'm wondering if it's wise in my setup to have the ZFS head nodes act simultaneously as a mirrored pair of Postgres database servers for my cluster? The only drawbacks I can see are:
- Less flexibility for scaling my infrastructure.
- Slightly lower level of redundancy.
- Limited memory and CPU resources for Postgres.
However, the advantage I see is that ZFS is pretty dumb about automatic failover anyway and I don't have to spend a lot of work getting each Postgres database server to figure out if a head node has failed since it will fail together with the head node.