0

I have an Openshift cluster which spans multiple datacenters. Each data center has a zone configured such that a NodeSelector can be used to choose which data center a pod gets allocated in. I also have a StatefulSet which configures 10 pods.

I would like to configure the StatefulSet such that there is typically an even distribution of pods across the data centers. This can't be a strictly defined configuration though, as in the event of a disaster in one data center, I would like the pods to be allocated to the remaining data center.

Is this possible without implementing a custom controller?

0

What you're looking for is affinities / anti-affinities [1], [2]

You should be able to add, in your StatefulSet definition, some mention of an anti-affinity rule:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: openldap
spec:
  selector:
    matchLabels:
      name: openldap
  replicas: 3
  template:
    metadata:
      labels:
        name: openldap
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: name
                  operator: In
                  values: [ openldap ]
              topologyKey: kubernetes.io/hostname
            weight: 42
...

The above would ensure that the scheduler won't place two Pods with a name=openldap label, onto machines sharing the same topologyKey.

While common samples would refer to kubernetes.io/hostname as a topology key, you could very well rely on some custom label. Say your nodes from DC-A have a label dc=a, and DC-B dc=b, then you could use dc as your topologyKey.

Note the above sample expresses a preference, rather than a mandatory rule. Instead of preferredDuringSchedulingIgnoredDuringExecution (or preferredDuringSchedulingRequiredDuringExecution), you could use requiredDuringSchedulingIgnoredDuringExecution or requiredDuringSchedulingRequiredDuringExecution -- whith a slightly different syntax, you won't be able to set weights on mandatory placement rules, ...

[1] https://docs.openshift.com/container-platform/3.11/admin_guide/scheduling/pod_affinity.html#admin-guide-sched-affinity-config-pod-pref

[2] https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.