2

I'm trying to deploy Postgres with a PersistentVolumeClaim in GKE. As per the description here (look at PGDATA), it is recommended to create a subdirectory if mounting an external volume. So, this is what I did with my PG deployment:

spec:
  containers:
    - name: postgres-dev
      image: "postgres:12.1"
      imagePullPolicy: IfNotPresent
      ports:
        - containerPort: 5432
      env:
        - name: PGDATA
          value: "/var/lib/postgresql/data/pgdata"
      volumeMounts:
        - name: postgres-persistent-storage-dev
          mountPath: "/var/lib/postgresql/data/pgdata"
  volumes:
    - name: postgres-persistent-storage-dev
      persistentVolumeClaim:
        claimName: postgres-pvc-dev

I create the /pgdata subdir in /postgresql/data, which would be the original mount point for Postgres' data.

But when I try to deploy the Postgres pod, it stalls at Back-off error, and I get the following events:

Events:
  Type     Reason                  Age               From                                                  Message
  ----     ------                  ----              ----                                                  -------
  Normal   Scheduled               20s               default-scheduler                                     Successfully assigned default/postgres-dev-7f549754d-nm2lf to gke-cluster-pool-1-4ce1e4d6-m33c
  Normal   SuccessfulAttachVolume  15s               attachdetach-controller                               AttachVolume.Attach succeeded for volume "pvc-eea8ee4d-9d00-11ea-a854-42010a8a0057"
  Normal   Pulled                  9s (x2 over 10s)  kubelet, gke-cluster-pool-1-4ce1e4d6-m33c             Container image "postgres:12.1" already present on machine
  Normal   Created                 9s (x2 over 10s)  kubelet, gke-cluster-pool-1-4ce1e4d6-m33c             Created container postgres-dev
  Normal   Started                 9s (x2 over 10s)  kubelet, gke-cluster-pool-1-4ce1e4d6-m33c             Started container postgres-dev
  Warning  BackOff                 7s (x2 over 8s)   kubelet, gke-cluster-pool-1-4ce1e4d6-m33c             Back-off restarting failed container

These are the logs from the Postgres container:

initdb: error: directory "/var/lib/postgresql/data/pgdata" exists but is not empty
It contains a lost+found directory, perhaps due to it being a mount point.
Using a mount point directly as the data directory is not recommended.
Create a subdirectory under the mount point.

So, I create the subdir for mountpoint, as recommented for initdb, and then it complains about it. The PVC was created just right before the deployment, so there is no data in it.

I tried with completely new, un-existent dir like /var/lib/postgres/data as well, but the effect is the same.

Another interesting point is that when I set this up in minikube, everything works fine.

How do I approach this?

2 Answers 2

7

After some more careful reading of the Postgres error log, I realized that the problem is that I set the mount point path and the data path to be the same dir. This specific part of the log message in the Postgres container is the crucial one:

Using a mount point directly as the data directory is not recommended.
Create a subdirectory under the mount point.

I changed the value of PGDATA env var to be /var/lib/postgresql/data/pgdata (which is the DB data dir) and the mount point to be /var/lib/postgresql/data (which is the dir that Postgres initializes itself from).

After this change, it worked like a charm.

1

You are declaring the PGDATA field that maybe the cause of the issue. I faced the same error, this comes because there's as LOST+FOUND folder already in that directory however, the container wants it to be a empty dir. Giving the subPath field solves this issue. Please try this it should solve the issue and you need not need any PGDATA field. Add subPath to some folder. Please go through following manifests.

https://github.com/mendix/kubernetes-howto/blob/master/postgres-deployment.yaml

https://www.bmc.com/blogs/kubernetes-postgresql/

it's a statefulset that usually you should go with and not a deployment when it comes to Database deployment.

        - name: postgredb
          mountPath: /var/lib/postgresql/data
         #setting subPath will fix your issue it can be pgdata or 
         #postgres or any other folder name according to your 
         #choice.
          subPath: postgres 

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .