I brought up a new Vault cluster using Terraform (https://github.com/sethvargo/vault-on-gke) and everything is working fine with Vault. However I can't seem to see any Container, Node or Pod logs in Stackdriver. The other GKE Cluster we have that was brought up manually is working fine.

I can view the logs on the pods using kubectl logs without a problem.

Existing cluster that works fine

Terraformed vault cluster

I'm completely stumped as to why it's not working. Stackdriver is enabled for the cluster, and the beta Stackdriver box is ticked - same as the working cluster. The Compute service account has Editor role, same in both account.

Any thoughts?

If you are looking for the equivalent of kubectl logs, you should be looking in 'kubernetes pod' or 'kubernetes container' for the logs. 'kubernetes cluster' shows you cluster activity such as resource patches and updates.

The next thing to check is to make sure that logging shows 'Enabled v2(beta)' and that the stackdriver agent is properly running in your cluster using 'kubectl get po -n kube-system | grep fluentd'

If all the above checks out, I recommend creating a public issue tracker as there is likely something wrong with either the cluster version of the beta logging

  • That's exactly my problem. I don't have Kubernetes pod or Kubernetes container showing up in Stackdriver for this cluster. Look at the screen shot in the link Terraformed vault cluster. – Max DiOrio Nov 8 at 17:39
  • Sorry about that Max, I've added more steps. This looks like a bug to me – Patrick W Nov 8 at 20:11
  • Beta is enabled and fluentd is running on all nodes. Thanks for the info. – Max DiOrio Nov 9 at 21:10

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.