11 votes

Use a wild card in StackDriver Logging

If you want to search for strings contained in text, just type that string directly. Unicorn (text:Unicorn) Finds all log entries containing unicorn, in any field and in any letter case. If you want ...
6 votes

Detecting Kubernetes OOMKilled Events in GKE Logs

Although the OOMKilled event isn't present in the logs, if you can detect that a pod was killed you can then use kubectl get pod -o go-template=... <pod-id> to determine the reason. As an ...
  • 808
4 votes

Unable to get Stackdriver Trace information in Google Cloud Console

You can look directly at the API reporting graphs to see that your calls to google.devtools.cloudtrace.v1.TraceService.PatchTraces fail with 403 Forbidden errors. 403 Forbidden is caused when a ...
  • 301
4 votes

How to disable Stackdriver Monitoring for a project

You need to remove the project from your stackdriver account. This will stop the monitoring and not affect your log collecting solution. To remove the project follow the steps listed here: https://...
4 votes

Stackdriver missing GKE Logs

You need to make sure that your kubernetes cluster has the correct permissions for logging and monitoring. To have a properly running cluster, the kubernetes master needs the following oauth scopes (...
  • 41
4 votes
Accepted

Why no sshd logs on Stackdriver?

Ended up simply adding an additional entry in /etc/google-fluentd/config.d/syslog.conf <source> @type tail # Parse the timestamp, but still collect the entire line as 'message' format /^(...
  • 179
4 votes
Accepted

How to extract raw plain text logs from Stackdriver?

Sorry to answer my own question, but I spent a lot of time researching this and hopefully by posting it here I can save others the hassle. The only way to achieve what I need seems to be use google ...
3 votes

How can I install Stackdriver's monitoring agent on Ubuntu 18.04LTS?

The Stackdriver Monitoring agent currently does not support Ubuntu 18.04 LTS. Operating systems There is no information at present as per when this version will be supported but there's seems to be ...
  • 607
3 votes

Does Google Cloud Firewall log allows and denies?

From the GCP firewall documentation: GCP firewall cannot log as an action. It can only accept or reject a connection. GCP does not collect statistics per rule at this time.
  • 131
3 votes

How to enable Stackdriver Monitoring and Stackdriver trace on existing GKE cluster?

To add to the previous response, it is possible to enable Stackdriver Monitoring by running the previously referenced gcloud alpha container clusters update --monitoring-service=monitoring.googleapis....
  • 331
3 votes

How to disable Stackdriver Monitoring for a project

You can unlink your GCP project from a Stackdriver workspace by Merging the workspace into a new, blank workspace. Before following these steps, you will need to create an empty project in GCP that ...
  • 131
3 votes

Use a wild card in StackDriver Logging

You can check the following common basic search example: Example: A use of string like: uni* (text:uni*) As explained in the above documentation "This is not a wildcard search. This search finds ...
  • 607
3 votes

Monitoring reports incorrect CPU usage on GCP

The CPU usage of main Dashboard of the GCP differs from that of an instance's individual CPU usage. The CPU usage shown in Google Developers Console is not that of the instance, but the CPU usage of ...
3 votes

How do i make stackdriver graphs show 24 hour format dates

The 24h format is not implemented in Stackdriver Monitoring yet. I recommend you to open a Feature Request in issuetracker explaining your needs, and the proper team will evaluate its implementation....
2 votes

Installing StackDriver Logging agent on cos image

At the moment COS images do not support the Stackdriver agent. There is a feature request to allow this functionality. This discussion provides a workaround using “container_vm” image. The image “...
  • 1,385
2 votes

Tailing logs with GCE and stackdriver

The only realtime like thing that you could use is to create a sink from Stackdriver to Pub/Sub and create a consumer that will receive in almost real time, messages from SD; the other options are ...
2 votes

Google Cloud load balancer POST data

The data in all responses and the data in POST requests are not logged. This is the default for most web server/proxy software. To inspect the data in the POST requests you'll need to look from the ...
  • 136
2 votes

How to use CloudWatch or Stackdriver to verify cron job ran?

There is currently no pre-configured metric in Stackdriver to check for snapshots. The workaround is the one you mentioned. Every snapshot that you create on GCP generates two entries in the ...
  • 1,385
2 votes
Accepted

Does Google Cloud Firewall log allows and denies?

This is now possible by enabling firewall rules logging. The logging is enabled in individual basis (for each firewall rule one is interested).
  • 1,385
2 votes
Accepted

How to view Google HTTP load balancer access logs without Stackdriver?

Log entries excluded from Stackdriver Logging will not appear in the Logs Viewer, logs-based metrics, or Stackdriver Error Reporting. The answer is in the same document you provided, as these future ...
  • 168
2 votes

structured logging from a cloud function

I posted this community wiki answer to make the recommendation provided by @Andrew Haines more visible. This question was answered here. Please find solution summary below: // global const { Logging } ...
2 votes
Accepted

Stackdriver monitor for valid HTTPS certificate?

Taking a look on the current documentation of Stackdriver metrics and monitoring agent metrics currently it doesn't support the monitoring of SSL certificates. However, seems that other users have ...
2 votes

Kubernetes Nginx Ingress Controllers log to Stackdriver

The problem is solved following this guide https://dev.to/bzon/send-gke-nginx-ingress-controller-logs-to-stackdriver-2ih4
  • 151
2 votes
Accepted

Set Stackdriver severity based on textPayload

The quick answer is No, you will need to change your code and implement proper logging with the correct severity for each message. You could export your Stackdriver logs to BigQuery, which is normal ...
2 votes

How can I be sure that a VM on GCP was not shut down due to it being preemptible? (after the fact, using stackdriver)

Shorty after asking this question I found https://cloud.google.com/logging/docs/audit/#system_event So filtering for logName="projects/<my-project-name>/logs/cloudaudit.googleapis.com%...
  • 21
2 votes

GCE stackdriver logging agent (fluentd) memory leak with COS

For anyone experiencing similar problems: cos-77-12371-1109-0 running on g1-small uses memory as follows: MiB Mem : 1692.4 total, 1409.3 free, 176.0 used, 107.1 buff/cache Cloud Monitoring ...
1 vote

Can't see pod logs in Stackdriver UI for cluster deployed on GKE

Need to make sure the cluster is set up with Stackdriver logging/monitoring enabled. Following the mentioned documentation, you should have a pod named counter. If you run: $ kubectl logs counter # ...
  • 41
1 vote

How to get the Compute Engine logs to end up in the structured jsonPayload instead of the textPayload?

I see that you would want to print a Json object into the jsonPayload field for Logging. After reading though some documents, here's what I think. You should look into a parser [1], also I wanted to ...
1 vote

GCP Autoscaling based on memory (stackdriver)

This error appears to suggest that you’re using the Stackdriver Basic Tier which allows ‘Logging agent only’ as per the documentation, section ‘Stackdriver account service tiers’. I presume you would ...
  • 283
1 vote
Accepted

How to determine what quota I'm violating?

Check the project number in the error whether it matches with your project number. The quota limit you are hitting may not be from your own projects as you are saying you could not see any ‘Quota’ ...
  • 196

Only top scored, non community-wiki answers of a minimum length are eligible