2

Is there a way to visualize current CPU usage of a pod in a K8S cluster?

I tried different Prometheus metrics like namespace_pod_name_container_name:container_cpu_usage_seconds_total:sum_rate and other similar ones, but I always get average value for the last 5 minutes, so I have "stairs" on my graphs even if workload raises abruptly (please, see the screenshot).

At the same time, kubectl top pod shows more precise values, so when I raise workload to 100%, it starts showing 100% utilization in a few seconds.

Is there a way to provide these values to Grafana?

Thanks!

1
  • namespace_pod_name_container_name:container_cpu_usage_seconds_total:sum_rate is a prometheus recording rule, for which there is a corresponding query, which you can find by seeing the recording rules in Prometheus; The recording rule most likely has an interval set for 5m with [5m]; Hence the average was for 5minutes Oct 26, 2023 at 9:03

2 Answers 2

2

If I understand you correctly you can try the query below:

sum (rate (container_cpu_usage_seconds_total{image!=""}[1m])) by (pod_name)

This would track the CPU usage of each of the pods and the results would be shown in 1 minute rate. Of course you can adjust the [1m] parameter (and others) as you need.

Please let me know if that helped.

0

in my case only sum works but when I add rate with [15m] then it gives error on the by (pod)

Did you miss the sum_rate in your solution? In my case I use sum_irate

sum( node_namespace_pod_container:container_cpu_usage_seconds_total:sum_irate{cluster="$cluster", namespace="seet"} ( * = multiply symbol) on(namespace,pod) group_left(workload, workload_type) namespace_workload_pod:kube_pod_owner:relabel{cluster="$cluster", namespace="seet", workload="seetbackend", workload_type="deployment"} ) by (pod)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .