0

i am looking at the vmstat output to assess certain performance degradation on our systems, specifically for long runqueue. i find frequently it looks like this on a 8 CPU m/c:

Mon Jun 13 12:14:02 2016 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ Mon Jun 13 12:14:02 2016 r b swpd free buff cache si so bi bo in cs us sy id wa st

Mon Jun 13 12:14:02 2016 4 0 0 15270068 1280508 12915056 0 0 0 100 10633 33907 10 8 82 0 0 Mon Jun 13 12:14:12 2016 3 0 0 15269820 1280508 12915308 0 0 0 102 10509 35014 10 8 82 0 0 Mon Jun 13 12:14:22 2016 2 0 0 15269820 1280508 12915568 0 0 0 100 10632 36260 11 8 82 0 0 Mon Jun 13 12:14:32 2016 5 0 0 15269448 1280508 12915828 0 0 0 95 10247 34199 9 7 84 0 0 Mon Jun 13 12:14:42 2016 8 0 0 15278872 1280508 12906312 0 0 0 130 10771 35063 10 8 83 0 0 Mon Jun 13 12:14:52 2016 6 0 0 15278624 1280508 12906556 0 0 0 98 10810 36104 11 8 81 0 0 Mon Jun 13 12:15:02 2016 1 0 0 15278128 1280508 12906824 0 0 0 106 11094 35502 11 8 81 0 0 Mon Jun 13 12:15:12 2016 4 0 0 15278128 1280516 12907080 0 0 0 111 10557 33806 9 7 83 0 0 Mon Jun 13 12:15:22 2016 2 0 0 15277632 1280524 12907316 0 0 0 112 10785 36547 11 8 81 0 0 Mon Jun 13 12:15:32 2016 2 0 0 15277508 1280524 12907576 0 0 0 96 10726 35535 10 8 82 0 0 Mon Jun 13 12:15:42 2016 4 0 0 15277260 1280524 12907848 0 0 0 104 10209 32558 9 7 84 0 0 Mon Jun 13 12:15:52 2016 2 0 0 15276888 1280524 12908116 0 0 0 108 10857 37032 10 8 82 0 0 Mon Jun 13 12:16:02 2016 4 0 0 15276696 1280524 12908312 0 0 0 96 9917 33991 9 7 83 0 0 Mon Jun 13 12:16:12 2016 2 0 0 15276504 1280524 12908612 0 0 0 96 9673 31575 9 7 84 0 0 Mon Jun 13 12:16:22 2016 13 0 0 15276392 1280524 12908852 0 0 0 96 9820 35383 9 7 84 0 0 Mon Jun 13 12:16:32 2016 4 0 0 15275772 1280524 12909120 0 0 0 105 10490 35719 10 8 82 0 0 Mon Jun 13 12:16:42 2016 6 0 0 15276020 1280524 12909360 0 0 0 105 10732 33434 10 7 83 0 0 Mon Jun 13 12:16:52 2016 1 0 0 15270788 1280524 12909592 0 0 0 98 10011 35487 9 7 84 0 0 Mon Jun 13 12:17:02 2016 11 0 0 15275092 1280524 12909884 0 0 0 114 10643 36631 10 8 82 0 0

Mon Jun 13 12:17:12 2016 4 0 0 15275464 1280524 12910148 0 0 0 96 10649 33390 10 7 83 0 0 Mon Jun 13 12:17:22 2016 11 0 0 15274596 1280524 12910408 0 0 0 104 10607 36475 10 8 82 0 0

Mon Jun 13 12:17:52 2016 12 0 0 15276680 1280524 12901428 0 0 0 102 10690 36078 11 9 81 0 0

Mon Jun 13 12:18:02 2016 3 0 0 15277052 1280528 12901688 0 0 0 99 10793 36164 11 8 81 0 0

Mon Jun 13 12:18:12 2016 1 0 0 15276060 1280528 12901936 0 0 0 110 10535 33704 10 7 83 0 0 Mon Jun 13 12:18:22 2016 5 0 0 15275688 1280528 12902192 0 0 0 101 10599 35153 10 8 82 0 0 Mon Jun 13 12:18:32 2016 0 0 0 15275620 1280528 12902448 0 0 0 94 10051 35292 9 7 83 0 0 Mon Jun 13 12:18:42 2016 7 0 0 15275496 1280528 12902712 0 0 0 101 10278 33878 9 7 84 0 0

Mon Jun 13 12:43:32 2016 13 0 0 15170236 1280584 12921492 0 0 0 99 9798 35555 9 7 84 0 0 Mon Jun 13 12:43:42 2016 8 0 0 15169616 1280584 12921752 0 0 0 95 9748 33785 9 7 84 0 0 Mon Jun 13 12:43:52 2016 4 0 0 15176184 1280584 12921952 0 0 0 144 9882 33844 10 7 83 0 0 Mon Jun 13 12:44:02 2016 10 0 0 15176052 1280584 12922260 0 0 0 104 9468 34687 9 7 84 0 0 Mon Jun 13 12:44:12 2016 0 0 0 15175804 1280584 12922488 0 0 0 96 9231 32830 9 7 85 0 0 Mon Jun 13 12:44:22 2016 13 0 0 15175212 1280584 12922764 0 0 0 92 10051 33812 9 7 83 0 0 Mon Jun 13 12:44:32 2016 0 0 0 15175716 1280584 12923020 0 0 0 115 9402 35465 8 7 85 0 0 Mon Jun 13 12:44:42 2016 1 0 0 15174972 1280584 12923308 0 0 0 106 9946 34740 10 7 83 0 0 Mon Jun 13 12:44:52 2016 5 0 0 15171724 1280584 12923536 0 0 0 96 9123 31900 8 6 85 0 0

i've emphasized the timestamps 12:14:42, 12:17:02, 12:17:22, etc. to point out the #procs in runquue

when i look at the times close to this in sar, i don't see similarly long runqueue as below:

-- 29920:Mon Jun 13 12:14:42 2016 12:14:32 runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15

29921-Mon Jun 13 12:14:42 2016 12:14:42 10 1236 2.71 2.90 3.02

37740:Mon Jun 13 12:18:32 2016 12:18:22 runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15

37741-Mon Jun 13 12:18:32 2016 12:18:32 12 1239 3.30 2.92 2.99

91460:Mon Jun 13 12:44:52 2016 12:44:42 runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15

91461-Mon Jun 13 12:44:52 2016 12:44:52 11 1240 3.60 3.52 3.35

110160:Mon Jun 13 12:54:02 2016 12:53:52 runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15

110161-Mon Jun 13 12:54:02 2016 12:54:02 11 1236 3.26 3.11 3.16

could someone explain why i have a seeming discrepancy between the two.

thanks for the response S.R.

4
  • Couldn't it simply be that more jobs are run around those times? That is just a few more processes than on average so it could be a really simple job. Jun 18, 2016 at 23:18
  • yes, there could be a few more processes. however, my question is the difff in runq shown by sar and vmstat for the same time stamp. only at the 4 instances i posted above, sar shows runq length of any significance while vmstat has a number of instances with runq length over 10. e.g., at Mon Jun 13 12:14:42 2016 12:14:42, sar shows runq length of 10 while
    – raven
    Jun 20, 2016 at 17:56
  • yes, possibly few more processes. but, why is the diff in runq shown by sar and vmstat for the same timestamp. only at 4 instances i posted above, sar shows runq length of any significance while vmstat has a number of instances with runq length over 10. e.g., from sar: Mon Jun 13 12:17:02 2016 12:16:52 runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 Mon Jun 13 12:17:02 2016 12:17:02 7 1236 2.29 2.76 2.96 and from vmstat: Mon Jun 13 12:17:02 2016 11 0 0 15275092 1280524 12909884 0 0 0 114 10643 36631 10 8 82. don't they use same krnl counters?
    – raven
    Jun 20, 2016 at 18:18
  • Is it possible that one counts the threads and the other just separate processes? Jun 21, 2016 at 3:47

0

You must log in to answer this question.

Browse other questions tagged .