Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

One of my Mysql replication servers has rebooted during the night and i'm trying to find the reason. Running the command last shows this line: reboot system boot 2.6.32-71.29.1.e Wed Jul 31 04:37 - 11:02 (06:24) I'm trying to read the vmcore file which has been created during the reboot. I've installed kernel-debuginfo package which created the vmlinux file. then I ran:

[root@dbs ~]# crash -s /usr/lib/debug/lib/modules/2.6.32-358.14.1.el6.centos.plus.x86_64/vmlinux /var/crash/127.0.0.1-2013-07-31-01:33:32/vmcore
crash 6.1.0-1.el6
crash: invalid kernel virtual address: 68f8e06b73fe5f1f  type: "possible"
WARNING: cannot read cpu_possible_map
crash: invalid kernel virtual address: fce96c6b73fe5f1f  type: "present"
WARNING: cannot read cpu_present_map
crash: invalid kernel virtual address: e10de07591befc69  type: "online"
WARNING: cannot read cpu_online_map
crash: /usr/lib/debug/lib/modules/2 and /var/crash/127.0.0.1-2013-07-31-01:33:32/vmcore do not match!

Usage:

  crash [OPTION]... NAMELIST MEMORY-IMAGE  (dumpfile form)
  crash [OPTION]... [NAMELIST]             (live system form)

Enter "crash -h" for details.
[root@dbs ~]# 

Can anyone please point me to what I'm doing wrong? I wonder how come vmcore and vmlinux do not much as the vmlinux was generated when I installed the kernel-debuginfo package...

[root@dbs 2.6.32-358.14.1.el6.centos.plus.x86_64]# rpm -qa |grep kernel-debug
kernel-debuginfo-common-x86_64-2.6.32-358.14.1.el6.centos.plus.x86_64
kernel-debug-2.6.32-358.14.1.el6.x86_64
kernel-debuginfo-2.6.32-358.14.1.el6.centos.plus.x86_64
[root@dbs 2.6.32-358.14.1.el6.centos.plus.x86_64]# uname -r
2.6.32-71.29.1.el6.x86_64
[root@dbs 2.6.32-358.14.1.el6.centos.plus.x86_64]#

Why did the kernel-debuginfo package generated the wrong vmlinux? uname -r shows kernel version 2.6.32-71.29.1 while the vmlinux which has been generated is of version 2.6.32-358.14.1 ?

[root@dbs ~]# yum list installed |grep kernel
abrt-addon-kerneloops.x86_64
dracut-kernel.noarch    004-33.2.el6_0  @updates                                
kernel.x86_64           2.6.32-71.el6   @anaconda-CentOS-201106060106.x86_64/6.0
kernel.x86_64           2.6.32-71.29.1.el6
kernel-debuginfo.x86_64 2.6.32-358.14.1.el6.centos.plus
kernel-debuginfo-common-x86_64.x86_64
kernel-headers.x86_64   2.6.32-71.29.1.el6
[root@dbs ~]#
share|improve this question
    
2.6.32-71.29.1.e is that some custom complied kernel? –  ALex_hha Jul 31 '13 at 8:28
    
As far as I know, not. –  Itai Ganot Jul 31 '13 at 8:34
    
I've added some more information to the question –  Itai Ganot Jul 31 '13 at 8:46
    
# yum list installed | grep kernel ? –  ALex_hha Jul 31 '13 at 9:15
    
Added to the post –  Itai Ganot Jul 31 '13 at 9:27
add comment

1 Answer

I found the solution! The steps I did:

[root@dbs ~]# uname -r
2.6.32-71.29.1.el6.x86_64
[root@dbs ~]#

I ran: [root@dbs ~]# yum provides kernel-debuginfo-2.6.32 And manually looked for kernel-debuginfo-2.6.32-71.29.1.el6.x86_64 I then ran: rpm -qa |grep ^kernel and yum remove'ed the current installed wrong version of kernel-debuginfo. Then I installed the correct kernel-debuginfo-2.6.32-71.29.1.el6.x86_64, ran updatedb and then locate'd the new vmlinux with the correct version. Then I used the next command: crash /usr/lib/debug/lib/modules/2.6.32-71.29.1.el6.x86_64/vmlinux /var/crash/127.0.0.1-2013-07-31-01\:33\:32/vmcore and the crash report ran successfully. Thanks for your help @ALex_hha .

share|improve this answer
    
I am having same issue. I checked the kernel version and the debug info kernel package version are same, however, I am still getting the error crash: page excluded: kernel virtual address: ffffffff80488620 type: "present" WARNING: cannot read cpu_present_map crash: page excluded: kernel virtual address: ffffffff804832e0 type: "online" WARNING: cannot read cpu_online_map crash: page excluded: kernel virtual address: ffffffff8048a240 type: "xtime" –  chandank Aug 20 '13 at 20:42
    
Does the running kernel need to be the debug kernel? If I run the debug kernel and then panic it, I am able to get it work with the dump, however, if I run the normal kernel and then panic it to get the dump it does not work –  chandank Aug 20 '13 at 21:27
    
I used it on a regular kernel, not a debug one –  Itai Ganot Aug 21 '13 at 6:55
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.