2

Got a disturbing crash while performing 'apachectl stop'. General system:

$ uname -a 
Linux www.example.com 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -d
Description:    Ubuntu 14.04 LTS

Lots of spare capacity in disk, memory, CPU. This is an Amazon EC2 cloud instance, running today at 1PM 5/7/2014, region us-east-1a, medium-size instance with 3.7GB mem/2CPU. My other instances on the same VPC and same region were fine.

I read elsewhere that in today's kernels, you don't get a crash like this unless the hardware is failing. Seems unlikely that Amazon would have faulty hardware in the cloud?? Or am I being pollyannish?

Anyway, the dump from dmesg (the system continued to operate by serving webpages and talking to the database, but new processes hung instantly, such as ps and ssh):

[27917995.400499] general protection fault: 0000 [#1] SMP 
[27917995.400515] Modules linked in: isofs crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd
[27917995.400537] CPU: 0 PID: 1672 Comm: apache2 Not tainted 3.13.0-24-generic #46-Ubuntu
[27917995.400545] task: ffff8800020117f0 ti: ffff88005f012000 task.ti: ffff88005f012000
[27917995.400551] RIP: e030:[]  [] devpts_kill_index+0x13/0x60
[27917995.400564] RSP: e02b:ffff88005f013d58  EFLAGS: 00010286
[27917995.400568] RAX: dc73af5e3df7dcab RBX: ffff880003f30400 RCX: 0000000181000079
[27917995.400574] RDX: 00000000ffffffff RSI: 0000000000000002 RDI: ffff8800aab76ff8
[27917995.400579] RBP: ffff88005f013d68 R08: 0000000000000000 R09: 0000000000000001
[27917995.400583] R10: ffffea0003a01180 R11: ffffffff8144a320 R12: 0000000000000002
[27917995.400588] R13: ffff8800e87a8001 R14: 0000000000000002 R15: 0000000000000001
[27917995.400598] FS:  00007f8d8b320780(0000) GS:ffff8800ef600000(0000) knlGS:0000000000000000
[27917995.400605] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[27917995.400610] CR2: 00007f8d79aea7e0 CR3: 0000000001c0e000 CR4: 0000000000002660
[27917995.400616] Stack:
[27917995.400619]  ffff880003f30400 ffff880003f30800 ffff88005f013d78 ffffffff8144caa8
[27917995.400628]  ffff88005f013d90 ffffffff81440e47 ffff880003f30400 ffff88005f013e38
[27917995.400636]  ffffffff81443159 ffff880003f30610 ffff880003f30628 ffff880003f30630
[27917995.400645] Call Trace:
[27917995.400656]  [] pty_unix98_shutdown+0x18/0x20
[27917995.400662]  [] release_tty+0x37/0x140
[27917995.400668]  [] tty_release+0x4b9/0x600
[27917995.400678]  [] __fput+0xe4/0x260
[27917995.400684]  [] ____fput+0xe/0x10
[27917995.400693]  [] task_work_run+0xc4/0xe0
[27917995.400701]  [] do_exit+0x2ab/0xa50
[27917995.400708]  [] ? vtime_account_user+0x54/0x60
[27917995.400717]  [] ? context_tracking_user_exit+0x4f/0xc0
[27917995.400723]  [] do_group_exit+0x3f/0xa0
[27917995.400729]  [] SyS_exit_group+0x14/0x20
[27917995.400738]  [] tracesys+0xe1/0xe6
[27917995.400742] Code: 0f 1f 84 00 00 00 00 00 48 83 c4 08 b8 fb ff ff ff 5b 41 5c 5d c3 66 90 66 66 66 66 90 55 48 89 e5 41 54 41 89 f4 53 48 8b 47 28  81 78 58 d1 1c 00 00 74 0b 48 8b 05 44 bf d7 00 48 8b 40 08 
[27917995.400796] RIP  [] devpts_kill_index+0x13/0x60
[27917995.400803]  RSP 
[27917995.400811] ---[ end trace 5b24303912015285 ]---
[27917995.400815] Fixing recursive fault but reboot is needed!

1 Answer 1

0

The cloud is made up of hardware still so a hardware fault is entirely possible. If you suspect a hardware issues just stop and restart the instance. This should put you on a new host.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .