Kernel panic. General protection fault

Администрирую несколько веб-серверов на Gentoo. На днях ни с того ни с сего приключился kernel panic.

Вот кусок dmesg по этому поводу:

Aug  3 03:10:01 fastvps0 kernel: [4215962.078076] general protection fault: 0000 [#1] SMP 
Aug  3 03:10:01 fastvps0 kernel: [4215962.078248] last sysfs file: /sys/devices/pci0000:00/0000:00:1f.3/i2c-0/name
Aug  3 03:10:01 fastvps0 kernel: [4215962.078567] CPU 3 
Aug  3 03:10:01 fastvps0 kernel: [4215962.078574] Modules linked in: tun dm_snapshot scsi_wait_scan
Aug  3 03:10:01 fastvps0 kernel: [4215962.078903] 
Aug  3 03:10:01 fastvps0 kernel: [4215962.079063] Pid: 30230, comm: apache2 Not tainted 2.6.36-gentoo-r5 #1 MSI X58 Pro-E (MS-7522)/MS-7522
Aug  3 03:10:01 fastvps0 kernel: [4215962.079388] RIP: 0010:[<ffffffff810310a7>]  [<ffffffff810310a7>] task_rq_lock+0x22/0x6b
Aug  3 03:10:01 fastvps0 kernel: [4215962.079718] RSP: 0018:ffff8802bf53fdd8  EFLAGS: 00010046
Aug  3 03:10:01 fastvps0 kernel: [4215962.079883] RAX: 0000000000000246 RBX: 0000000000012d80 RCX: 0000000000000057
Aug  3 03:10:01 fastvps0 kernel: [4215962.080204] RDX: 0000000000000000 RSI: ffff8802bf53fe20 RDI: 2e66c3f300459a40
Aug  3 03:10:01 fastvps0 kernel: [4215962.080525] RBP: ffff8802bf53fdf8 R08: ffff8802bf53fd88 R09: 00000002bf53fde8
Aug  3 03:10:01 fastvps0 kernel: [4215962.080846] R10: 0000000000000400 R11: ffffffff81b802b0 R12: 0000000000012d80
Aug  3 03:10:01 fastvps0 kernel: [4215962.081166] R13: ffff8802bf53fe20 R14: 2e66c3f300459a40 R15: 0000000000000003
Aug  3 03:10:01 fastvps0 kernel: [4215962.081486] FS:  00007f3ef36c5740(0000) GS:ffff880001ec0000(0000) knlGS:0000000000000000
Aug  3 03:10:01 fastvps0 kernel: [4215962.081808] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug  3 03:10:01 fastvps0 kernel: [4215962.081975] CR2: 0000000001040310 CR3: 0000000261f49000 CR4: 00000000000006e0
Aug  3 03:10:01 fastvps0 kernel: [4215962.082296] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Aug  3 03:10:01 fastvps0 kernel: [4215962.082617] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Aug  3 03:10:01 fastvps0 kernel: [4215962.082938] Process apache2 (pid: 30230, threadinfo ffff8802bf53e000, task ffff88027a4e6740)
Aug  3 03:10:01 fastvps0 kernel: [4215962.083261] Stack:
Aug  3 03:10:01 fastvps0 kernel: [4215962.083421]  2e66c3f300459a40 ffffffff81a25720 0000000000000000 000000000000000f
Aug  3 03:10:01 fastvps0 kernel: [4215962.083596] <0> ffff8802bf53fe58 ffffffff810379bf 0000000000000005 0000000081a25750
Aug  3 03:10:01 fastvps0 kernel: [4215962.083926] <0> ffff8802bf53fe48 0000000000000246 ffff88000730ccd0 ffff88000730ccd0
Aug  3 03:10:01 fastvps0 kernel: [4215962.084409] Call Trace:
Aug  3 03:10:01 fastvps0 kernel: [4215962.084574]  [<ffffffff810379bf>] try_to_wake_up+0x2f/0x220
Aug  3 03:10:01 fastvps0 kernel: [4215962.084740]  [<ffffffff81037bdc>] wake_up_process+0x10/0x12
Aug  3 03:10:01 fastvps0 kernel: [4215962.084907]  [<ffffffff81197204>] freeary+0x1a9/0x1e6
Aug  3 03:10:01 fastvps0 kernel: [4215962.085071]  [<ffffffff8119755f>] T.503+0x8e/0xc4
Aug  3 03:10:01 fastvps0 kernel: [4215962.085238]  [<ffffffff8150aeff>] ? do_page_fault+0x25d/0x28a
Aug  3 03:10:01 fastvps0 kernel: [4215962.085405]  [<ffffffff81197caa>] sys_semctl+0x74/0x81
Aug  3 03:10:01 fastvps0 kernel: [4215962.085574]  [<ffffffff810029eb>] system_call_fastpath+0x16/0x1b
Aug  3 03:10:01 fastvps0 kernel: [4215962.085740] Code: 26 6c 4d 00 5b 41 5c c9 c3 55 48 89 e5 41 56 49 89 fe 41 55 49 89 f5 41 54 49 c7 c4 80 2d 01 00 53 9c 58 fa 49 89 45 00 4c 89 e3 <49> 8b 46 08 8b 40 18 48 03 1c c5 10 fe ac 81 48 89 df e8 e8 6b 
Aug  3 03:10:01 fastvps0 kernel: [4215962.086390] RIP  [<ffffffff810310a7>] task_rq_lock+0x22/0x6b
Aug  3 03:10:01 fastvps0 kernel: [4215962.086563]  RSP <ffff8802bf53fdd8>
Aug  3 03:10:01 fastvps0 kernel: [4215962.086981] ---[ end trace 5c52f3a7773aa044 ]---
Aug  3 03:10:01 fastvps0 kernel: [4215962.086985] BUG: scheduling while atomic: apache2/1643/0x10000001
Aug  3 03:10:01 fastvps0 kernel: [4215962.086988] Modules linked in: tun dm_snapshot scsi_wait_scan
Aug  3 03:10:01 fastvps0 kernel: [4215962.086992] Pid: 1643, comm: apache2 Tainted: G      D     2.6.36-gentoo-r5 #1
Aug  3 03:10:01 fastvps0 kernel: [4215962.086993] Call Trace:
Aug  3 03:10:01 fastvps0 kernel: [4215962.086994]  <#SS>  [<ffffffff81035289>] __schedule_bug+0x57/0x5b
Aug  3 03:10:01 fastvps0 kernel: [4215962.087004]  [<ffffffff81506424>] schedule+0xe6/0x64a
Aug  3 03:10:01 fastvps0 kernel: [4215962.087008]  [<ffffffff8150aef6>] ? do_page_fault+0x254/0x28a
Aug  3 03:10:01 fastvps0 kernel: [4215962.087011]  [<ffffffff81038aa8>] __cond_resched+0x25/0x30
Aug  3 03:10:01 fastvps0 kernel: [4215962.087012]  [<ffffffff8150aef6>] ? do_page_fault+0x254/0x28a
Aug  3 03:10:01 fastvps0 kernel: [4215962.087015]  [<ffffffff81506aa9>] _cond_resched+0x27/0x32
Aug  3 03:10:01 fastvps0 kernel: [4215962.087018]  [<ffffffff81004e45>] is_valid_bugaddr+0x11/0x2a
Aug  3 03:10:01 fastvps0 kernel: [4215962.087021]  [<ffffffff811daf35>] report_bug+0x1c/0xe0
Aug  3 03:10:01 fastvps0 kernel: [4215962.087023]  [<ffffffff81005dd8>] die+0x34/0x5e
Aug  3 03:10:01 fastvps0 kernel: [4215962.087025]  [<ffffffff81508885>] do_trap+0x11c/0x12b
Aug  3 03:10:01 fastvps0 kernel: [4215962.087027]  [<ffffffff81003c75>] do_stack_segment+0x6f/0x8d
Aug  3 03:10:01 fastvps0 kernel: [4215962.087029]  [<ffffffff815082ff>] stack_segment+0x1f/0x30
Aug  3 03:10:01 fastvps0 kernel: [4215962.087031]  [<ffffffff8150aef6>] ? do_page_fault+0x254/0x28a
Aug  3 03:10:01 fastvps0 kernel: [4215962.087032]  <<EOE>>  [<ffffffff8150835f>] ? page_fault+0x1f/0x30
Aug  3 03:10:01 fastvps0 kernel: [4215962.087044] stack segment: 0000 [#2] SMP 
Aug  3 03:10:01 fastvps0 kernel: [4215962.087046] last sysfs file: /sys/devices/pci0000:00/0000:00:1f.3/i2c-0/name
Aug  3 03:10:01 fastvps0 kernel: [4215962.087048] CPU 1 
Aug  3 03:10:01 fastvps0 kernel: [4215962.087048] Modules linked in: tun dm_snapshot scsi_wait_scan
Aug  3 03:10:01 fastvps0 kernel: [4215962.087050] 
Aug  3 03:10:01 fastvps0 kernel: [4215962.087052] Pid: 1643, comm: apache2 Tainted: G      D     2.6.36-gentoo-r5 #1 MSI X58 Pro-E (MS-7522)/MS-7522
Aug  3 03:10:01 fastvps0 kernel: [4215962.087054] RIP: 0010:[<ffffffff8150aef6>]  [<ffffffff8150aef6>] do_page_fault+0x254/0x28a
Aug  3 03:10:01 fastvps0 kernel: [4215962.087057] RSP: 0000:ffff88032a933f08  EFLAGS: 00010246
Aug  3 03:10:01 fastvps0 kernel: [4215962.087058] RAX: 0000000000000000 RBX: 0000000001006788 RCX: 0000000000000001
Aug  3 03:10:01 fastvps0 kernel: [4215962.087059] RDX: 0000000001006788 RSI: ffff88032a933f58 RDI: 0000000000000005
Aug  3 03:10:01 fastvps0 kernel: [4215962.087061] RBP: 2a933f482a933f48 R08: 0000000001006788 R09: 0000000000000005
Aug  3 03:10:01 fastvps0 kernel: [4215962.087062] R10: ffffea0009b2bcf8 R11: ffff88033d306180 R12: ffff88032a933f58
Aug  3 03:10:01 fastvps0 kernel: [4215962.087063] R13: 0000000000000007 R14: ffff8800069ea170 R15: ffff8802e9f7e740
Aug  3 03:10:01 fastvps0 kernel: [4215962.087065] FS:  00007f3e9c5fe710(0000) GS:ffff880001e40000(0000) knlGS:0000000000000000
Aug  3 03:10:01 fastvps0 kernel: [4215962.087067] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug  3 03:10:01 fastvps0 kernel: [4215962.087068] CR2: 0000000001006788 CR3: 00000002bd032000 CR4: 00000000000006e0
Aug  3 03:10:01 fastvps0 kernel: [4215962.087070] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Aug  3 03:10:01 fastvps0 kernel: [4215962.087071] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Aug  3 03:10:01 fastvps0 kernel: [4215962.087073] Process apache2 (pid: 1643, threadinfo ffff88032a932000, task ffff8802e9f7e740)
Aug  3 03:10:01 fastvps0 kernel: [4215962.087074] Stack:
Aug  3 03:10:01 fastvps0 kernel: [4215962.087075]  ffff88033d3061e0 ffff88033d306180 ffff88032a933f48 0000000000000000
Aug  3 03:10:01 fastvps0 kernel: [4215962.087076] <0> 00007f3e9c5fde6c 0000000000000001 00007f3e9c5fde60 0000000000000080
Aug  3 03:10:01 fastvps0 kernel: [4215962.087078] <0> 0000000001006490 ffffffff8150835f 0000000000000080 00007f3e9c5fde60
Aug  3 03:10:01 fastvps0 kernel: [4215962.087080] Call Trace:
Aug  3 03:10:01 fastvps0 kernel: [4215962.087082]  [<ffffffff8150835f>] ? page_fault+0x1f/0x30
Aug  3 03:10:01 fastvps0 kernel: [4215962.087083] Code: 49 ff 87 a8 03 00 00 48 89 da 4c 89 e6 bf 06 00 00 00 eb 12 49 ff 87 a0 03 00 00 48 89 da 4c 89 e6 bf 05 00 00 00 e8 26 8b b1 ff <48> 8b 7d c0 e8 e1 da b4 ff eb 1c 31 c9 4c 89 f6 85 d2 48 8b 7d 
Aug  3 03:10:01 fastvps0 kernel: [4215962.087094] RIP  [<ffffffff8150aef6>] do_page_fault+0x254/0x28a
Aug  3 03:10:01 fastvps0 kernel: [4215962.087096]  RSP <ffff88032a933f08>

В чем может быть проблема? Был ли апач причиной падения или жертвой? Что предпринять, чтобы собрать больше информации?

З.Ы. По поводу железа хостер уверяет, что все исправно.

Апач не может быть проблемой

Апач не может быть причиной по-определению. Проблема в ядре.

(*)

https://lkml.org/lkml/2011/7/19/245
https://bugzilla.kernel.org/show_bug.cgi?id=27142

Искал так - взял три строки после "Call Trace" и очистил от "шелухи".
Получил try_to_wake_up wake_up_process freeary
Это и гуглил.

lspci | grep 00:1fпокажи

lspci | grep 00:1f

покажи плиз ...

gentoo на kvm guest?

Вот вывод 00:1f.0 ISA

Вот вывод

00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller

Система стоит на реальном железе.

Сохранить мир подчас труднее, чем выстоять в самой жестокой схватке...

Не верь хостеру - проблема

Не верь хостеру - проблема скорее всего (90%) аппаратная.

Это так называемый CPU exception, почитать можно тут: http://wiki.osdev.org/Exceptions#General_Protection_Fault

Вы назвали последствие, но не

Вы назвали последствие, но не назвали причину. 90% случаев у хостеров - это тупые проблемы юзверей и 10% это то что они сами СЗБ (пытаются по макимому использовать ресурсы железа накидав туда побольше пользователей).

Хардварный exception из-за

Хардварный exception из-за юзера? оО

Настройки просмотра комментариев

Выберите нужный метод показа комментариев и нажмите "Сохранить установки".