kernel client BUGs during shutdown with recent kernels
Bryce Harrington
bryce at osdl.org
Wed Jul 19 16:23:58 EDT 2006
Here is the /var/log/messages file, up to the ending of the test, with
the slab corruption data:
http://crucible.osdl.org/runs/1052/sysinfo/nfs03.messages
Here is the dmesg for the machine:
http://crucible.osdl.org/runs/1052/sysinfo/nfs03.2/INFO/dmesg
I'm not seeing a message about what file was in use; presumably it was
an iozone data file.
Bryce
On Wed, Jul 19, 2006 at 01:13:01PM -0700, Bryce Harrington wrote:
> Hi Trond,
>
> With the recent kernel patch we've started noticing frequent (but
> intermittent) issues during shutdown. We are seeing two different BUG
> messages, although possibly they are just two symptoms of the same bug.
> The issue was also present on both the client and server branches in
> GIT, although we did not see the crash in the latest nfs-server-stable
> checkin yesterday. Are you aware of this one already? I can provide
> additional details if needed.
>
> Thanks,
> Bryce
>
>
>
> ----- Forwarded message from Jason Neighbors <jasonn at osdl.org> -----
>
> Date: Fri, 14 Jul 2006 18:49:03 -0700
> From: Jason Neighbors <jasonn at osdl.org>
> To: bryce at osdl.org
> Subject: kernel crashes: 2.6.18-rc1-g3efad9f-nfs-client-stable
>
> Seems to happen on reboots frequently. So far just on our client systems, but they do reboot quite a bit.
>
> ------------[ cut here ]------------
> kernel BUG at fs/dcache.c:611!
> invalid opcode: 0000 [#1]
> PREEMPT SMP
> Modules linked in:
> CPU: 0
> EIP: 0060:[<c0170042>] Not tainted VLI
> EFLAGS: 00010246 (2.6.18-rc1-g3efad9f-nfs-client-stable #1)
> EIP is at shrink_dcache_for_umount_subtree+0x150/0x20b
> eax: 0000005a ebx: f43bc2a0 ecx: c043b2a4 edx: 00000001
> esi: f3f5b448 edi: f765ce40 ebp: f6469d1c esp: f6469cf4
> ds: 007b es: 007b ss: 0068
> Process 180-boot_kernel (pid: 9380, ti=f6469000 task=f4bc6550 task.ti=f6469000)
> Stack: c03d7800 f43bc2a0 00000072 f43bc310 00000001 c0419e2c f6ee9f24 f6ee9d84
> c047e3a0 c04cc324 f6469d2c c017012d f765ce40 f6ee9d84 f6469d40 c0160f77
> f6ee9d84 00000011 c047e400 f6469d50 c01617c8 f6ee9d84 f6ee9d84 f6469d60
> Call Trace:
> [<c010378e>] show_stack_log_lvl+0x8a/0x92
> [<c01038ef>] show_registers+0x11d/0x186
> [<c0103adb>] die+0x10c/0x1da
> [<c0103c17>] do_trap+0x6e/0x8a
> [<c0103e6e>] do_invalid_op+0x90/0x97
> [<c0103461>] error_code+0x39/0x40
> [<c017012d>] shrink_dcache_for_umount+0x30/0x54
> [<c0160f77>] generic_shutdown_super+0x1b/0xfd
> [<c01617c8>] kill_anon_super+0x10/0x3a
> [<c016180d>] kill_litter_super+0] mntput_no_expire+0x48/0x6a
> [<c01795f7>] simple_release_fs+0x42/0x4b
> [<c039af51>] rpc_put_mount+0x12/0x14
> [<c038dffd>] rpc_destroy_client+0xca/0xda
> [<c038df2b>] rpc_shutdown_client+0xf2/0xfa
> [<c01bd5ce>] nfs_free_client+0x99/0xe5
> [<c01bd686>] nfs_put_client+0x6c/0x72
> [<c01be3cf>] nfs_free_server+0x8d/0xc1
> [<c01c4c9b>] nfs_kill_super+0x19/0x1e
> [<c0160ecd>] deactiva3fc7>] mntput_no_expire+0x48/0x6a
> [<c011efb8>] do_exit+0x22f/0x404
> [<c011f241>] sys_exit_group+0x0/0x12
> [<c0127c2f>] get_signal_to_deliver+0x281/0x2aa
> [<c0102773>] do_signal+0x57/0x104
> [<c010284d>] do_notify_resume+0x2d/0x3c
> [<c0102a26>] work_notifysig+0x13/0x19
> Code: d2 05 a0 01 00 00 50 8b 43 60 8b 40 1c ff 30 8b 03 50 ff 73 34 8b 43 1c 85 c0 74 03 8b 50 20 52 53 68 fe cb fa ff <0
> EIP: [<c0170042>] shrink_dcache_for_umount_subtree+0x150/0x20b SS:ESP 0068:f6469cf4
> <1>Fixing recursive fault but reboot is needed!
>
>
> ------------[ cut here ]------------
> kernel BUG at kernel/timer.c:396!
> invalid opcode: 0000 [#1]
> PREEMPT SMP
> Modules linked in:
> CPU: 0
> EIP: 0060:[<c01247c3>] Not tainted VLI
> EFLAGS: 00010883 (2.6.18-rc1-g3efad9f-nfs-client-stable #1)
> EIP is at cascade+0x40/0x65
> eax: f7287418 ebx: c0478678 ecx: c0504f94 edx: c0552e00
> esi: c0504f94 edi: 00000026 ebp: c0504fa8 esp: c0504f94
> ds: 007b es: 007b ss: 0068
> Process swapper (pid: 0, ti=c0504000 task=c0438820 task.ti=c04cd000)
> Stack: f7287418 c19abad8 00000000 c0552e00 c04fc380 c0504fd8 c0125546 c0552e00
> c055361c 00000026 c02f9025 c0504fc0 c0504fc0 00000000 c04c2b08 00000001
> c04fc380 c0504ff8 c0121317 c04c2b08 00000000 0000000a c04cdf6c c04cd000
> Call Trace:
> [<c010378e>] show_stack_log_lvl+0x8a/0x92
> [<c01038ef>] show_registers+0x11d/0x186
> [<c0103adb>] die+0x10c/0x1da
> [<c0103c17>] do_trap+0x6e/0x8a
> [<c0103e6e>] do_invalid_op+0x90/0x97
> [<c0103461>] error_code+0x39/0x40
> [<c0125546>] run_timer_softirq+0x64/0x180
> [<c0121317>] __do_softirq+0x5d/0xc6
> [<c0104c76>] do_softirq+0x5b/0xaa
> =======================
> [<c01213b9>] irq_exit+0x39/0x46
> [<c0110221>] smp_apic_timer_interrupt+0x5a/0x5f
> [<c01033af>] apic_timer_interrupt+0x1f/0x24
> [<c0100c3f>] cpu_idle+0xae/0xdb
> [<c01002d2>] _stext+0x3a/0x3c
> [<c04d27f4>] start_kernel+0x184/0x186
> [<c0100210>] 0xc0100210
> Code: 00 00 03 45 0c 8b 10 89 55 ec 89 4a 04 8b 50 04 89 0a 89 55 f0 89 00 89 40 04 8b 45 ec 39 c8 8b 18 74 23 8b 55 08 39
> EIP: [<c01247c3>] cascade+0x40/0x65 SS:ESP 0068:c0504f94
> <0>Kernel panic - not syncing: Fatal exception in interrupt
>
>
> --
> Jason Neighbors
> x1939
>
> ----- End forwarded message -----
>
> The bugs have occurred on the following test runs:
>
>
> dcache.c:611
> 1023: 07/12 20:00-21:00, nfs13 linux-2.6.18-rc1-CITI_NFS4_ALL-1
> 1023: 07/12 21:00-22:00, nfs13 ""
> 1026: 07/13 01:00-02:00, nfs03 linux-2.6.18-rc1-gae9dce8-nfs-server-stable
> 1039: 07/13 20:00-21:00, nfs13 linux-2.6.18-rc1-g3efad9f-nfs-client-stable
> 1052: 07/14 22:00-23:00, nfs03 ""
> 1052: 07/15 00:00-01:00, nfs03 ""
>
> timer.c:396
> 1027: 07/13 01:00-02:00, nfs13 linux-2.6.18-rc1-gae9dce8-nfs-server-stable
> 1039: 07/13 22:00-23:00, nfs13 linux-2.6.18-rc1-g3efad9f-nfs-client-stable
> 1052: 07/15 01:00-02:00, nfs03 ""
>
> Bryce
> _______________________________________________
> NFSv4 mailing list
> NFSv4 at linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4
More information about the NFSv4
mailing list