[nfsv4] [PATCH 0/7] Permit filesystem local caching and NFS superblock sharing [try #13]

Fri Sep 1 19:32:07 EDT 2006

Here are some test results for David's patch...

----- Forwarded message from Bryce Harrington <bryce at osdl.org> -----

Date: Thu, 31 Aug 2006 19:40:24 -0700
From: Bryce Harrington <bryce at osdl.org>
To: David Howells <dhowells at redhat.com>
Cc: jasonn at osdl.org
Subject: Re: [nfsv4] [PATCH 0/7] Permit filesystem local caching and NFS superblock sharing [try #13]

On Thu, Aug 31, 2006 at 11:04:03AM +0100, David Howells wrote:
> Bryce Harrington <bryce at osdl.org> wrote:
> 
> > We can provide some testing of this patchset in the nfsv4 testing
> > environment, if it might help.  This will give you some data to show how it
> > compares, or at least provide some proof it doesn't break things.
> > 
> > Could you tell us which version of the kernel we should apply this patch
> > against?
> 
> Trond's NFS GIT tree.

Hi David,

Here is a first test run against this patch.  It applied fine against
Trond's tree, and here are the raw logs:

   http://crucible.osdl.org/runs/1741/test_output/

Some information about the machines themselves is available here:

   Server:  http://crucible.osdl.org/runs/1741/sysinfo/nfs06.1/
   Client:  http://crucible.osdl.org/runs/1741/sysinfo/nfs06.1/

Basically, they're ordinary x86 32-bit, 1-CPU P4 systems with 1G ram
and 1G ethernet connected through a 1Gbit HP switch.

For reference, here is the test data from trond's code by itself:

   http://crucible.osdl.org/runs/1690/test_output/

I find newpynfs to be the best tool for finding regressions, and here's
what we're seeing:

1690 (Trond's git tree)
-----------------------
sys:   164 Skipped, 47 Failed, 2 Warned, 365 Passed
krb5:  158 Skipped, 45 Failed, 2 Warned, 373 Passed
krb5i:   5 Skipped, 61 Failed, 17 Warned, 495 Passed
krb5p:   6 Skipped, 63 Failed, 16 Warned, 493 Passed

1741 (Trond's git tree + DHowell's fs_cache patch)
--------------------------------------------------
sys:    13 Skipped, 57 Failed, 17 Warned, 491 Passed
krb5:  158 Skipped, 46 Failed, 2 Warned, 372 Passed
krb5i:   5 Skipped, 62 Failed, 17 Warned, 494 Passed
krb5p: 127 Skipped, 406 Failed, 1 Warned, 44 Passed

These two runs are not perfectly apples-to-apples comparison since they
ran on different actual machines.  However they're close enough.  After
your runs are done, we can rerun trond's patch on the same machines to
make double-sure the results are comparable.

The difference in sys cases is mildly curious but not surprising.  In
any case, those numbers are 100% typical so they indicate goodness.

For the krb5 case, the one new error is this one:

CIDCF3   st_setclientidconfirm.testAllCases                       : FAILURE
           operation OP_SETCLIENTID_CONFIRM should return
           NFS4_OK, instead got NFS4ERR_CLID_INUSE

For the krb5i case, the one new error is:

OPEN10   st_open.testZeroLenName                                  : FAILURE
           operation OP_SETCLIENTID_CONFIRM should return
           NFS4_OK, instead got NFS4ERR_CLID_INUSE

Personally, neither of these look that interesting; I see variances in
these test cases all the time, and I would guess they don't indicate
anything important.

The krb5p errors, on the other hand, are quite interesting.  I notice
the server Oopsed, and I'm guessing it was during this run.  Take a look
at the console log:

   http://crucible.osdl.org/runs/1741/nfs06.console.log

cthon, ltp, and iozone all run after newpynfs, so since the server
crashed, data from those tests isn't valid.  

Hope this helps.  I'm going to re-run the test and see if I can get the
server to crash again.  Let me know if you have any thoughts on this
issue, or additional patches you'd like us to try out.

Bryce

----- End forwarded message -----

Last night we re-ran the test, and it did Oops again, in a similar
manner, but this time when running newpynfs with AUTHSYS: 

   http://crucible.osdl.org/runs/1750/sysinfo/nfs06.console

BUG: unable to handle kernel paging request at virtual address 6b6b6b73
printing eip:
c040f26d
*pde = 00000000
Oops: 0000 [#1]
PREEMPT SMP 
Modules linked in:
CPU:    0
EIP:    0060:[<c040f26d>]    Not tainted VLI
EFLAGS: 00010246   (2.6.18-rc5-gd4a4640-nfs-client-stable #1) 
EIP is at pmap_getport_done+0x17/0xae
eax: 6b6b6b6b   ebx: dfc56370   ecx: 00000000   edx: dfc56370
esi: dfc562b0   edi: 00000000   ebp: f7aa2f04   esp: f7aa2ef8
ds: 007b   es: 007b   ss: 0068
Process rpciod/0 (pid: 7635, ti=f7aa2000 task=f7421550 task.ti=f7aa2000)
Stack: dfc56370 dfc563d8 00000000 f7aa2f18 c040a23f dfc56370 dfc4aa34
dfc56370 
f7aa2f30 c040a2f7 dfc56370 dfc563e0 dfc563e4 f6b07d24 f7aa2f3c c040a440 
dfc56370 f7aa2f60 c012b398 dfc56370 dfc56370 c040a435 00000213 f6b07d24 
Call Trace:
[<c01037da>] show_stack_log_lvl+0x8a/0x92
[<c010393b>] show_registers+0x11d/0x186
[<c0103b27>] die+0x10c/0x1c2
[<c0113f31>] do_page_fault+0x3e0/0x4bc
[<c01034ad>] error_code+0x39/0x40
[<c040a23f>] rpc_exit_task+0x1e/0x59
[<c040a2f7>] __rpc_execute+0x7d/0x19c
[<c040a440>] rpc_async_schedule+0xb/0xd
[<c012b398>] run_workqueue+0x85/0xc7
[<c012b4d7>] worker_thread+0xfd/0x131
[<c012e3d2>] kthread+0x84/0xad
[<c0100f19>] kernel_thread_helper+0x5/0xb
Code: 05 c0 00 00 00 50 e8 d8 ae ff ff 5a 8d 65 f4 5b 5e 5f 5d c3 55 89
e5 57 56 8b 45 0c 8b 55 08 53 8b 70 10 8b 7a 18 8b 46 10 85 ff <8b> 58
08 79 0d 8b 03 6a 00 53 ff 50 14 89 7e 18 eb 1b 8b 55 0c 
EIP: [<c040f26d>] pmap_getport_done+0x17/0xae SS:ESP 0068:f7aa2ef8

Bryce