Testing forecast roadmap
From Linux NFS
(Difference between revisions)
Revision as of 21:25, 13 June 2005
Linux NFSv4 Testing Roadmap
As of May 2005, we've completed a detailed itemization of testing work necessary for NFSv4, in a spreadsheet "Test Matrix", identifying items as High, Medium, and Low.
This document seeks to divide the high priority work into achievable chunks that we as a community can strive to attain.
I tried to prioritize for things that look reasonably easy to complete in the near term, that have owners, and/or are most likely to reveal issues that developers want to know about.
Contents |
Milestone 0: Developer Testing Support
These testing tasks seek to put into place tools and practices that will help support development activities (including testing) on an ongoing basis.
Functionality:
I.B.1 [ ] [OSDL] Automate running applicable existing functionality
tests on codebase periodically: cthon, pynfs,
iozone
I.B.3 [X] [OSDL] Automate cross-compile testing for major platforms
on each NFSv4 patchset
I.C.2 [ ] [ ] Test install on Fedora Core of NFSv4 server &
client, krb5, ldap, et al
I.E.9 [ ] [ ] Verify ability to force operations (close files,
change states, unmount)
I.G.5 [ ] [Bull] Test compilation and functionality on IA-64
I.G.7 [ ] [Bull] Test compilation and functionality on PPC-64
I.K.2 [ ] [ ] Check that NFS HOWTO is updated with sufficient
nfsv4 info
Interop:
II.A.1 [ ] [Bull] Kerberos (MIT) - verify basic functionality. Mount
w/ krb5, etc.
Robustness:
III.A.3 [ ] [OSDL] Run NFS server for 2 wks with random configuration
changes
III.C.1 [ ] [Bull] Run LTP NFS fstress in a std config on each release
Performance:
IV.A.1 [ ] [Bull] Time to perform sequence of unique read/write
operations
IV.A.2 [ ] [Bull] Time to perform sequence of cacheable read/write
operations
IV.F.4 [ ] [ ] Measure memory/network/CPU efficiency of client for
fixed workload
IV.G.3 [ ] [Bull] Measure effects of scaling up number of
connections. Measure number of mounts per second
on client and server.
IV.J.2 [ ] [Bull] Measure performance when operating with Kerberos 5
integrity and privacy
Security:
V.A.6 [ ] [ ] Audit the mountd code
V.B.4 [ ] [OSDL] Run sparse as regression test periodically
V.L.1 [ ] [ ] Ensure there is high level design documentation of
NFSv4 security
V.L.2 [ ] [ ] Ensure there is inline documentation for security
related code in kernel
V.L.3 [ ] [ ] Ensure there is inline documentation for GSS API
(libgssapi, librpcsecgss)
Misc:
* [ ] [OSDL] Establish bug tracking process for NFSv4
Milestone 1: Distro Quality Assurance
This milestone strives to attain a "Top Ten" list of testing that is felt will emphasize that NFSv4 is stable enough to be included in all enterprise distros. The focus is robustness, and items that would cause distros the most headaches if they're busted.
I.A.5 [ ] [ ] Protocol implementation interoperability between
Linux server and client
I.C.5 [ ] [ ] Test install on SLES of NFSv4 server & client,
krb5, ldap, et al
I.C.6 [ ] [ ] Test install on RHEL of NFSv4 server & client,
krb5, ldap, et al
I.H.1 [ ] [ ] Verify compatibility with glibc
I.H.9 [ ] [ IBM] Verify compatibility with automounter
I.H.11 [ ] [ ] Verify compatibility with basic system tools
(file utils, core utils, util-linux, mount, sar,
iostat, etc.)
I.K.4 [ ] [ ] Check that Network Admin Guide at tldp.org is
updated for nfsv4
II.E.2 [ ] [ ] Verify features of the Ext3 file system work under
NFSv4
III.A.1 [ ] [ ] Run iozone for 2 wks on basic client/server
operations
III.A.2 [ ] [ ] Run automounter use case for 2 wks on amd, autofs,
and autong
III.A.4 [ ] [ ] Run connectathon locking tests against NFS server
for 2 weeks
III.E.1 [ ] [OSDL] Test short & long term local network failure
(unplugged cable, ifdown eth0, etc.)
III.E.2 [ ] [OSDL] Test short & long duration remote network partition
III.E.3 [ ] [OSDL] Test behavior during crash/reboot of server with
clients holding various states
V.D.1 [ ] [ ] Review Authentication/ACL feature design
V.E.1 [ ] [ ] Ensure a functionality test sufficiently tests
Authentication/ACL
Milestone 2: Drill Down for One Use Case
This milestone strives to push NFSv4 through one complete use case, to give it some thorough exercise and scaling, and to establish a "case study" we can use as an example of NFSv4's value.
I.E.5 [ ] [ ] Verify sufficient monitoring functionality within
Ethereal
I.F.1 [ ] [ ] Client notification to server of locking, write,
read, etc.
I.P.1 [ ] [OSDL] Database functionality on NFS
I.L.1 [ ] [ ] Test compatibility with TCP protocol
IV.A.7 [ ] [ ] NFSv3 vs. v4: Metadata - open/close intensive
workload
IV.A.8 [ ] [ ] NFSv3 vs. v4: Metadata - directory scanning
IV.A.9 [ ] [ ] NFSv3 vs. v4: Metadata - create/delete
IV.A.10 [ ] [ ] NFSv3 vs. v4: Metadata - changing attributes
(chown, chmod) while dir scanning
IV.E.1 [ ] [Bull] Test performance with large numbers of small (<4k)
files
IV.E.2 [ ] [ ] Test performance with a few very large (>1G) files
IV.F.4 [ ] [Bull] Measure memory/network/CPU efficiency of client
for fixed workload
IV.G.1 [ ] [ ] Verify server scalability with clients generating
various basic requests (ACCESS, GETATTR, et al)
IV.G.2 [ ] [ ] Verify server scalability with clients using
compound requests
V.D.6 [ ] [ ] Penetration testing for client callback
implementation
V.E.6 [ ] [ ] Ensure a functionality test sufficiently covers
penetration testing for client callback
implementation
Milestone 3: Enterprise Usability
This milestone focuses on testing that verifies "basic" usability of NFSv4 in data centers. It strives to establish that NFSv4 is able to perform basic functions in enterprise-class deployments.
I.F.2 [ ] [ ] Reboot recovery
I.F.4 [ ] [ ] Open with shares / deny
I.F.5 [ ] [ ] Bumping a sequence ID
I.F.6 [ ] [ ] Network partition recovery
I.F.8 [ ] [ ] Locking (Non-blocking locks)
I.H.3 [ ] [ ] Verify compatibility with Kerberos
I.H.4 [ ] [ ] Verify compatibility with Ipsec
I.H.5 [ ] [ ] Verify compatibility with POSIX ACLs
I.H.6 [ ] [ ] Verify compatibility with NFS ACLs
II.B.1 [ ] [Bull] Interoperability between 32-bit and 64-bit for
client and server: IA32/AIX-PPC, IA32/Linux-PPC
II.B.2 [ ] [Bull] Interoperability between little endian and big
endian
II.C.1 [ ] [Bull] Interoperability for Linux IA-32 client ? Solaris
10 server
II.C.4 [ ] [Bull] Interoperability for Linux IA-32 client ? AIX 5.3
server
III.D.7 [ ] [ ] Test correctness of NFS client when backed by a
large (>100GB) cachefs
IV.E.5 [ ] [ ] NFS "Cluster" scenario with 1000+ clients and
several servers
V.A.7 [ ] [ ] Audit the RPC authentication code (gssd, authsys,
etc.)
V.D.2 [ ] [ ] Review each security flavor feature design:
Krb5, Spkm
V.E.2 [ ] [ ] Ensure a functionality test sufficiently covers
each security flavor
Milestone 4: Enterprise Readiness
This milestone builds on the previous one by ensuring that NFSv4 will behave well and perform robustly under typical enterprise-class loads.
III.B.2 [ ] [ ] Test stability of client in out of memory
situation
III.B.3 [ ] [ ] Test stability of client in out of disk space on
server situation
III.B.4 [ ] [ ] Test stability of client in out of inode situation
III.B.5 [ ] [ ] Test stability of client in out of swap space
situation
III.B.6 [ ] [ ] Test stability of server in out of pid situation
III.B.7 [ ] [ ] Test stability of server in out of memory
situation
III.B.8 [ ] [ ] Test stability of server in out of disk space
situation
III.B.9 [ ] [ ] Test stability of server in out of inode situation
III.B.10 [ ] [ ] Test stability of server in out of swap space
situation
III.D.1 [ ] [ ] Find maximum number of connections to Linux IA-32
server
III.D.4 [ ] [ ] Find maximum number of mounted file systems on
client
III.E.4 [ ] [ ] Test multiple clients using, locking, etc. same
files
III.E.5 [ ] [ ] Test behavior of server with failed storage device
III.E.6 [ ] [ ] Test behavior during crash of client with open
delegations and locks
III.E.7 [ ] [ ] Test recovery from denied permission
III.E.8 [ ] [ ] Test recovery from JUKEBOX/DELAY
III.E.9 [ ] [ ] Test recovery from ESTALE
III.E.10 [ ] [ ] Test server callback mechanism (c.f. III.E.1,
III.E.2)
III.G.2 [ ] [ ] Test concurrent access tests for races in
automounter
IV.G.4 [ ] [ ] Measure effects of increasing number of files
IV.G.5 [ ] [ ] Measure effects of increasing file size
(with/without cache)
IV.G.6 [ ] [ ] Measure effects when increasing size of
on-the-wire NFS read or write operations
V.J.1 [ ] [ ] Identify security issues assuming attack from
client-side
