Matrix performance section

From Linux NFS

(Difference between revisions)
Jump to: navigation, search
(→Evaluation in various load scenarios)
m
 
(21 intermediate revisions not shown)
Line 11: Line 11:
|Time to perform sequence of unique read/write operations
|Time to perform sequence of unique read/write operations
|Iozone
|Iozone
-
|'''Open'''
+
|'''In progress'''
|'''Bull'''
|'''Bull'''
|Done by Bull in 2004
|Done by Bull in 2004
Line 18: Line 18:
|Time to perform sequence of cacheable read/write operations
|Time to perform sequence of cacheable read/write operations
|Iozone
|Iozone
-
|'''New'''
+
|'''Open'''
-
|
+
|'''Bull'''
|
|
|-
|-
Line 25: Line 25:
|Random reads/writes/opens from many clients to one server
|Random reads/writes/opens from many clients to one server
|Iozone
|Iozone
-
|'''Open'''
+
|'''In progress'''
|'''Bull'''
|'''Bull'''
|Done by Bull in 2004
|Done by Bull in 2004
Line 47: Line 47:
|Iozone
|Iozone
|'''New'''
|'''New'''
-
|'''Bull'''
+
|''''''
|
|
|-
|-
Line 54: Line 54:
|Iozone
|Iozone
|'''New'''
|'''New'''
-
|Bull
+
|
|
|
|-
|-
Line 60: Line 60:
|Metadata - directory scanning
|Metadata - directory scanning
|Iozone
|Iozone
-
|'''Open'''
+
|'''Done'''
|Bull
|Bull
-
|
+
|Directory scanning over NFSv4 is analysed [http://nfsv4.bullopensource.org/tools/tests/page19.php here]. Time to stat a directory is O(n²) where n is the number of files in the directory.
|-
|-
|IV.A.9
|IV.A.9
Line 68: Line 68:
|Iozone
|Iozone
|'''New'''
|'''New'''
-
|Bull
+
|
|
|
|-
|-
Line 75: Line 75:
|IOZone
|IOZone
|'''New'''
|'''New'''
-
|Bull
+
|
|
|
|-
|-
|IV.A.11
|IV.A.11
|How many locks can be made and released over time
|How many locks can be made and released over time
-
|FFsB
+
|LTP
-
|'''New'''
+
|'''Open'''
|Bull
|Bull
|
|
Line 88: Line 88:
|Comparison of speeds attainable for different NIC cards
|Comparison of speeds attainable for different NIC cards
|
|
-
|
+
|'''New'''
|
|
|
|
Line 124: Line 124:
|Analyze whether file system choice affects performance
|Analyze whether file system choice affects performance
|Iozone
|Iozone
-
|'''Done'''
+
|'''DONE'''
|'''Bull'''
|'''Bull'''
|[http://nfsv4.bullopensource.org/tools/tests/page2.php NFSv4 performances] do not depend on the local file-system used
|[http://nfsv4.bullopensource.org/tools/tests/page2.php NFSv4 performances] do not depend on the local file-system used
Line 131: Line 131:
|Test performance with Ext2 on server with metadata /acls
|Test performance with Ext2 on server with metadata /acls
|IOZone/FFsB
|IOZone/FFsB
-
|
+
|'''New'''
|
|
|
|
Line 225: Line 225:
|'''In progress'''
|'''In progress'''
|Bull
|Bull
-
|While most NFS functionnalities are not affected by the number of files (1 000 000 of empty files), some shell functions are not responding : ls (answer take more than 15 hours). This question will be splited in sub topics soon.
+
|While most NFS functionnalities are not affected by the number of files (2 000 000 of empty files).
 +
|-
 +
|*IV.E.1
 +
  *Sub topic 1
 +
|Test performance with large numbers of small (<4k) files - stat function ; empty files  -
 +
|addhoc tool
 +
|'''Done'''
 +
|Bull
 +
|Stat answer is O(n²). More details [http://nfsv4.bullopensource.org/tools/tests/page19.php here]
 +
|-
 +
|IV.E.1
 +
*Sub topic 2
 +
|Test performance with large numbers of small (<4k) files - Open function ; empty files -
 +
|addhoc tool
 +
|'''Near done'''
 +
|Bull
 +
|Open function is O(n). There is a bottle neck for n=1620000. More details [http://nfsv4.bullopensource.org/tools/tests/page20.php here]. Comparisons with [http://nfsv4.bullopensource.org/tools/tests/page21.php local file system] and [http://nfsv4.bullopensource.org/tools/tests/page22.php NFSv3].
|-
|-
|IV.E.2
|IV.E.2
Line 238: Line 254:
|Mail/user dir
|Mail/user dir
|'''New'''
|'''New'''
-
|Bull
+
|
|
|
|-
|-
Line 249: Line 265:
|-
|-
|IV.E.5
|IV.E.5
-
|NFS "Cluster" scenario with 1000+ clients and several servers
+
|NFS "Cluster" scenario with 1000 clients and several servers
|Film industry, HPC or visualization workload
|Film industry, HPC or visualization workload
|'''New'''
|'''New'''
Line 256: Line 272:
|-
|-
|IV.E.6
|IV.E.6
-
|NFS front end with cluster backend; 100+ clients
+
|NFS front end with cluster backend; 100 clients
|
|
|'''New'''
|'''New'''
Line 263: Line 279:
|-
|-
|IV.E.7
|IV.E.7
-
|Pure cluster; 100+ clients
+
|Pure cluster; 100 clients
|
|
|'''New'''
|'''New'''
Line 340: Line 356:
|Measure effects of scaling up number of connections
|Measure effects of scaling up number of connections
|IOZone
|IOZone
-
|'''New'''
+
|'''Open'''
|Bull
|Bull
|SMP - Measure number of mounts per second on client and server
|SMP - Measure number of mounts per second on client and server
Line 363: Line 379:
*Measure for stable latency behavior
*Measure for stable latency behavior
|Iozone
|Iozone
-
|'''New'''
+
|'''Open'''
-
|
+
|Bull
|
|
|-
|-
|IV.G.7
|IV.G.7
-
|Measure performance when scaling CPU count per node on SMP&NUMA
+
|Measure performance when scaling CPU count per node on SMP
-
|Iozone
+
-
|'''New'''
+
-
|
+
-
|
+
-
|-
+
-
|IV.G.8
+
-
|Measure performance when scaling memory per node on SMP&NUMA
+
-
|Iozone
+
-
|'''New'''
+
-
|
+
-
|
+
-
|-
+
-
|IV.G.9
+
-
|Measure performance when scaling NIC count per node
+
-
|Iozone
+
-
|'''New'''
+
-
|
+
-
|
+
-
|-
+
-
|IV.G.10
+
-
|Measure performance when scaling disk count per node
+
-
|Iozone
+
-
|'''New'''
+
-
|
+
-
|
+
-
|}
+
-
 
+
-
+
-
 
+
-
== Identify best practices for performance tuning ==
+
-
 
+
-
{|border="1" width="100%" cellpadding="1" cellspacing="0" style="font-size: 85%; border: gray solid 1px; border-collapse: collapse; text-align: center; width: 100%
+
-
!style="background: #ececec;"|'''ID
+
-
!style="background: #ececec;"|'''test'''
+
-
!style="background: #ececec;"|'''tool test'''
+
-
!style="background: #ececec;"|'''status'''
+
-
!style="background: #ececec;"|'''owner'''
+
-
!style="background: #ececec;"|'''notes'''
+
-
|-
+
-
|IV.H
+
-
|Identify best practices for performance tuning:
+
-
*--- Auto-tuning for max performance during installation?
+
-
*Tuning NFS transfer size
+
-
*Details will need to be captured as we complete other testing
+
-
|
+
-
|'''New'''
+
-
|Bull
+
-
|Not clear. There is few tunables points on NFSv4.
+
-
|}
+
-
 
+
-
== Performance effects of security features ==
+
-
 
+
-
{|border="1" width="100%" cellpadding="1" cellspacing="0" style="font-size: 85%; border: gray solid 1px; border-collapse: collapse; text-align: center; width: 100%
+
-
!style="background: #ececec;"|'''ID
+
-
!style="background: #ececec;"|'''test'''
+
-
!style="background: #ececec;"|'''tool test'''
+
-
!style="background: #ececec;"|'''status'''
+
-
!style="background: #ececec;"|'''owner'''
+
-
!style="background: #ececec;"|'''notes'''
+
-
|-
+
-
|IV.J.1
+
-
|Measure performance when operating with IPSec integrity and privacy
+
-
|IOzone
+
-
|'''New'''
+
-
|
+
-
|
+
-
|-
+
-
|IV.J.2
+
-
|Measure performance when operating with Kerberos 5 integrity and privacy
+
-
|IOzone
+
-
|'''New'''
+
-
|
+
-
|
+
-
|-
+
-
|IV.J.3
+
-
|Measure performance when operating with SPKM integrity and privacy
+
-
|IOzone
+
-
|'''New'''
+
-
|
+
-
|
+
-
|-
+
-
|IV.J.4
+
-
|Measure performance when operating with Lipkey integrity and privacy
+
-
|IOzone
+
-
|'''New'''
+
-
|
+
-
|
+
-
|}
+
-
 
+
-
== Performances over Internet==
+
-
 
+
-
{|border="1" width="100%" cellpadding="1" cellspacing="0" style="font-size: 85%; border: gray solid 1px; border-collapse: collapse; text-align: center; width: 100%
+
-
!style="background: #ececec;"|'''ID
+
-
!style="background: #ececec;"|'''test'''
+
-
!style="background: #ececec;"|'''tool test'''
+
-
!style="background: #ececec;"|'''status'''
+
-
!style="background: #ececec;"|'''owner'''
+
-
!style="background: #ececec;"|'''notes'''
+
-
|-
+
-
|IV.K.1
+
-
|Measure performances for various fixed workoads over WAN
+
-
|Iozone - FFsB
+
-
|'''Open'''
+
-
|Bull - CITI
+
-
|Tests between CITI and Bull
+
-
|-
+
-
|IV.K.2
+
-
|Measure performances for various fixed workoads with Kerberos enabled over WAN
+
-
|Iozone - FFsB
+
-
|'''Open'''
+
-
|Bull - CITI
+
-
|Tests between CITI and Bull
+
-
|-
+
-
|IV.K.3
+
-
|Measure performances for metadata manipulation over WAN (directory lookup, attributes manipulations ...)
+
-
|FFsB
+
-
|'''Open'''
+
-
|Bull - CITI
+
-
|Tests between CITI and Bull
+
-
|}
+

Latest revision as of 13:30, 14 July 2007

Contents

Comparison of NFSv4 vs. NFSv3 for common use cases

ID test tool test status owner notes
IV.A.1 Time to perform sequence of unique read/write operations Iozone In progress Bull Done by Bull in 2004
IV.A.2 Time to perform sequence of cacheable read/write operations Iozone Open Bull
IV.A.3 Random reads/writes/opens from many clients to one server Iozone In progress Bull Done by Bull in 2004
IV.A.4 Industry standard loads SpecSFS, Specweb99 New Tools does not exist.
IV.A.5 Time to read file from beginning to end and then rewrite it IOzone In progress Bull Part of IOZone standard tests
IV.A.6 Time for appending info to a log file sporadically over time Iozone New '
IV.A.7 Metadata - open/close intensive workload Iozone New
IV.A.8 Metadata - directory scanning Iozone Done Bull Directory scanning over NFSv4 is analysed here. Time to stat a directory is O(n²) where n is the number of files in the directory.
IV.A.9 Metadata - create/delete Iozone New
IV.A.10 Metadata - changing attributes (chown, chmod) while dir scanning IOZone New
IV.A.11 How many locks can be made and released over time LTP Open Bull
IV.A.12 Comparison of speeds attainable for different NIC cards New

NFSv4 on TCP vs. RDMA

ID test tool test status owner notes
IV.B Compare latency, throughput, etc. of NFSv4 on TCP vs. RDMA New Only prototypes exist currently; possibly will be more fully implemented by end of 2005

Test performance on different local filesystems

ID test tool test status owner notes
IV.C.1 Analyze whether file system choice affects performance Iozone DONE Bull NFSv4 performances do not depend on the local file-system used
IV.C.2 Test performance with Ext2 on server with metadata /acls IOZone/FFsB New
IV.C.3 Test performance with ext3 on server with metadata / acls IOZone/FFsB New
IV.C.4 Test performance with Reiser3 on server with metadata / acls IOZone/FFsB New
IV.C.5 Test performance with xfs on server with metadata / acls IOZone/FFsB New
IV.C.6 Test performance with jfs on server with metadata / acls IOZone/FFsB New
IV.C.7 Test performance with Reiser4 on server with metadata /acls IOZone/FFsB New

Test perfomance on different cluster filesystems

ID test tool test status owner notes
IV.D.1 Test performance when using GFS cluster file system New
IV.D.2 Test performance when using Luster cluster file system New
IV.D.3 Test performance when using GPFS cluster file system New
IV.D.4 Test performance when using Polyserve cluster file system New

Evaluation in various load scenarios

ID test tool test status owner notes
IV.E.1 Test performance with large numbers of small (<4k) files addhoc tool In progress Bull While most NFS functionnalities are not affected by the number of files (2 000 000 of empty files).
*IV.E.1 
*Sub topic 1
Test performance with large numbers of small (<4k) files - stat function ; empty files - addhoc tool Done Bull Stat answer is O(n²). More details here
IV.E.1 
*Sub topic 2
Test performance with large numbers of small (<4k) files - Open function ; empty files - addhoc tool Near done Bull Open function is O(n). There is a bottle neck for n=1620000. More details here. Comparisons with local file system and NFSv3.
IV.E.2 Test performance with a few very large (>1G) files IOzone Open Bull Goals need clarifications: are we manipulating files (acl/metadatas/moving...) or reading/writing files?
IV.E.3 4-16 clients generating high load on 1 server in lab environment Mail/user dir New
IV.E.4 2000-5000 clients on 5-10 servers in production environment Clusters New NetApps
IV.E.5 NFS "Cluster" scenario with 1000 clients and several servers Film industry, HPC or visualization workload New
IV.E.6 NFS front end with cluster backend; 100 clients New
IV.E.7 Pure cluster; 100 clients New

Evaluation in stress scenarios

ID test tool test status owner notes
IV.F.1 Measure performance of server when in limited resource situations :
  • Low memory / heavy swap space usage
  • High inode count situations
New
IV.F.2 Measure performance of client when in limited resource situations :
  • Low memory / heavy swap space usage
  • Low/saturated network bandwidth
New
IV.F.3 Graceful failure mode New See Chuck for more info
IV.F.4 Measure memory/network/CPU efficiency of client for fixed workload IOzone - FFsB In progress Bull

Scalability (performance)

ID test tool test status owner notes
IV.G. Verify server scalability with clients generating various basic requests (ACCESS, GETATTR, et al) Iozone New
IV.G.2 Verify server scalability with clients using compound requests Iozone New
IV.G.3 Measure effects of scaling up number of connections IOZone Open Bull SMP - Measure number of mounts per second on client and server
IV.G.4 Measure effects of increasing number of files Addhoc tool Open Bull
IV.G.5 Measure effects of increasing file size (with/without cache) IOzone Open Bull
IV.G.6 Measure effects when increasing size of on-the-wire NFS read or write operations :
  • Mount options from the client side; find optimal settings
  • Measure for stable latency behavior
Iozone Open Bull
IV.G.7 Measure performance when scaling CPU count per node on SMP
Retrieved from "http://linux-nfs.org/wiki/index.php/Matrix_performance_section"
Personal tools