[nfsv4] Performance drop in iozone for reading large files

[Bryce Harrington]

On Fri, Aug 25, 2006 at 05:23:37PM -0400, Trond Myklebust wrote:
> On Fri, 2006-08-25 at 14:01 -0700, Bryce Harrington wrote:
> > There is a performance issue in READ operations we've noticed in iozone
> > runs for a while now.  If you look at this plot, of NFSv4 with 1G mem:
> > 
> >    http://crucible.osdl.org/runs/1604/test_output/iozone.sys.log.png
> > 
> > You can see that we're getting fairly steady performance up until the
> > 1048576kb case, where read performance drops to about a third.
> > 
> > Does anyone have an idea why the dropoff is occuring?  Is the memory
> > size expected to affect NFS performance this way?  Is anyone else
> > experiencing similar effects?
> 
> Is the readahead algorithm working correctly? Could you please try
> applying the following patch, and then check your syslog for warnings?

I just want to follow up on this one and make sure I understand it
properly.

Dean says this is due to hitting the memory limit of server-side
caching, which seems to explain it.  Our server has 1G memory, and when
shrinking it to lower ram, the issue does occur earlier.

  From: Dean Hildebrand <dhildebz at eecs.umich.edu>

  So your server only has 1 GB of RAM in the original case?  If your
  server only has 1 GB of RAM, then reading a file greater than 1 GB will
  definitely reduce performance as the server is now going to disk.  Since
  the server operating system also requires memory, it would be normal to
  see a performance drop off much earlier, maybe around 600-800 MB.

Does this explain the situation sufficiently, or does it deserve
additional investigation?

Thanks,
Bryce