[pnfs] client bug with boundary

[Dean Hildebrand]

I found a bug which I think we kind of knew about, but didn't realize 
that it is a really big problem in the current code.  Thankfully, I 
think it has an easy fix.

When writing data (but a similar problem will probably occur on read 
too), the pages are all put into the page cache before LAYOUTGET is 
initiated.  This means that the 'boundary' value in the layout driver == 
0, which then means that the pNFS client does not coalesce data into 
chunks that don't cross stripe boundaries.  The coalesced chunks are 
never bigger than the wsize, which works out for us now since usually 
our stripe size is the same as the wsize, but the wsize'd chunks will 
cross stripe boundaries.  Not sure if anyone else has noticed this.

I noticed the problem when my application tried to do the following on 
the file system (2MB stripe size and wsize)

WRITE 1000000 at 7864320

This should be divided into 2 write requests to not exceed the 8388608 
byte stripe boundary: one of size 524288 and one of size 475712.  
Unfortunately, since boundary==0 and stripe_size==2MB, it is sent to the 
server as one write request for the entire size of 1000000.  For file 
layout with gpfs, this is just annoying, for netapp with distinct data 
servers, this will cause corruption.

Andy and I talked about this quickly and believe the fix is to issue the 
layoutget as soon as the pNFS client knows a I/O request is about to 
happen.  This way, all the information will be available for processing 
as the I/O request gets processed by the pNFS client.  The best place 
seems to be pnfs_file_read 
<ident?v=pnfs-2.6.17;i=pnfs_file_read>/write.  Issuing at open time 
solves the problem for some layouts too.  I think there may be some 
downsides for moving the layoutget to pnfs_file_read 
<ident?v=pnfs-2.6.17;i=pnfs_file_read>/write, such as the fact that the 
offset and extent will not match the actual request issued, but maybe we 
can discuss them tomorrow at the meeting.

-- 
Dean Hildebrand
Ph.D. Candidate
University of Michigan