[pnfs] page sync model

[Benny Halevy]

On Apr. 03, 2008, 18:26 +0300, Benny Halevy <bhalevy at panasas.com> wrote:
> On Apr. 03, 2008, 17:11 +0300, Trond Myklebust <trond.myklebust at fys.uio.no> wrote:
>> On Thu, 2008-04-03 at 16:42 +0300, Benny Halevy wrote:
>>> On Apr. 03, 2008, 15:03 +0300, Benny Halevy <bhalevy.lists at gmail.com> wrote:
>>>> Image updated and saved as gif (looks better)
>>>> http://www.bhalevy.com/pnfs/pnfs-page-sync-model.gif
>>> or even better
>>> http://www.bhalevy.com/pnfs/pnfs-page-sync-model.pdf
>> There are plenty of 'fail' loops in that graph. In those cases, how do
>> you decide which method to try next, and when do you decide to give up?
> 
> That's an important thing to define.
> In general I'd say that the decision should be made based on:
> - what failed
> - why it failed
> - is the error transient or permanent.
> 
> The analysis and recommendation should be up to the layout-driver
> since it knows best what exactly happened.
> 
> Now, regarding what to try next: if the layout driver recommended
> retrying with pNFS we should attempt to get a new layout (for simplicity
> reasons I think we should always refresh the layout, although it might
> not be strictly necessary in all cases).  If we failed to get the layout
> then if the error is permanent we plan to mark the _inode_ with
> NFS_INO_LAYOUT_FAILED which will avoid pNFS on that inode (until it's
> evicted from memory).  Otherwise, if the error is transient we suspend
> layout get on the inode and retry later (this is currently implemented
> in a rather naive way in pnfs_get_layout_done()).
> The number of retries over pNFS could be determined by the layout driver
> (e.g. some layout drivers would like to retry the pNFS forever forever
> as long as the errors are transient)

A couple more topics to mention:
- on the pNFS error path, I'd like to optionally send a layout commit
  with the layoutreturn (this is not drawn in the chart I posted yet).
  This allows the pnfs-obj LD to send the latest and greatest attribute
  update to the server, just flagged with an "ioerr".
  Then, in the same compound, we can also send a LAYOUTGET to refresh
  the layout (if we decide to try over pNFS again).

- Retrying on transient errors can be done with polling, or 
  we could set loga_signal_layout_avail, mark the page as pending_layout
  (this should probably a new state in the model), and wait for a 
  CB_RECALLABLE_OBJ_AVAIL callback.

- soft vs. hard mounts
  Timeout while either polling or waiting for the callback applies with
  soft mount should cause returning an error to the application.
  Signal handling should not be changed either.

> 
> Benny
> 
>> Trond
>>
>> _______________________________________________
>> pNFS mailing list
>> pNFS at linux-nfs.org
>> http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs
> 
> _______________________________________________
> pNFS mailing list
> pNFS at linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs