[pnfs] CVS: nfsv4

[Iyer, Rahul]

So, here's what happens. Nfs4_clear_inode() is called on every cached
inode during umount, in order to clear the inodes from the cache. In
nfs4_clear_inode(), free_layout() and pnfs_return_layout() are called
for the particular inode in question. The former frees the layout, and
the latter issues the LAYOUTRETURN RPC. Now, if the LAYOUTRETURN RPC is
asynchronous, the rpc call returns after issuing the rpc. This causes
pnfs_return_layout to return, followed by nfs4_clear_inode. Now, the
umount code assumes that on  clear_inode(), the inode is clear and the
inode can be thrown away. Further, once all inodes are cleaned, it
assumes that the rpc_clnt struct is no longer needed. So, it destroys
it, terminating any outstanding RPCs. Now, at this time, the
LAYOUTRETURN rpc is still outstanding, and it gets destroyed. So, you
would get an error of -EIO from the LAYOUTRETURN rpc most of the time...

In the case of a sync call, pnfs_return_layout() does not return until
the LAYOUTRETURN rpc completes. So, the above case does not arise.

Regards
Rahul

-----Original Message-----
From: Marc Eshel [mailto:eshel at almaden.ibm.com] 
Sent: Tuesday, August 01, 2006 1:58 PM
To: Iyer, Rahul
Cc: pnfs at linux-nfs.org; Rahul Iyer (NetApp)
Subject: RE: [pnfs] CVS: nfsv4

"Iyer, Rahul" <Rahul.Iyer at netapp.com> wrote on 08/01/2006 01:26:12 PM:

> Using the rpc_clnt struct gets the a reference to the counter, but the

> rpc shutdown code forcefully causes the task to exit with -EIO.
> Unfortunately, having a referennce does not help.

What happens to sync rpc and other layout that are cached in the case of
rpc shutdown?

> 
> Incidentally, returning of delegations is also done in a similar 
> fashion (async call followed by a wait for its completion) Regards 
> Rahul
> 
> 
> -----Original Message-----
> From: William A.(Andy) Adamson [mailto:andros at citi.umich.edu]
> Sent: Tuesday, August 01, 2006 12:55 PM
> To: Iyer, Rahul
> Cc: Marc Eshel; Rahul Iyer (NetApp); pnfs at linux-nfs.org; 
> andros at citi.umich.edu
> Subject: Re: [pnfs] CVS: nfsv4
> 
> > Hi,
> > I mean the inode freeing routines, not the free_layout () call. 
> > Here's
> 
> > the problem...
> > The xdr encode routine takes the inode as part of the arguments. It 
> > uses this to get the file handle for the PUTFH in the compound 
> > containing the LAYOUTRETURN. I agree, we could get the file handle 
> > earlier and then pass this down instead of the inode.
> > 
> > But the other problem is that we need the rpc_clnt struct too. This 
> > is
> 
> > also present in the inode. We could extract this also, but we wind 
> > up in a race with umount. During umount, the rpc_shutdown_clients 
> > kills all outstanding rpc_clnt structs.
> > 
> > So, here's my plan. I plan to make ir sort of async, but not
exactly. 
> > I will issue the rpc asynchronously, but wait for it to complete. 
> > While this is not truly asynchronous, it avoids the races we have. 
> > It however, is slightly better than synchronous because the user can

> > now interrupt it.
> 
> can't you hold a reference to the structures that you need while the 
> async rpc is in flight?
> 
> -->Andy
> 
> > Thanks
> > Regards
> > Rahul
> > 
> > 
> > -----Original Message-----
> > From: Marc Eshel [mailto:eshel at almaden.ibm.com]
> > Sent: Tuesday, August 01, 2006 9:20 AM
> > To: Iyer, Rahul
> > Cc: Rahul Iyer (NetApp); pnfs at linux-nfs.org
> > Subject: RE: [pnfs] CVS: nfsv4
> > 
> > What exactly is the problem? Can't you remove the pointer 
> > current_layout from the inode (set it to NULL) and than you can 
> > either
> 
> > free it even before the async call or if you need it for some reason

> > free it in rpc_call_done ?
> > I think that this call should be async.
> > Marc.
> > 
> > pnfs-bounces at linux-nfs.org wrote on 07/31/2006 10:29:22 PM:
> > 
> > > PFA the patch for the following commit. 
> > > Regards
> > > Rahul
> > > 
> > > 
> > > -----Original Message-----
> > > From: Rahul Iyer (NetApp) [mailto:iyer at citi.umich.edu]
> > > Sent: Monday, July 31, 2006 10:28 PM
> > > To: pnfs at linux-nfs.org
> > > Subject: [pnfs] CVS: nfsv4
> > > 
> > > CVSROOT:   /cvs
> > > Module name:   nfsv4
> > > Changes by:   iyer at citi.   2006/08/01 01:28:20
> > > 
> > > Modified files:
> > >    cvs/pnfs/fs/nfs: nfs4proc.c
> > > 
> > > Log message:
> > > Making LAYOUTRETURN synchronous. Having LAYOUTRETURN be 
> > > asynchronous
> 
> > > causes race condition between LAYOUTRETURN and the inode freeing 
> > > routines.
> > > 
> > > _______________________________________________
> > > pNFS mailing list
> > > pNFS at linux-nfs.org
> > > http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs
> > > [attachment "diff" deleted by Marc Eshel/Almaden/IBM] 
> > > _______________________________________________
> > > pNFS mailing list
> > > pNFS at linux-nfs.org
> > > http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs
> > _______________________________________________
> > pNFS mailing list
> > pNFS at linux-nfs.org
> > http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

_______________________________________________
pNFS mailing list
pNFS at linux-nfs.org
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs