NFS Hangs ... NFS4ERR_DELAY

[Norman Elton]

Bruce --

I've posted the results from an strace on the client's idmapd:

http://pastebin.com/f760d62b5

Running strace on the server's idmapd doesn't seem to produce  
anything. It looks like idmapd is only consulted on the client's side.

I see some errors about /var/lib/nfs/rpc_pipefs/nfs/clnt4d/idmap not  
existing. In my case, /var/lib/nfs/rpc_pipefs/nfs is empty.

I'm running RHEL's kernel 2.6.18-53.el5xen (both client and server are  
xen virtual machines inside the same host).

Any thoughts?

Thanks,

Norman

On Dec 10, 2007, at 12:26 PM, J. Bruce Fields wrote:

> On Mon, Dec 10, 2007 at 12:21:04PM -0500, Norman Elton wrote:
>> Thanks for your note. I've posted the lsof output. Both appear to be
>> working, but maybe you'll spot something:
>>
>> Client - http://pastebin.com/f2e02caec
>> Server - http://pastebin.com/f72a777bb
>
> Yeah, looks fine--both the nametoid/channel and idtoname/channel files
> that rpc.idmapd needs to get requests from the server kernel are  
> there.
> Darn, that was the easy explanation....
>
> How about strace'ing rpc.idmapd while the mount is in progress?
>
> Also, remind me which kernel this is (approximately)?
>
>> I'm using RHEL5, which appears to build NFS as a kernel module (it
>> shows up in lsmod). I'm pretty sure that NFS starts after idmapd, but
>> I'm double checking now.
>>
>> Also... in idmapd.conf... should the "domain" be the DNS domain, or
>> the Kerberos domain? In our case, they're different.
>
> Probably the former.  But the only real requirement is that you choose
> the same domain on the client and server.
>
> --b.
>
>>> On Mon, Dec 10, 2007 at 11:44:30AM -0500, Norman Elton wrote:
>>>> Hello,
>>>>
>>>> I wrote last week regarding NFS hangs as we're migrating towards  
>>>> NFS4
>>>> + Kerberos. I believe I've got all the kerberos issues straightened
>>>> out, but we're experiencing hung connections, both when mounting  
>>>> and
>>>> when doing an "ls" of individual shares.
>>>>
>>>> Here's the dialogue I've observed with tcpdump when mounting a  
>>>> share:
>>>>
>>>>>>> V4 Null Call
>>>> <<< V4 Null Reply
>>>>>>> V4 Null Call
>>>> <<< V4 Null Reply
>>>>>>> V4 Null Call, now with GSS Token
>>>> <<< V4 Null Reply, again, with GSS information
>>>>>>> V4 Compound Request (PUTROOTFH, GETFH, GETATTR)
>>>> <<< V4 Compound Reply (PUTROOTFH=ok, GETFH=ok,  
>>>> GETATTR=nfs4err_delay)
>>>>
>>>> This last transaction goes back and forth a few times, with the
>>>> GETATTR operation consistently returning NFS4ERR_DELAY.  
>>>> Eventually I
>>>> have to kill the mount request.
>>>>
>>>> A little googling indicates that this could be a problem with  
>>>> idmapd.
>>>> I've configured our DNS domain in idmapd.conf, no luck. I've run
>>>> idmapd in debug mode ("-f -vvv") and do not get any output. I'm not
>>>> sure if idmapd is functioning properly or not.
>>>>
>>>> Any ideas?
>>>
>>> Could you check which files idmapd has open?  (ls -l /proc/`pidof
>>> idmapd`/fd/, or lsof rpc.idmapd).
>>>
>>> Also, is nfsd built as a module, and is there a chance you're  
>>> starting
>>> rpc.idmapd before loading that module?
>>>
>>> --b.
>>
>> _______________________________________________
>> NFSv4 mailing list
>> NFSv4 at linux-nfs.org
>> http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4