nfs/svn/apache strange interaction

[J. Bruce Fields]

On Mon, Apr 21, 2008 at 06:05:09PM +0200, Guillaume Rousse wrote:
> Hello.
> 
> I recently ran into troubles with Apache serving user home pages from a
> nfs-mounted read-only filesystem. Basically, and modification from a
> file in ~user/public_html on the machine hosting the filesystem (the nfs
> server) through svn client triggers a 404 issue on apache side, with the
> following message in the logs:
> [Mon Apr 21 15:49:56 2008] [error] [client 195.83.212.57] (116)Stale NFS
> file handle: access to /~zalinesc/robots.txt failed
> 
> Any modification with a text editor on nfs server works OK, so  I first
> suspected classical NFS/SVN interaction, described at
> https://bugzilla.redhat.com/show_bug.cgi?id=236308
> However, mounting the file system with noac option didn't help. Also,
> accessing the file from a shell on the apache server works OK, even as
> apache user (su apache -c "cat ~user/public_html/file").
> 
> I then suspected an Apache/NFS issue. From explanations given at
> http://marc.info/?l=apache-httpd-users&m=115410720121248&w=2, it seems
> apache caches NFS inodes a few seconds, and any underlying modification
> is susceptible to result in such error.
> 
> On apache side, all options known to trigger issue with NFS have been
> disabled:
>  <Directory /home/members/*/public_html>
>        [..]
>         EnableSendfile Off
>         EnableMMAP Off
>     </Directory>
> 
> I'm using nfs4, with the following exports options:
> /home	-async,rw,no_subtree_check,fsid=0 chatelet(ro)
> and the following mount options:
> rw,nosuid,nodev,hard,intr
> 
> I've tried changing async to sync for exports options, with no result,
> or to add noac option for mount options, with no result either.
> Restarting apache doesn't help, the only solution is to umount and
> remount the filesystem, which seem to imply some kind of cached
> information expiry.
> 
> I also tried to enable verbose nfs debugging, as suggested
> http://wiki.linux-nfs.org/wiki/index.php/General_troubleshooting_recommendations,
> but I couldn't find any useful information (maybe the value 3 is not
> adequate, tough).

The value "3" is (1 & 2), so turns on two different types of debugging;
the exact meaning of the flags depends on the syctl you set.  See the
header files referred to on that wiki page for definitions.

I think it'd be most interesting to look at the server, in particular
at:
	NFSDDBG_FH | NFSDDBG_EXPORT | NFSDDBG_PROC | NFSDDBG_FILEOP
	== 0x02 & 0x04 & 0x10 & 0x20 == 54

So:

	echo 54 >/proc/sys/sunrpc/nfsd_debug

--b.

> 
> I'm using linux, nfs-utils 1.1.1 and kernel 2.6.24.4 on both side
> (client and server). I'm not really sure it is nfs4-related, but I can't
> easily test with older nfs3 version, due to firewalling issues. Also, it
> happens just a week after a system upgrade for both the server and the
> client, which is quite susceptible to have introduced this change.
> 
> Any hint welcome.
> -- 
> Guillaume Rousse
> Moyens Informatiques - INRIA Futurs
> Tel: 01 69 35 69 62
> _______________________________________________
> NFSv4 mailing list
> NFSv4 at linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4