NFSv4 client's BUG?
Wei Yongjun
yjwei at cn.fujitsu.com
Fri Jul 11 05:02:29 EDT 2008
Hi:
J. Bruce Fields wrote:
> On Wed, Jul 02, 2008 at 09:55:16AM +0800, Wei Yongjun wrote:
>
>> J. Bruce Fields wrote:
>>
>>> On Wed, Jul 02, 2008 at 08:39:16AM +0800, Wei Yongjun wrote:
>>>
>>>
>>>> Ellard, Daniel wrote:
>>>>
>>>>
>>>>> You're never closing filedes[i], and eventually you're running out of
>>>>> file descriptors.
>>>>>
>>>>> Add
>>>>>
>>>>> close(filedes[i]);
>>>>>
>>>>> After
>>>>>
>>>>> write(filedes[i], "...
>>>>>
>>>>>
>>>> This is not the problem. Because I used ulimit -n 5000 to let program
>>>> can open 5000 files.
>>>> The "fopen: Input/output error" may happend when I just open 40
>>>> files, it is random.
>>>>
>>>>
>>> What's the server? Could you watch the traffic in wireshark and see
>>> whether the server is returning an error on the failing open?
>>>
>>>
>> I have tcpdump the packets, there is no error return by the server.
>>
>
> Does tcpdump ever tell you about nfsv4 errors? I'm not sure it actually
> knows where to look for the nfsv4 status.
>
> Wireshark would be more helpful.
>
The error return by the server is NFSERR_OPENMODE in setattr.
646 nfsd4_setattr(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
647 struct nfsd4_setattr *setattr)
648 {
649 __be32 status = nfs_ok;
650
651 if (setattr->sa_iattr.ia_valid & ATTR_SIZE) {
652 nfs4_lock_state();
653 status = nfs4_preprocess_stateid_op(&cstate->current_fh,
654 &setattr->sa_stateid, CHECK_FH | WR_STATE, NULL);
655 nfs4_unlock_state();
2002 nfs4_preprocess_stateid_op(struct svc_fh *current_fh, stateid_t *stateid, int flags, struct file **filpp)
...
2049 if (stp) {
2050 if ((status = nfs4_check_openmode(stp,flags)))
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2051 goto out;
2052 renew_client(stp->st_stateowner->so_client);
2053 if (filpp)
2054 *filpp = stp->st_vfs_file;
2055 }
...
This error only happend when NFSv4 callback is enabled. See the
following code, if cb->cb_set is 1, deleg will be alloc, if cb->cb_set
is 0, the error will never happend. I confirm this by drop callback tcp
packet used iptables.
if cb->cb_set is 1, the message "NFSD: setlease failed [11], no
delegation" is print when open file with NFS4_SHARE_ACCESS_WRITE mode,
when vfs_setlease(), the dir and inode count is 2 (may be one is hold by
stp, the other is hold by dp), so the setlease for F_WRLCK always fail.
1599 static void
1600 nfs4_open_delegation(struct svc_fh *fh, struct nfsd4_open *open, struct nfs4_stateid *stp)
1601 {
1602 struct nfs4_delegation *dp;
1603 struct nfs4_stateowner *sop = stp->st_stateowner;
1604 struct nfs4_callback *cb = &sop->so_client->cl_callback;
1605 struct file_lock fl, *flp = &fl;
1606 int status, flag = 0;
1607
1608 flag = NFS4_OPEN_DELEGATE_NONE;
1609 open->op_recall = 0;
1610 switch (open->op_claim_type) {
1611 case NFS4_OPEN_CLAIM_PREVIOUS:
1612 if (!atomic_read(&cb->cb_set))
1613 open->op_recall = 1;
1614 flag = open->op_delegate_type;
1615 if (flag == NFS4_OPEN_DELEGATE_NONE)
1616 goto out;
1617 break;
1618 case NFS4_OPEN_CLAIM_NULL:
1619 /* Let's not give out any delegations till everyone's
1620 * had the chance to reclaim theirs.... */
1621 if (nfs4_in_grace())
1622 goto out;
1623 if (!atomic_read(&cb->cb_set) || !sop->so_confirmed)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1624 goto out;
1625 if (open->op_share_access & NFS4_SHARE_ACCESS_WRITE)
1626 flag = NFS4_OPEN_DELEGATE_WRITE;
1627 else
1628 flag = NFS4_OPEN_DELEGATE_READ;
1629 break;
1630 default:
1631 goto out;
1632 }
1633
1634 dp = alloc_init_deleg(sop->so_client, stp, fh, flag);
1635 if (dp == NULL) {
1636 flag = NFS4_OPEN_DELEGATE_NONE;
1637 goto out;
1638 }
1639 locks_init_lock(&fl);
1640 fl.fl_lmops = &nfsd_lease_mng_ops;
1641 fl.fl_flags = FL_LEASE;
1642 fl.fl_type = flag == NFS4_OPEN_DELEGATE_READ? F_RDLCK: F_WRLCK;
1643 fl.fl_end = OFFSET_MAX;
1644 fl.fl_owner = (fl_owner_t)dp;
1645 fl.fl_file = stp->st_vfs_file;
1646 fl.fl_pid = current->tgid;
1647
1648 /* vfs_setlease checks to see if delegation should be handed out.
1649 * the lock_manager callbacks fl_mylease and fl_change are used
1650 */
1651 if ((status = vfs_setlease(stp->st_vfs_file, fl.fl_type, &flp))) {
1652 dprintk("NFSD: setlease failed [%d], no delegation\n", status);
1653 unhash_delegation(dp);
1654 flag = NFS4_OPEN_DELEGATE_NONE;
1655 goto out;
1656 }
1657
1658 memcpy(&open->op_delegate_stateid, &dp->dl_stateid, sizeof(dp->dl_stateid));
1659
1660 dprintk("NFSD: delegation stateid=(%08x/%08x/%08x/%08x)\n\n",
1661 dp->dl_stateid.si_boot,
1662 dp->dl_stateid.si_stateownerid,
1663 dp->dl_stateid.si_fileid,
1664 dp->dl_stateid.si_generation);
1665 out:
1666 if (open->op_claim_type == NFS4_OPEN_CLAIM_PREVIOUS
1667 && flag == NFS4_OPEN_DELEGATE_NONE
1668 && open->op_delegate_type != NFS4_OPEN_DELEGATE_NONE)
1669 dprintk("NFSD: WARNING: refusing delegation reclaim\n");
1670 open->op_delegate_type = flag;
1671 }
Any idea?
Regards.
More information about the NFSv4
mailing list