NFSv4 client's BUG?
J. Bruce Fields
bfields at fieldses.org
Fri Jul 11 15:29:13 EDT 2008
On Fri, Jul 11, 2008 at 05:02:29PM +0800, Wei Yongjun wrote:
> Hi:
>
> J. Bruce Fields wrote:
>> On Wed, Jul 02, 2008 at 09:55:16AM +0800, Wei Yongjun wrote:
>>
>>> J. Bruce Fields wrote:
>>>
>>>> On Wed, Jul 02, 2008 at 08:39:16AM +0800, Wei Yongjun wrote:
>>>>
>>>>> Ellard, Daniel wrote:
>>>>>
>>>>>> You're never closing filedes[i], and eventually you're running out of
>>>>>> file descriptors.
>>>>>>
>>>>>> Add
>>>>>>
>>>>>> close(filedes[i]);
>>>>>>
>>>>>> After
>>>>>>
>>>>>> write(filedes[i], "...
>>>>>>
>>>>> This is not the problem. Because I used ulimit -n 5000 to let
>>>>> program can open 5000 files.
>>>>> The "fopen: Input/output error" may happend when I just open 40
>>>>> files, it is random.
>>>>>
>>>> What's the server? Could you watch the traffic in wireshark and see
>>>> whether the server is returning an error on the failing open?
>>>>
>>> I have tcpdump the packets, there is no error return by the server.
>>>
>>
>> Does tcpdump ever tell you about nfsv4 errors? I'm not sure it actually
>> knows where to look for the nfsv4 status.
>>
>> Wireshark would be more helpful.
>>
>
> The error return by the server is NFSERR_OPENMODE in setattr.
>
> 646 nfsd4_setattr(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> 647 struct nfsd4_setattr *setattr)
> 648 {
> 649 __be32 status = nfs_ok;
> 650
> 651 if (setattr->sa_iattr.ia_valid & ATTR_SIZE) {
> 652 nfs4_lock_state();
> 653 status = nfs4_preprocess_stateid_op(&cstate->current_fh,
> 654 &setattr->sa_stateid, CHECK_FH | WR_STATE, NULL);
> 655 nfs4_unlock_state();
>
> 2002 nfs4_preprocess_stateid_op(struct svc_fh *current_fh, stateid_t *stateid, int flags, struct file **filpp)
> ...
> 2049 if (stp) {
> 2050 if ((status = nfs4_check_openmode(stp,flags)))
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 2051 goto out;
> 2052 renew_client(stp->st_stateowner->so_client);
> 2053 if (filpp)
> 2054 *filpp = stp->st_vfs_file;
> 2055 }
>
> ...
>
> This error only happend when NFSv4 callback is enabled. See the
> following code, if cb->cb_set is 1, deleg will be alloc, if cb->cb_set
> is 0, the error will never happend. I confirm this by drop callback tcp
> packet used iptables.
> if cb->cb_set is 1, the message "NFSD: setlease failed [11], no
> delegation" is print when open file with NFS4_SHARE_ACCESS_WRITE mode,
> when vfs_setlease(), the dir and inode count is 2 (may be one is hold by
> stp, the other is hold by dp), so the setlease for F_WRLCK always fail.
That's normal; the server doesn't support write delegations (and doesn't
give out read delegations on write opens).
So, is the problem occuring because the client is doing a setattr to
modify the size using a sequence id returned from a read delegation?
--b.
> 1599 static void
> 1600 nfs4_open_delegation(struct svc_fh *fh, struct nfsd4_open *open, struct nfs4_stateid *stp)
> 1601 {
> 1602 struct nfs4_delegation *dp;
> 1603 struct nfs4_stateowner *sop = stp->st_stateowner;
> 1604 struct nfs4_callback *cb = &sop->so_client->cl_callback;
> 1605 struct file_lock fl, *flp = &fl;
> 1606 int status, flag = 0;
> 1607
> 1608 flag = NFS4_OPEN_DELEGATE_NONE;
> 1609 open->op_recall = 0;
> 1610 switch (open->op_claim_type) {
> 1611 case NFS4_OPEN_CLAIM_PREVIOUS:
> 1612 if (!atomic_read(&cb->cb_set))
> 1613 open->op_recall = 1;
> 1614 flag = open->op_delegate_type;
> 1615 if (flag == NFS4_OPEN_DELEGATE_NONE)
> 1616 goto out;
> 1617 break;
> 1618 case NFS4_OPEN_CLAIM_NULL:
> 1619 /* Let's not give out any delegations till everyone's
> 1620 * had the chance to reclaim theirs.... */
> 1621 if (nfs4_in_grace())
> 1622 goto out;
> 1623 if (!atomic_read(&cb->cb_set) || !sop->so_confirmed)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 1624 goto out;
> 1625 if (open->op_share_access & NFS4_SHARE_ACCESS_WRITE)
> 1626 flag = NFS4_OPEN_DELEGATE_WRITE;
> 1627 else
> 1628 flag = NFS4_OPEN_DELEGATE_READ;
> 1629 break;
> 1630 default:
> 1631 goto out;
> 1632 }
> 1633
> 1634 dp = alloc_init_deleg(sop->so_client, stp, fh, flag);
> 1635 if (dp == NULL) {
> 1636 flag = NFS4_OPEN_DELEGATE_NONE;
> 1637 goto out;
> 1638 }
> 1639 locks_init_lock(&fl);
> 1640 fl.fl_lmops = &nfsd_lease_mng_ops;
> 1641 fl.fl_flags = FL_LEASE;
> 1642 fl.fl_type = flag == NFS4_OPEN_DELEGATE_READ? F_RDLCK: F_WRLCK;
> 1643 fl.fl_end = OFFSET_MAX;
> 1644 fl.fl_owner = (fl_owner_t)dp;
> 1645 fl.fl_file = stp->st_vfs_file;
> 1646 fl.fl_pid = current->tgid;
> 1647
> 1648 /* vfs_setlease checks to see if delegation should be handed out.
> 1649 * the lock_manager callbacks fl_mylease and fl_change are used
> 1650 */
> 1651 if ((status = vfs_setlease(stp->st_vfs_file, fl.fl_type, &flp))) {
> 1652 dprintk("NFSD: setlease failed [%d], no delegation\n", status);
> 1653 unhash_delegation(dp);
> 1654 flag = NFS4_OPEN_DELEGATE_NONE;
> 1655 goto out;
> 1656 }
> 1657
> 1658 memcpy(&open->op_delegate_stateid, &dp->dl_stateid, sizeof(dp->dl_stateid));
> 1659
> 1660 dprintk("NFSD: delegation stateid=(%08x/%08x/%08x/%08x)\n\n",
> 1661 dp->dl_stateid.si_boot,
> 1662 dp->dl_stateid.si_stateownerid,
> 1663 dp->dl_stateid.si_fileid,
> 1664 dp->dl_stateid.si_generation);
> 1665 out:
> 1666 if (open->op_claim_type == NFS4_OPEN_CLAIM_PREVIOUS
> 1667 && flag == NFS4_OPEN_DELEGATE_NONE
> 1668 && open->op_delegate_type != NFS4_OPEN_DELEGATE_NONE)
> 1669 dprintk("NFSD: WARNING: refusing delegation reclaim\n");
> 1670 open->op_delegate_type = flag;
> 1671 }
>
>
> Any idea?
>
> Regards.
>
More information about the NFSv4
mailing list