[pnfs] 19 questions
Dean Hildebrand
seattleplus at gmail.com
Wed Feb 6 16:35:43 EST 2008
William A. (Andy) Adamson wrote:
>
>
> On Feb 6, 2008 2:27 PM, Dean Hildebrand <seattleplus at gmail.com
> <mailto:seattleplus at gmail.com>> wrote:
>
>
>
> William A. (Andy) Adamson wrote:
> >
> >
> > On Feb 6, 2008 1:26 PM, Dean Hildebrand <seattleplus at gmail.com
> <mailto:seattleplus at gmail.com>
> > <mailto:seattleplus at gmail.com <mailto:seattleplus at gmail.com>>>
> wrote:
> >
> >
> > William A. (Andy) Adamson wrote:
> > >
> > >
> > > On 2/4/08, *Dean Hildebrand* <seattleplus at gmail.com
> <mailto:seattleplus at gmail.com>
> > <mailto:seattleplus at gmail.com <mailto:seattleplus at gmail.com>>
> > > <mailto:seattleplus at gmail.com
> <mailto:seattleplus at gmail.com> <mailto:seattleplus at gmail.com
> <mailto:seattleplus at gmail.com>>>>
> > wrote:
> > >
> > > Some questions about draft 19 (not 19 questions):
> > >
> > > 1) On the server I need the superblock so I can call the
> > getdeviceinfo
> > > export operation.
> > >
> > >
> > > why? the deviceid space is global.
> > Well, the devices and the device ids still come from the
> file system,
> > and the only way I know how to talk to the file system is
> through the
> > export ops, which requires a superblock.
> >
> > To make them globally unique, through coincidence or the
> fact that we
> > are all very like minded people, we had already coded up the
> > deviceid on
> > the server as Benny suggested:
> >
> > struct pnfs_deviceid {
> > u64 ex_fsid;
> > __be64 devid;
> > };
> >
> > One limitation here is that in include/linux/nfsd/nfsfh.h it
> > defines the
> > sizes for various types of fsids:
> > static inline int key_len(int type)
> > {
> > switch(type) {
> > case FSID_DEV: return 8;
> > case FSID_NUM: return 4;
> > case FSID_MAJOR_MINOR: return 12;
> > case FSID_ENCODE_DEV: return 8;
> > case FSID_UUID4_INUM: return 8;
> > case FSID_UUID8: return 8;
> > case FSID_UUID16: return 16;
> > case FSID_UUID16_INUM: return 24;
> > default: return 0;
> > }
> > }
> >
> > So if we are limiting the fsid to be only 8 bytes, then
> there are
> > several types of fsids we cannot support. I'm wondering if
> this is a
> > major problem for all OSs or just a hassle for all OSs.
> There are
> > workarounds I guess (replace fsid with a made up id in
> > /etc/exports) but
> > they seem like a hassle. What do people think?
> >
> >
> > svc_export->ex_dentry->d_inode->i_sb gives you a super block for an
> > export.
> >
> > the server can remember which exports (only need one) are pNFS
> vi any
> > pNFS
> > call (such as the fs_layout_type GETATTR) with a new
> svc_export->ex_flag.
> > e.g. EX_PNFS_FILE, EX_PNFS_BLOCK, etc.
> >
> > else, the server can check for the existence of
> export_op->layout_type
> > (e.g. call nfsd_layout_verify) on each exported super_block until it
> > finds one that matches the GETDEVICELIST layout type.
> Hmmm, the problem I see is that there could be multiple exported file
> systems supporting the same layout type.
>
>
> this is a good question. if the MDS exports two file layout GPFS
> filesystems, one could expect a device ID to be "global" across both
> file systems.
> but, if the MDS export one file layout GPFS file system and one spNFS
> file system - no way!
>
> what does a "global device id" mean? global to layout type/MDS? no.
> global to layout type/vendor?
In 3.3.14, I also found a definition of deviceid which I wish we could
keep but seems to not have any support:
" The device ID is qualified by the layout type and are unique per file
system (FSID). "
I think it should say,
" The device ID is qualified by the layout type and may be different for
each clientid"
Not sure why a server would want to give different deviceids per
clientid, but that seems to be the agreement from the conf call.
>
> i say we add a currentfh to GETDEVICE[INFO] and be done with it.
From what I remember, the reason they didn't want the currentfh with
getdeviceinfo is notifications. If deviceids are shared across 2
fsid's, they don't want to have to send 2 getdeviceinfo calls to set
notifications for the same device. I think that was the issue.
My current thought, if we keep the current spec, is that nfsd will pass
a unique identifier (fsid) as an argument to the getdevicelist and
getlayout export ops, so that when the file system builds the deviceid
it can make it unique.
Also,
From 3.3.14: "A client must not assume that device IDs are valid across
metadata server reboots."
Which I think means that after an MDS reboot, the client may find that
devices don't work, but the server should do everything it can to ensure
they are the same and that they do work. Another interpretation is that
the client must return all its layouts and retrieve them all over again.
Dean
>
> -->Andy
>
> So matching the exported
> layout type to the getdeviceinfo layout type isn't sufficient. You
> would have to call export_op->getdeviceinfo on each exported file
> system
> that matches the layout type until you find one that recognizes the
> deviceid. ugh.
>
> So I think that we still need to somehow encapsulate which export
> we are
> referring to in the deviceid. But if fsid's can be up to 24 bytes
> long,
> we have a problem.
>
> Dean
>
> > > The problem is that getdeviceinfo no longer includes
> > > the current_fh. Is there a way on the server to get the
> > superblock
> > > without the filehandle? (maybe using something in the
> > > session?) If not,
> > > then we will have to encode something in the device id
> that
> > allows
> > > us to
> > > map to the superblock (is the fsid enough?) Any ideas?
> > >
> > > 2) Can getdeviceinfo return a device for the wrong layout
> > type? It
> > > seems that getdeviceinfo returns the layout type, which
> > seems a little
> > > silly to me. If it should return the layout type, and it
> > doesn't
> > > match
> > > the requesting layout type, what should the client do?
> > >
> > > 3) In draft-19, should bitmap4 not have <>?
> > > bitmap4<> typedef uint32_t bitmap4<>;
> > > ^^^^
> > > For reference, in rfc3530 it does not have the <>. (other
> > ops have the
> > > <> added as well)
> > >
> > > Thanks,
> > > Dean
> > > _______________________________________________
> > > pNFS mailing list
> > > pNFS at linux-nfs.org <mailto:pNFS at linux-nfs.org>
> <mailto:pNFS at linux-nfs.org <mailto:pNFS at linux-nfs.org>>
> > <mailto:pNFS at linux-nfs.org <mailto:pNFS at linux-nfs.org>
> <mailto:pNFS at linux-nfs.org <mailto:pNFS at linux-nfs.org>>>
> > > http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs
> > >
> > >
> >
> >
>
>
More information about the pNFS
mailing list