[pnfs] [PATCH 1/3] pnfs-gfs2: initial LAYOUT* work for pNFS/GFS2 integration
david m. richter
richterd at citi.umich.edu
Mon Jun 2 12:58:08 EDT 2008
On Sun, 1 Jun 2008, Benny Halevy wrote:
> On May. 30, 2008, 19:30 +0300, "david m. richter" <richterd at citi.umich.edu> wrote:
> > thanks for the fixups, dean :) i had been wondering if/how the
> > "numDests" that we saw in cxiSystem.c was going to affect making the
> > layout filehandles in the gfs2 world, but you took care of that nicely.
>
> FYI: I didn't apply any of Dean's comments.
> Please resend the fixed patch (based on the newest version in the tree
> please) or a squashable patch with the fixups...
>
> Benny
hm, seems i should work through my mailbox backwards :)
i'll gin up a squashable patch straightaway -- thanks, benny,
d
.
>
> >
> > thanks!
> >
> > On Thu, 29 May 2008, Dean Hildebrand wrote:
> >
> >>
> >> David M. Richter wrote:
> >>> Frank Filz's work on the layout_type() and layout_get() export operations,
> >>> with stubs for layout_commit() and layout_return(). Tested at Connectathon.
> >>>
> >>> Signed-off-by: Frank Filz <ffilzlnx at us.ibm.com>
> >>> Signed-off-by: David M. Richter <richterd at citi.umich.edu>
> >>> ---
> >>> fs/gfs2/ops_export.c | 109
> >>> ++++++++++++++++++++++++++++++++++++++++++++++++++
> >>> 1 files changed, 109 insertions(+), 0 deletions(-)
> >>>
> >>> diff --git a/fs/gfs2/ops_export.c b/fs/gfs2/ops_export.c
> >>> index 334c7f8..b62cc66 100644
> >>> --- a/fs/gfs2/ops_export.c
> >>> +++ b/fs/gfs2/ops_export.c
> >>> @@ -27,6 +27,16 @@
> >>> #include "rgrp.h"
> >>> #include "util.h"
> >>> +#if defined(CONFIG_PNFSD)
> >>> +#include <linux/nfs_fs.h>
> >>> +#include <linux/sunrpc/svc.h>
> >>> +#include <linux/nfsd/state.h>
> >>> +#include <linux/nfsd/nfsfh.h>
> >>> +#include <linux/nfsd/pnfsd.h>
> >>> +#include <linux/nfsd/nfs4layoutxdr.h>
> >>> +#include <linux/nfs4_pnfs.h>
> >>> +#endif /* CONFIG_PNFSD */
> >>> +
> >>> #define GFS2_SMALL_FH_SIZE 4
> >>> #define GFS2_LARGE_FH_SIZE 8
> >>> #define GFS2_OLD_FH_SIZE 10
> >>> @@ -294,11 +304,110 @@ static struct dentry *gfs2_fh_to_parent(struct
> >>> super_block *sb, struct fid *fid,
> >>> }
> >>> }
> >>> +#if defined(CONFIG_PNFSD)
> >>> +static int gfs2_layout_type(void)
> >>> +{
> >>> + return LAYOUT_NFSV4_FILES;
> >>> +}
> >>> +
> >>> +/* like GPFS does, use a multiple of the underlying fs' blocksize */
> >>>
> >> no so important now, but in the long run I'd remove any mention of gpfs.
> >>> +static int get_stripe_unit(int blocksize)
> >>> +{
> >>> + if (blocksize >= NFSSVC_MAXBLKSIZE)
> >>> + return blocksize;
> >>> +
> >>> + return NFSSVC_MAXBLKSIZE - NFSSVC_MAXBLKSIZE % blocksize;
> >>>
> >> I would add () just to make sure it is clear.
> >>> +}
> >>> +
> >>> +/*
> >>> + * Retrieve and encode a file layout onto the xdr stream.
> >>> + * @inode: inode for which to retrieve layout
> >>> + * @arg->xdr: xdr stream for encoding
> >>> + * @arg->func: a call into file system to encode the layout on xdr stream.
> >>> + */
> >>> +static int gfs2_layout_get(struct inode *inode, struct pnfs_layoutget_arg
> >>> *arg)
> >>> +{
> >>> + int i, rc = 0;
> >>> + int num_dests = 1; /* XXX: revisit */
> >>> + struct pnfs_filelayout_layout *layout = NULL;
> >>> + struct knfsd_fh *fh_list = NULL;
> >>> +
> >>> + printk(KERN_DEBUG "%s: LAYOUT_GET\n", __func__);
> >>> +
> >>> + /* Set layout indept response args */
> >>> + arg->seg.layout_type = LAYOUT_NFSV4_FILES;
> >>> + arg->seg.offset = 0;
> >>> + arg->seg.length = inode->i_sb->s_maxbytes; /* The maximum file size */
> >>> +
> >>> + layout = kzalloc(sizeof(*layout), GFP_KERNEL);
> >>> + if (layout == NULL) {
> >>> + rc = -ENOMEM;
> >>> + goto error;
> >>> + }
> >>> +
> >>> + /* Set file layout response args */
> >>> + layout->lg_layout_type = LAYOUT_NFSV4_FILES;
> >>> + layout->lg_stripe_type = STRIPE_SPARSE;
> >>> + layout->lg_commit_through_mds = true;
> >>> + layout->lg_stripe_unit = get_stripe_unit(inode->i_sb->s_blocksize);
> >>> + layout->lg_fh_length = num_dests;
> >>> + layout->device_id.pnfs_fsid = arg->fsid;
> >>> + layout->device_id.pnfs_devid = 1; /*FSFTEMP*/
> >>> + layout->lg_first_stripe_index = 0; /*FSFTEMP*/
> >>> + layout->lg_pattern_offset = 0;
> >>> +
> >>> + fh_list = kmalloc(sizeof(*fh_list) * num_dests, GFP_KERNEL);
> >>> + if (fh_list == NULL) {
> >>> + rc = -ENOMEM;
> >>> + goto error;
> >>> + }
> >>> +
> >>> + memcpy(&fh_list[0], arg->fh, sizeof(*fh_list));
> >>> + pnfs_fh_mark_ds(&fh_list[0]);
> >>> +
> >>> + if (num_dests > 1)
> >>> + for (i = 1; i < num_dests; i++)
> >>> + memcpy(&fh_list[i], &fh_list[0], sizeof(*fh_list));
> >>>
> >> We just noticed this in gpfs, but we only need to set 1 file handle.
> >>
> >> from draft23:
> >> nfl_fh_list: An array of data server filehandles for each list of data servers
> >> in each element of the nflda_multipath_ds_list array. The number of elements
> >> in nfl_fh_list depends on whether sparse or dense packing is being used.
> >>
> >> * If sparse packing is being used, the number of elements in
> >> nfl_fh_list MUST be one of three values:
> >> o Zero. This means that filehandles used for each data server
> >> are the same as the filehandle returned by the OPEN
> >> operation from the metadata server.
> >> o One. This means that every data server uses the same
> >> filehandle: what is specified in nfl_fh_list[0].
> >> o The same number of elements in nflda_multipath_ds_list.
> >> Thus, in this case, when issuing an I/O to any data server
> >> in nflda_multipath_ds_list[X], the filehandle in
> >> nfl_fh_list[X] MUST be used.
> >>
> >>
> >>> + layout->lg_fh_list = fh_list;
> >>> +
> >>> + /* Call nfsd to encode layout */
> >>> + rc = arg->func(&arg->xdr, layout);
> >>> +exit:
> >>> + kfree(layout);
> >>> + kfree(fh_list);
> >>> + return rc;
> >>> +
> >>> +error:
> >>> + arg->seg.length = 0;
> >>> + goto exit;
> >>> +}
> >>> +
> >>> +static int gfs2_layout_commit(struct inode *inode, void *p)
> >>> +{
> >>> + printk(KERN_DEBUG "%s: LAYOUT_COMMIT (unimplemented)\n", __func__);
> >>> +
> >>> + return 0;
> >>> +}
> >>> +
> >>> +static int gfs2_layout_return(struct inode *inode, void *p)
> >>> +{
> >>> + printk(KERN_DEBUG "%s: LAYOUT_RETURN (unimplemented)\n", __func__);
> >>> +
> >>> + return 0;
> >>> +}
> >>> +#endif /* CONFIG_PNFSD */
> >>> +
> >>> const struct export_operations gfs2_export_ops = {
> >>> .encode_fh = gfs2_encode_fh,
> >>> .fh_to_dentry = gfs2_fh_to_dentry,
> >>> .fh_to_parent = gfs2_fh_to_parent,
> >>> .get_name = gfs2_get_name,
> >>> .get_parent = gfs2_get_parent,
> >>> +#if defined(CONFIG_PNFSD)
> >>> + .layout_type = gfs2_layout_type,
> >>> + .layout_get = gfs2_layout_get,
> >>> + .layout_commit = gfs2_layout_commit,
> >>> + .layout_return = gfs2_layout_return,
> >>> +#endif /* CONFIG_PNFSD */
> >>> };
> >>>
>
>
More information about the pNFS
mailing list