[pnfs] device export ops

[Dean Hildebrand]

Hi All,

At today's meeting people wanted a refresher on the changes to the device export operations I am planning to make.  This is a summary of an email I sent in October.  As a side note, if all goes well with this interface, I will propose doing the same thing for layouts, i.e., replacing the export ops layout_encode,layout_free, and layout_get with a single export op that is called in nfsd/nfs4xdr.c which retrieves and encodes the layout.

=======

At bakeathon 2007 in AA we talked reducing the number of
export ops and data copies it requires to retrieve device information.
 The current ops retrieve the device, encode it, and them free it,
which is quite cumbersome and requires extra data copies.  We noticed
that readdir only has a single vfs op, so why can't we do something
similar.  In addition, nfs readdir only calls this vfs op at xdr time,
not in nfs4proc.

To that end, Benny suggested moving to an iterator concept.  The main
goals of this plan is to:
a) Reduce the number of export ops
b) Copy device information only once (onto the xdr buffer).

The current 4 export ops are replaced with 2 export ops.  These two
ops satisfy both getdeviceinfo and getdevicelist.
1. get_device_iter - An iterator function which NFSD keeps calling until it has retrieved
all device ids. (using cookies, etc as an index)
2. get_device_info - An function to encode a specific device id.

On the server, GETDEVICEINFO will perform a subset of GETDEVICELIST:

For GETDEVICEINFO:
1. Call get_device_info with the args from the client.  This
will encode the device info for the supplied device id onto the output stream.

For GETDEVICELIST:
1. Call get_device_iter with the cookie and verifier from either the client
or the previous invocation of get_device_iter.
2. Call get_device_info with the device id returned by get_device_iter

Note on encoding the device
============================
get_device_info takes a function pointer pnfs_encodedev_t as an argument. 
This argument allows NFSD to supply a function to encode the device, instead
of having the file system do the encoding.  The file system takes the
pnfs_encodedev_t argument and uses it to encode its devices.

This is only useful for layout types which exist in the kernel, 
i.e., only file layout for now, object andblock will set this 
function pointer to NULL.  In the future, when object or block 
is in the kernel, they can use this function to move
device encoding into the common nfsd code.

DATA STRUCTURES
===============
/* Args:
 * xdr - xdr stream
 * device - pointer to device to be encoded
*/
typedef int (*pnfs_encodedev_t)(struct pnfs_xdr_info* xdr, void *device);

struct pnfs_xdr_info {
      u32 *p;            /* in */
      u32 *end;          /* in */
      u32 bytes_written; /* out */
};

struct pnfs_devinfo_arg {
      u32 gdi_type;        /* request */
      u32 gdi_maxcount;    /* request */
      u64 gdi_devid;       /* request */
};

struct pnfs_deviter_arg {
      u32 gdl_type;        /* request */
      u64 gdl_cookie;      /* req/resp */
      u64 gdl_verf;        /* req/resp */
      u64 gdl_devid;       /* resp */
};

EXPORT OPERATIONS ADDED
=======================
/* Args:
 * sb - superblock
 * arg - layout type, device id, maxcount
 * xdr - xdr stream for encoding
 * func - Optional function called by file system to encode device on xdr stream.
 *        Currently only used by file, block and object will leave it NULL.
 */
int (*get_device_info) (struct super_block *sb,
                        struct pnfs_devinfo_arg *arg,
                        struct pnfs_xdr_info *xdr,
                        pnfs_encodedev_t func););


int (*get_device_iter) (struct super_block *sb,
                        struct pnfs_deviter_arg *arg);

EXPORT OPERATIONS REMOVED
=========================
get_deviceinfo
get_devicelist
devaddr_free
devaddr_encode

Dean