Ipv6PlanningDocument
From Linux NFS
Chucklever (Talk | contribs) m (indent some headings) |
Chucklever (Talk | contribs) (Update work-to-date and fix a broken link) |
||
| (22 intermediate revisions not shown) | |||
| Line 3: | Line 3: | ||
This article contains a rough implementation roadmap for IPv6 support in the Linux NFS client and server. | This article contains a rough implementation roadmap for IPv6 support in the Linux NFS client and server. | ||
| - | + | == Project Goals == | |
This effort will provide support in the Linux NFS client and server implementations for IPv6 networking. This includes all current versions of the NFS protocol (2, 3 and 4) and any supported auxiliary protocols (MNT, NLM, NSM, NFS_ACL, NFSv4 callback, rpcbind). User space support (idmapd, gssd, svcgssd, statd, sm-notify, mount/umount/showmount, mountd, exportfs) is also included. | This effort will provide support in the Linux NFS client and server implementations for IPv6 networking. This includes all current versions of the NFS protocol (2, 3 and 4) and any supported auxiliary protocols (MNT, NLM, NSM, NFS_ACL, NFSv4 callback, rpcbind). User space support (idmapd, gssd, svcgssd, statd, sm-notify, mount/umount/showmount, mountd, exportfs) is also included. | ||
| Line 9: | Line 9: | ||
In order to provide an interoperating implementation, kernel and user space RPC implementations will also be upgraded to support Transport-Independent RPC (TI-RPC). This includes the sunrpc.ko module, the IP socket-based transport capability, the in-kernel GSSD implementation, rpcbind/portmaper, libtirpc, and rpcsecgss/gssglue/krb5. | In order to provide an interoperating implementation, kernel and user space RPC implementations will also be upgraded to support Transport-Independent RPC (TI-RPC). This includes the sunrpc.ko module, the IP socket-based transport capability, the in-kernel GSSD implementation, rpcbind/portmaper, libtirpc, and rpcsecgss/gssglue/krb5. | ||
| - | + | The addition of IPv6 support should not effect the behavior or performance of bog standard IPv4 NFS installations. | |
| - | As of | + | For now, we assume that IPv4 will always be present. Communication between separate components on the same host will continue to use IPv4 loopback. |
| + | |||
| + | == What Is Taking So Long? == | ||
| + | |||
| + | Before support for NFS over IPv6 can be introduced, some major infrastructure overhaul has been required. | ||
| + | |||
| + | The NFS client's mount stack has been converted to using an in-kernel MNT client; mount options are now passed to the kernel via a C string instead of in a versioned binary blob. There are several advantages to using a C string here, but the main benefit for IPv6 support is that we can easily support larger addresses and additional mount options without co-ordinating a version change between the kernel and the mount command. | ||
| + | |||
| + | An RPC transport capability switch has been added to the kernel (both client and server) to enable IPv6-capable network transports. Support for TI-RPC has been introduced to the Linux user space via libtirpc and rpcbind. | ||
| + | |||
| + | The TI-RPC library presents some special challenges. First, it was implemented against the streams API, not for sockets. We took our port from the FreeBSD port, which was converted to use sockets. In addition, more contemporary versions of TI-RPC exist but are licensed under CDDL, which is not compatible with GPL. The stock Sun implementation is now about 15 years old. There are no longer any engineers at Sun with implementation experience in this area, and the documentation and specifications are incomplete, to say the least, so much of this has been done by guesswork and extensive source code archaeology. Finally, TI-RPC is close, but not perfectly API- and ABI-compatible with legacy RPC implementations such as the one in glibc. | ||
| + | |||
| + | The mount command was also challenging because the nfs-utils code base is over a decade old, but there are no unit tests or requirements documents that explain exactly how its features are supposed to work. The Linux mount command implements some features (such as probing the server to see what transport protocols and NFS versions are supported) that are difficult to implement using stock TI-RPC library calls. At least two complete re-implementations of the RPCB_GETADDR/PMAP_GETPORT RPC calls have to be implemented to support these legacy features, as TI-RPC hides much of this complexity. | ||
| + | |||
| + | Lack of adequate documentation of legacy feature requirements has been a major stumbling block. The man pages are either not complete, or are not specific about certain behaviors. There is little or no documentation of specific use cases for which these features are designed. In addition, it has been difficult to locate general IPv6 expertise. | ||
| + | |||
| + | As you can tell, these pre-requisites are already a substantial amount of development effort. | ||
| + | |||
| + | == Work To Date == | ||
| + | |||
| + | As of January, 2010, the following items have been completed upstream: | ||
* implementation of a new transport capability switch in the kernel's RPC client and server implementation | * implementation of a new transport capability switch in the kernel's RPC client and server implementation | ||
* support for additional address families in the kernel's RPC client and server implementation and server-side authentication cache | * support for additional address families in the kernel's RPC client and server implementation and server-side authentication cache | ||
* support for IPv6 networking in the IP socket transport in the kernel's RPC client and server implementation | * support for IPv6 networking in the IP socket transport in the kernel's RPC client and server implementation | ||
| - | * support for IPv6 and rpcbind protocol version 4 in the kernel's rpcbind client implementation | + | * support for IPv6 and rpcbind protocol version 4 (port-only) in the kernel's rpcbind client implementation |
* NFS mount command split out of the stock Linux mount command into a subcommand | * NFS mount command split out of the stock Linux mount command into a subcommand | ||
* conversion of the NFS mount operation to send mount options via a C string instead of a binary blob | * conversion of the NFS mount operation to send mount options via a C string instead of a binary blob | ||
| - | * support for rpcbind v4 GETADDR requests in the mount.nfs command | + | * support for port-only rpcbind v4 GETADDR requests in the mount.nfs command |
* support for AF_INET6 family addresses in the NFS client and server implementation | * support for AF_INET6 family addresses in the NFS client and server implementation | ||
* support for IPv6 server address parsing in the NFS client | * support for IPv6 server address parsing in the NFS client | ||
* support for IPv6 in the client's NFSv4 callback service | * support for IPv6 in the client's NFSv4 callback service | ||
| - | * replacement of Linux's portmapper implementation with a port of rpcbind | + | * replacement of Linux's user space portmapper implementation with a port of rpcbind |
* introduction of a port of libtirpc to user space | * introduction of a port of libtirpc to user space | ||
* support for IPv6 in the Linux kernel's lockd and NSM implementation | * support for IPv6 in the Linux kernel's lockd and NSM implementation | ||
* support for IPv6 and netids in the mount.nfs command | * support for IPv6 and netids in the mount.nfs command | ||
| - | + | * support for IPv6 in gssd | |
| - | + | * support for IPv6 in rpc.nfsd | |
| - | + | ||
* support for IPv6 and TI-RPC for NSM reboot notification (sm-notify command and rpc.statd daemon) | * support for IPv6 and TI-RPC for NSM reboot notification (sm-notify command and rpc.statd daemon) | ||
| - | |||
* support for IPv6 in the kernel's NFS server | * support for IPv6 in the kernel's NFS server | ||
| - | |||
| - | |||
| - | |||
| - | |||
* TCP wrapper support for IPv6 | * TCP wrapper support for IPv6 | ||
| - | * | + | * support specifying a netid instead of a protocol name via a mount option |
| + | * support for IPv6 and TI-RPC for server-side MNT protocol (rpc.mountd and exportfs) | ||
| + | * support IPv6 in Linux pNFS client and server | ||
* support for IPv6 in Linux port of Kerberos | * support for IPv6 in Linux port of Kerberos | ||
| - | * support for IPv6 in | + | * support for IPv6 in svcgssd |
| - | + | ||
* replace legacy RPC implementation in glibc with libtirpc | * replace legacy RPC implementation in glibc with libtirpc | ||
| + | * support in lockd for multiple caller_names | ||
| + | * implement AF_UNIX transport in kernel RPC client | ||
| + | |||
| + | === Now In Prototype === | ||
| + | |||
| + | === Remaining To Do === | ||
| + | |||
| + | * implement AF_UNIX transport in kernel RPC server | ||
| + | * full documentation for using IPv6 with the NFS client and server | ||
* audit kernel RPCSEC GSSAPI implementation for IPv6 requirements | * audit kernel RPCSEC GSSAPI implementation for IPv6 requirements | ||
| - | * audit rpc.idmapd implementation for IPv6 requirements | + | * audit rpc.idmapd implementation for IPv6 requirements (LDAP) |
| - | * complete | + | * test support of IPv6 addresses during NFSv4 migration events |
| + | * complete audit of work to date with eye towards eliminating code duplication | ||
| + | * full support for link-local IPv6 addresses | ||
| + | * check Linux implementation against issues in [http://tools.ietf.org/search/draft-alexrn-nfsv4-ipv6-00 draft-alexrn-nfsv4-ipv6-00] | ||
| - | = | + | [[https://bugzilla.redhat.com/show_bug.cgi?id=463530 Red Hat bugzilla 463530]] tracks efforts to incorporate this upstream work into RHEL 6. |
| - | + | == Future Work == | |
| - | + | * harden libtirpc | |
| + | * support native IPv6 (no IPv4 present at all) | ||
| + | * merge librpcsecgss into libtirpc | ||
| - | + | === Corner Cases === | |
| - | + | * how to deal with netids in the kernel (no way to invoke getnetconfig) | |
| + | * handle same client mounting same server:/export with both protocol families | ||
| - | + | == Rough Test Plan == | |
| - | + | NetApp, Oracle/Sun, EMC, and Red Hat meet during NFS Connectathon and bake-a-thon events to perform IPV6 interoperability testing between prototype NetApp filers and systems running Linux and Solaris. So far we have focused on running the Connectathon suite over IPv6, with additional specific tests for NSM lock recovery and NFSv4 delegation callback. We have also spent some effort to make sure that version and transport protocol fallback work for NFSv2/v3 mounts. | |
| - | + | See also: [http://oss.sgi.com/projects/nfs/testtools/ SGI NFS Test Tools] | |
| - | + | == Presentations == | |
| - | + | * [http://nfsv4bat.org/Documents/ConnectAThon/2010/ipv6.pdf Connectathon 2010] | |
Latest revision as of 14:20, 21 August 2012
Contents |
NFS on IPv6 Working Document
This article contains a rough implementation roadmap for IPv6 support in the Linux NFS client and server.
Project Goals
This effort will provide support in the Linux NFS client and server implementations for IPv6 networking. This includes all current versions of the NFS protocol (2, 3 and 4) and any supported auxiliary protocols (MNT, NLM, NSM, NFS_ACL, NFSv4 callback, rpcbind). User space support (idmapd, gssd, svcgssd, statd, sm-notify, mount/umount/showmount, mountd, exportfs) is also included.
In order to provide an interoperating implementation, kernel and user space RPC implementations will also be upgraded to support Transport-Independent RPC (TI-RPC). This includes the sunrpc.ko module, the IP socket-based transport capability, the in-kernel GSSD implementation, rpcbind/portmaper, libtirpc, and rpcsecgss/gssglue/krb5.
The addition of IPv6 support should not effect the behavior or performance of bog standard IPv4 NFS installations.
For now, we assume that IPv4 will always be present. Communication between separate components on the same host will continue to use IPv4 loopback.
What Is Taking So Long?
Before support for NFS over IPv6 can be introduced, some major infrastructure overhaul has been required.
The NFS client's mount stack has been converted to using an in-kernel MNT client; mount options are now passed to the kernel via a C string instead of in a versioned binary blob. There are several advantages to using a C string here, but the main benefit for IPv6 support is that we can easily support larger addresses and additional mount options without co-ordinating a version change between the kernel and the mount command.
An RPC transport capability switch has been added to the kernel (both client and server) to enable IPv6-capable network transports. Support for TI-RPC has been introduced to the Linux user space via libtirpc and rpcbind.
The TI-RPC library presents some special challenges. First, it was implemented against the streams API, not for sockets. We took our port from the FreeBSD port, which was converted to use sockets. In addition, more contemporary versions of TI-RPC exist but are licensed under CDDL, which is not compatible with GPL. The stock Sun implementation is now about 15 years old. There are no longer any engineers at Sun with implementation experience in this area, and the documentation and specifications are incomplete, to say the least, so much of this has been done by guesswork and extensive source code archaeology. Finally, TI-RPC is close, but not perfectly API- and ABI-compatible with legacy RPC implementations such as the one in glibc.
The mount command was also challenging because the nfs-utils code base is over a decade old, but there are no unit tests or requirements documents that explain exactly how its features are supposed to work. The Linux mount command implements some features (such as probing the server to see what transport protocols and NFS versions are supported) that are difficult to implement using stock TI-RPC library calls. At least two complete re-implementations of the RPCB_GETADDR/PMAP_GETPORT RPC calls have to be implemented to support these legacy features, as TI-RPC hides much of this complexity.
Lack of adequate documentation of legacy feature requirements has been a major stumbling block. The man pages are either not complete, or are not specific about certain behaviors. There is little or no documentation of specific use cases for which these features are designed. In addition, it has been difficult to locate general IPv6 expertise.
As you can tell, these pre-requisites are already a substantial amount of development effort.
Work To Date
As of January, 2010, the following items have been completed upstream:
- implementation of a new transport capability switch in the kernel's RPC client and server implementation
- support for additional address families in the kernel's RPC client and server implementation and server-side authentication cache
- support for IPv6 networking in the IP socket transport in the kernel's RPC client and server implementation
- support for IPv6 and rpcbind protocol version 4 (port-only) in the kernel's rpcbind client implementation
- NFS mount command split out of the stock Linux mount command into a subcommand
- conversion of the NFS mount operation to send mount options via a C string instead of a binary blob
- support for port-only rpcbind v4 GETADDR requests in the mount.nfs command
- support for AF_INET6 family addresses in the NFS client and server implementation
- support for IPv6 server address parsing in the NFS client
- support for IPv6 in the client's NFSv4 callback service
- replacement of Linux's user space portmapper implementation with a port of rpcbind
- introduction of a port of libtirpc to user space
- support for IPv6 in the Linux kernel's lockd and NSM implementation
- support for IPv6 and netids in the mount.nfs command
- support for IPv6 in gssd
- support for IPv6 in rpc.nfsd
- support for IPv6 and TI-RPC for NSM reboot notification (sm-notify command and rpc.statd daemon)
- support for IPv6 in the kernel's NFS server
- TCP wrapper support for IPv6
- support specifying a netid instead of a protocol name via a mount option
- support for IPv6 and TI-RPC for server-side MNT protocol (rpc.mountd and exportfs)
- support IPv6 in Linux pNFS client and server
- support for IPv6 in Linux port of Kerberos
- support for IPv6 in svcgssd
- replace legacy RPC implementation in glibc with libtirpc
- support in lockd for multiple caller_names
- implement AF_UNIX transport in kernel RPC client
Now In Prototype
Remaining To Do
- implement AF_UNIX transport in kernel RPC server
- full documentation for using IPv6 with the NFS client and server
- audit kernel RPCSEC GSSAPI implementation for IPv6 requirements
- audit rpc.idmapd implementation for IPv6 requirements (LDAP)
- test support of IPv6 addresses during NFSv4 migration events
- complete audit of work to date with eye towards eliminating code duplication
- full support for link-local IPv6 addresses
- check Linux implementation against issues in draft-alexrn-nfsv4-ipv6-00
[Red Hat bugzilla 463530] tracks efforts to incorporate this upstream work into RHEL 6.
Future Work
- harden libtirpc
- support native IPv6 (no IPv4 present at all)
- merge librpcsecgss into libtirpc
Corner Cases
- how to deal with netids in the kernel (no way to invoke getnetconfig)
- handle same client mounting same server:/export with both protocol families
Rough Test Plan
NetApp, Oracle/Sun, EMC, and Red Hat meet during NFS Connectathon and bake-a-thon events to perform IPV6 interoperability testing between prototype NetApp filers and systems running Linux and Solaris. So far we have focused on running the Connectathon suite over IPv6, with additional specific tests for NSM lock recovery and NFSv4 delegation callback. We have also spent some effort to make sure that version and transport protocol fallback work for NFSv2/v3 mounts.
See also: SGI NFS Test Tools
