Ipv6PlanningDocument

From Linux NFS

Revision as of 07:54, 10 March 2009

NFS on IPv6 Working Document

This article contains a rough implementation roadmap for IPv6 support in the Linux NFS client and server.

Project Goals

This effort will provide support in the Linux NFS client and server implementations for IPv6 networking. This includes all current versions of the NFS protocol (2, 3 and 4) and any supported auxiliary protocols (MNT, NLM, NSM, NFS_ACL, NFSv4 callback, rpcbind). User space support (idmapd, gssd, svcgssd, statd, sm-notify, mount/umount/showmount, mountd, exportfs) is also included.

In order to provide an interoperating implementation, kernel and user space RPC implementations will also be upgraded to support Transport-Independent RPC (TI-RPC). This includes the sunrpc.ko module, the IP socket-based transport capability, the in-kernel GSSD implementation, rpcbind/portmaper, libtirpc, and rpcsecgss/gssglue/krb5.

The addition of IPv6 support should not effect the behavior or performance of bog standard IPv4 NFS installations.

What Is Taking So Long?

Before support for NFS over IPv6 can be introduced, some major infrastructure overhaul has been required.

The NFS client's mount stack has been converted to using an in-kernel MNT client; mount options are now passed to the kernel via a C string instead of in a versioned binary blob. There are several advantages to using a C string here, but the main benefit for IPv6 support is that we can easily support larger addresses and additional mount options without co-ordinating a version change between the kernel and the mount command.

An RPC transport capability switch has been added to the kernel (both client and server) to enable IPv6-capable network transports. Support for TI-RPC has been introduced to the Linux user space via libtirpc and rpcbind.

The TI-RPC library presents some special challenges. First, it was implemented against the streams API, not for sockets. We took our port from the FreeBSD port, which was converted to use sockets. In addition, more contemporary versions of TI-RPC exist but are licensed under CDDL, which is not compatible with GPL. The stock Sun implementation is now about 15 years old. There are no longer any engineers at Sun with implementation experience in this area, and the documentation and specifications are incomplete, to say the least, so much of this has been done by guesswork and extensive source code archaeology. Finally, TI-RPC is close, but not perfectly API- and ABI-compatible with legacy RPC implementations such as the one in glibc.

The mount command was also challenging because the nfs-utils code base is over a decade old, but there are no unit tests or requirements documents that explain exactly how its features are supposed to work. The Linux mount command implements some features (such as probing the server to see what transport protocols and NFS versions are supported) that are difficult to implement using stock TI-RPC library calls. At least two complete re-implementations of the RPCB_GETADDR/PMAP_GETPORT RPC calls have to be implemented to support these legacy features, as TI-RPC hides much of this complexity.

Lack of adequate documentation of legacy feature requirements has been a major stumbling block. The man pages are either not complete, or are not specific about certain behaviors. There is little or no documentation of specific use cases for which these features are designed. In addition, it has been difficult to locate general IPv6 expertise.

As you can tell, these pre-requisites are already a substantial amount of development effort.

Work To Date

As of March, 2009, the following items have been completed:

• implementation of a new transport capability switch in the kernel's RPC client and server implementation
• support for additional address families in the kernel's RPC client and server implementation and server-side authentication cache
• support for IPv6 networking in the IP socket transport in the kernel's RPC client and server implementation
• support for IPv6 and rpcbind protocol version 4 in the kernel's rpcbind client implementation
• NFS mount command split out of the stock Linux mount command into a subcommand
• conversion of the NFS mount operation to send mount options via a C string instead of a binary blob
• support for rpcbind v4 GETADDR requests in the mount.nfs command
• support for AF_INET6 family addresses in the NFS client and server implementation
• support for IPv6 server address parsing in the NFS client
• support for IPv6 in the client's NFSv4 callback service
• replacement of Linux's portmapper implementation with a port of rpcbind
• introduction of a port of libtirpc to user space
• support for IPv6 in the Linux kernel's lockd and NSM implementation
• support for IPv6 and netids in the mount.nfs command

Now In Prototype

• support for IPv6 and TI-RPC for NSM reboot notification (sm-notify command and rpc.statd daemon)
• support for IPv6 server address parsing in the NFS server
• support for IPv6 in the kernel's NFS server

Remaining To Do

• support for IPv6 and TI-RPC for server-side MNT protocol (rpc.mountd and exportfs)
• TCP wrapper support for IPv6
• full documentation for using IPv6 with the NFS client and server
• support for IPv6 in Linux port of Kerberos
• support for IPv6 in gssd and svcgssd
• harden libtirpc
• merge librpcsecgss into libtirpc
• replace legacy RPC implementation in glibc with libtirpc
• audit kernel RPCSEC GSSAPI implementation for IPv6 requirements
• audit rpc.idmapd implementation for IPv6 requirements
• support IPv6 addresses during NFSv4 migration events
• complete review of work to date with eye towards eliminating code duplication

Possible Projects

• reimplement libtirpc, rpcbind, and rpcgen from scratch
• remove some or all DNS dependence from statd
• support specifying a netid instead of a protocol name via a mount option
• migrate statd and sm-notify to use xlog() instead of a local syslog implementation
• use "" for netid during kernel RPCB_GETADDR calls
• implement rpcbind in the kernel

Corner Cases

• full support for link-local addresses (universal addresses, used by NFSv4 and rpcbind v4, cannot express scope IDs)
• how to deal with netids in the kernel (no way to invoke getnetconfig)

Rough Test Plan

NetApp, Oracle and now Red Hat meet during NFS Connectathon and bake-a-thon events to perform IPV6 interoperability testing between prototype NetApp filers and systems running Linux and Solaris. So far we have focused on running the Connectathon suite over IPv6, with additional specific tests for NSM lock recovery and NFSv4 delegation callback. We have also spent some effort to make sure that version and transport protocol fallback work for NFSv2/v3 mounts.

We are moving towards gssd and idmapd support, and identifying problems with the newly ported libtirpc and rpcbind implemenations.