[pnfs] Multiple tcp streams per mount point NFS patch

[Benny Halevy]

On Mar. 14, 2008, 20:54 +0200, "William A. (Andy) Adamson" <andros at citi.umich.edu> wrote:
> I've ported (and tested) the experimental multiple rpc_xprt per rpc_clnt
> patch forward to the latest pnfs kernel (2.6.24-pnfs). it's one large patch.
> 
> troy.citi.umich.edu:/vol/home/andros/pnfs-file-ibm/multiple_tcp/2.6.24-pnfs-client-multiple-rpc_xprt-per-rpc_clnt.patch
> .
> 
> pnfs: is there interest in this experimental patch?   should we have a

I think there might be interest in CMU. In particular, with using a NFS-RDMA
transport.  Given that NFS-RDMA is in 2.6.25 and taking into account the
related changes there, I think we better aim at 2.6.25 rather than 2.6.24.

> 'performance testing' branch of the pnfs git tree where experimental code
> such as this patch can be kept up to date?

I'm inclined to push back a little for features that are not targeted to
be pushed upstream any time soon, yet the benefits of keeping this code
up to speed may be significant enough to justify having it out in the main
development tree.  I anticipate that help will be needed to do the actual
porting work if anything non-trivial comes up while rebasing (e.g. porting
to 2.6.25).

Benny

> 
> the new feature is controlled by a new sysctl, which needs to be set while
> the server is not mounted.
> 
> TEN% sysctl -a | grep sunrpc
> sunrpc.rpc_debug = 0
> sunrpc.nfs_debug = 0
> sunrpc.nfsd_debug = 0
> sunrpc.nlm_debug = 0
> sunrpc.udp_slot_table_entries = 16
> sunrpc.tcp_slot_table_entries = 16
> sunrpc.min_resvport = 665
> sunrpc.max_resvport = 1023
> sunrpc.tcp_xprt_table_entries = 2
> 
> sunrpc.tcp_xprt_table_entries has a default value of 1. increasing it's
> value increases the number of tcp streams from the client to all mounted
> servers.
> 
> tcp_slot_table_entries are allocated in a round robin fashion, with a wait
> queue if no slots are available.  the new code chooses the next rpc_xprt
> table, which represents a tcp connection from a unique port, at the same
> time. e.g. once a slot is available, the same code chooses the next
> rpc_xprt.
> 
> so the window size is now  the number of
> sunrpc.tcp_slot_table_entriesdivided by the number of
> sunrpc.tcp_xprt_table_entries.
> 
> in the above sysctl output, there are 8
> sunrpc.tcp_slot_table_entriesavailable to each tcp connection.
> 
> this gives the NFS client (v3/4/4.1) the same knob that FDT (
> http://monalisa.cern.ch/FDT/    ) has on the client side, to manipulate the
> number of tcp connections to the server.
> 
> -->Andy
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> pNFS mailing list
> pNFS at linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs