server side bind mounts and high client concurrency

[Greg Banks]

Dennis J. wrote:
> Hi,
> I'm currently looking at NFSv3 mounting a 30TB Storage device on a couple 
> of servers in a rework of an old infrastructure. In the old one 182 
> directories got exported to 30 servers in a 
> "<country>/<first-letter-of-dir-name>" scheme. The letters are distributed 
> across several HD's so which is why each letter has to be mounted 
> separately. I'm wondering if I can somehow reduce that number by creating a 
> directory with all the letters for a country and bind-mounting the letters 
> on the server from their physical locations on those directories and then 
> just mount that directory for the country on the clients. Would that have 
> any negative implications compared to the old direct solution?
>   

Bind mounts are expected to work.  You may want to read up on the
"nohide" (aka crossmnt) export option.
> Also I wonder if someone has any tips for parameters in a high concurrency 
> scenario. The client system will request mostly small files (95% of the 
> files are about 8kb in size) but they will request a lot of them in 
> parallel. What are the best options here for client and server 
> configuration? Would udp or tcp be better suited for such a scenario and 
>   

Don't use UDP.  The only scenario for which UDP is useful is one where
you don't like your data anymore.  Also, if your server and workload are
large enough the single UDP server socket will be a throughput bottleneck.

> what is a realistic number for the number of nfsd processes to keep 
> running? The default is 8 but how high can I set this number realistically 
> without hitting other bottlenecks?
>
>
>   

There's a hardcoded maximum of 8192.  You can't get there because the
NFS server will fail catastrophically with more than 1020.  In practice,
with current code 128 is likely to be more than enough; in fact you will
probably only need about 32 to 64.  With current code, having more nfsds
than you need is likely to result in a slowdown, as is having too few,
and predicting exactly how many you need is difficult.  Your best bet is
to start with 8, run your workload, and keep increasing the number of
nfsds and re-running your workload until your workload doesn't get any
faster.

-- 
Greg Banks, P.Engineer, SGI Australian Software Group.
the brightly coloured sporks of revolution.
I don't speak for SGI.