Increasing the server wsize/rsize

[Greg Banks]

On Thu, 2006-08-03 at 19:12, Neil Brown wrote:
> On Thursday August 3, gnb at melbourne.sgi.com wrote:
> > On Thu, 2006-08-03 at 17:23, Neil Brown wrote:
> > > On Wednesday August 2, bfields at fieldses.org wrote:
> > > > On Fri, Jul 28, 2006 at 07:34:43PM -0400, Dean Hildebrand wrote:
> > > > > Thanks for the updates Greg.  One issue I've found is that if we want to 
> > > > > go larger than 1MB, we still need to fix svc_tcp_recvfrom to use the 
> > > > > heap instead of the stack.
> > > > 
> > > > Even with only 1MB, we're still putting a 1k array on the stack in that
> > > > one function, which seems like a problem.
> > > > 
> > > 
> > > But do we really need to scale the 'vec' array at all?
> > > Why not just call svc_recvfrom multiple times.
> > 
> > svc_recvfrom() calls kernel_recvmsg() which traverses LSM security
> > hooks, futzes with the socket lock, and potentially sends an ACK
> > to the peer.  All of these are things you want to minimise doing.
> 
> True, but we also want to minimise the size of the 'vec' array (or
> allocating is part of struct svc_rqst).
> So where is an appropriate trade-off?  My '16' that I picked out of
> the air means one call per 64K.  Is that too often?

Hard to say without some measurement.

One thing I'd be worried about is the on-the-wire impact of giving
TCP more opportunities to send an ACK to the client.  It might be
positive...or not.

> We don't start calling svc_recvfrom until we know there are enough
> bytes available.  So these newly arriving packets have nothing much to
> do with the current rpc message.

Agreed.

> And a stray call to data_ready just calls svc_sock_enqueue which will
> quickly exit before SK_BUSY is set - no room for havoc there.

Yes, it looks like SK_BUSY and SK_DATA should handle this case.

> So you've raised some issues worth considering, but I'm not sure what
> conclusion they really point to...

That some experiment and measurement is necessary.

Greg.
-- 
Greg Banks, R&D Software Engineer, SGI Australian Software Group.
I don't speak for SGI.