[pnfs] [PATCH] preallocate a backchannel request in xs_tcp_data_ready

[Benny Halevy]

On Nov. 29, 2007, 16:29 +0200, Trond Myklebust <trond.myklebust at fys.uio.no> wrote:
> On Thu, 2007-11-29 at 16:14 +0200, Benny Halevy wrote:
>> On Nov. 29, 2007, 15:35 +0200, Trond Myklebust <trond.myklebust at fys.uio.no> wrote:
>>> On Thu, 2007-11-29 at 09:20 +0200, Benny Halevy wrote:
>>>> On Nov. 29, 2007, 7:06 +0200, Trond Myklebust <trond.myklebust at fys.uio.no> wrote:
>>>>> On Wed, 2007-11-28 at 18:59 -0800, Ricardo Labiaga wrote:
>>>>>> For my own education, what happens if xs_tcp_data_ready() can not
>>>>>> allocate the back channel request and does not read the data off the
>>>>>> socket?  Will xs_tcp_data_ready() get called again because the data is
>>>>>> still there?
>>>>> The remaining data would be handled by xs_tcp_read_discard(), and would
>>>>> therefore just be junked.
>>>> But that happens only via xs_tcp_data_recv which is not called
>>>> in this case.  I count on the tcp socket layer to call data_ready again
>>>> whenever new data is coming in, however I am still worried about stalling
>>>> if the receive window will fill-up after we've hit this out-of-memory
>>>> condition since the back pressure will prevent any new data from coming
>>>> in.  The solution to that could be to somehow kick start data_ready if needed
>>>> at xprt_free_bc_request time.
>>> I don't understand. xs_tcp_data_recv() is the only caller of
>>> xs_tcp_data_read_request(). Normally the loop there will ensure that
>>> xs_tcp_read_discard() cleans up any data that was not read by the other
>>> xs_tcp_read*() routines. Why wouldn't it do so in this case?
>> The idea is to *not* call xs_tcp_data_recv from xs_tcp_data_ready()
>> if there is no req available for processing the callback.
> 
> That would be a totally unacceptable slowdown on the forward channel,
> and is quite unnecessary. You have a session and you have given the
> server a set number of slots for the back channel. You therefore _know_
> how much space you need to preallocate in order to be able to service
> all requests that will be arriving on the back channel.

You're right and it'd be nice if, in addition to the per-slot resources,
we'll have per xprt memory to process any callback request up to
the sessions slot table layer so if all slots are busy we can
return the most appropriate error.

Benny

> 
> Trond
>