Hi All,

I did an analysis of the use of API Lock in the develop branch. I
think right now the situation is that using it has made the API safe,
but it is also not optimal for performance.  Because API Lock is used
in csoundPerformKsmps, there's a potential delay in response to
calling API methods that is proportional to the value of ksmps and
amount of work done per sample.

Two things that lead me to this:

1. In Blue, using its Virtual Keyboard, I saw that when I played it
and dragged the mouse it would be sluggish when using ksmps = 64, but
near instantaneous when ksmps = 1.

2. Using Csound in server mode and live coding from Vim, I saw that
changing ksmps changed the response of code evaluations.  (Some
evaluations would sometimes get very far delayed by many seconds when
ksmps > 1.)

My thought is that the API should be redone to remove the lock and
instead use messages and an atomic message queue.  A ring buffer with
pre-allocated messages (aka, the Disruptor pattern:
https://www.slideshare.net/trishagee/introduction-to-the-disruptor)
would, I think be good for dealing with this.  (At least, I did an
implementation in my system Pink recently and it's been very good:
zero allocations at runtime, lock-free, wait-free).

The way it works is externally, API users call API methods which in
turn queue up actions as messages.  There's a process of checking out
a message, modifying the message, then pushing it back onto the queue.
Internally, in the perfKsmps, Csound would check out the current read
index, read and process all messages from the ring buffer, then update
the read index for the next pass.

One change this would introduce is a maximum number of incoming
messages.  It's a tradeoff so that we can get the other features (zero
allocations at runtime, lock-free, wait-free). Because it'd introduce
a kind of behavior change for the API, I think it might be more
appropriate for CS7 at this point.

Thoughts?

Thanks!

A queue with a lock is actually not so bad. The
include/csound_threaded.hpp file is a self-contained example of this.
It could also be rewritten to use a lock-free queue of course (my
original version did use a lock-free queue, but I wanted this to use
only standard C++). The pattern could of course be extended to all API
calls. It should be easy to duplicate all this logic inside the Csound
"C" API. In fact you could implement it in C++11 which would save some
development trouble.

Regards,
Mike

-----------------------------------------------------
Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com


On Tue, Oct 10, 2017 at 3:54 PM, Stephen Kyne  wrote:
> Yeah good idea. There shouldn't be any locking/system calls on the real-time
> thread if possible. I'm not sure if you have to use something complicated
> like distruptor even. Maybe just a simple lock-free queue. There are a
> number of implementations on github.
>
> Stephen
> ________________________________
> From: Csound-developers  on behalf of Steven
> Yi 
> Sent: 10 October 2017 20:30
> To: CSOUND-DEV@LISTSERV.HEANET.IE
> Subject: [Csnd-dev] API Lock
>
> Hi All,
>
> I did an analysis of the use of API Lock in the develop branch. I
> think right now the situation is that using it has made the API safe,
> but it is also not optimal for performance.  Because API Lock is used
> in csoundPerformKsmps, there's a potential delay in response to
> calling API methods that is proportional to the value of ksmps and
> amount of work done per sample.
>
> Two things that lead me to this:
>
> 1. In Blue, using its Virtual Keyboard, I saw that when I played it
> and dragged the mouse it would be sluggish when using ksmps = 64, but
> near instantaneous when ksmps = 1.
>
> 2. Using Csound in server mode and live coding from Vim, I saw that
> changing ksmps changed the response of code evaluations.  (Some
> evaluations would sometimes get very far delayed by many seconds when
> ksmps > 1.)
>
> My thought is that the API should be redone to remove the lock and
> instead use messages and an atomic message queue.  A ring buffer with
> pre-allocated messages (aka, the Disruptor pattern:
> https://www.slideshare.net/trishagee/introduction-to-the-disruptor)
> would, I think be good for dealing with this.  (At least, I did an
> implementation in my system Pink recently and it's been very good:
> zero allocations at runtime, lock-free, wait-free).
>
> The way it works is externally, API users call API methods which in
> turn queue up actions as messages.  There's a process of checking out
> a message, modifying the message, then pushing it back onto the queue.
> Internally, in the perfKsmps, Csound would check out the current read
> index, read and process all messages from the ring buffer, then update
> the read index for the next pass.
>
> One change this would introduce is a maximum number of incoming
> messages.  It's a tradeoff so that we can get the other features (zero
> allocations at runtime, lock-free, wait-free). Because it'd introduce
> a kind of behavior change for the API, I think it might be more
> appropriate for CS7 at this point.
>
> Thoughts?
>
> Thanks!

Sounds good to me.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

> On 10 Oct 2017, at 20:31, Steven Yi  wrote:
> 
> Hi All,
> 
> I did an analysis of the use of API Lock in the develop branch. I
> think right now the situation is that using it has made the API safe,
> but it is also not optimal for performance.  Because API Lock is used
> in csoundPerformKsmps, there's a potential delay in response to
> calling API methods that is proportional to the value of ksmps and
> amount of work done per sample.
> 
> Two things that lead me to this:
> 
> 1. In Blue, using its Virtual Keyboard, I saw that when I played it
> and dragged the mouse it would be sluggish when using ksmps = 64, but
> near instantaneous when ksmps = 1.
> 
> 2. Using Csound in server mode and live coding from Vim, I saw that
> changing ksmps changed the response of code evaluations.  (Some
> evaluations would sometimes get very far delayed by many seconds when
> ksmps > 1.)
> 
> My thought is that the API should be redone to remove the lock and
> instead use messages and an atomic message queue.  A ring buffer with
> pre-allocated messages (aka, the Disruptor pattern:
> https://www.slideshare.net/trishagee/introduction-to-the-disruptor)
> would, I think be good for dealing with this.  (At least, I did an
> implementation in my system Pink recently and it's been very good:
> zero allocations at runtime, lock-free, wait-free).
> 
> The way it works is externally, API users call API methods which in
> turn queue up actions as messages.  There's a process of checking out
> a message, modifying the message, then pushing it back onto the queue.
> Internally, in the perfKsmps, Csound would check out the current read
> index, read and process all messages from the ring buffer, then update
> the read index for the next pass.
> 
> One change this would introduce is a maximum number of incoming
> messages.  It's a tradeoff so that we can get the other features (zero
> allocations at runtime, lock-free, wait-free). Because it'd introduce
> a kind of behavior change for the API, I think it might be more
> appropriate for CS7 at this point.
> 
> Thoughts?
> 
> Thanks!

Sounds like we're on the same page so far, thanks all!  I've filed an
issue to track this for CS7:

https://github.com/csound/csound/issues/859



On Tue, Oct 10, 2017 at 4:27 PM, Victor Lazzarini
 wrote:
> Sounds good to me.
>
> Victor Lazzarini
> Dean of Arts, Celtic Studies, and Philosophy
> Maynooth University
> Ireland
>
>> On 10 Oct 2017, at 20:31, Steven Yi  wrote:
>>
>> Hi All,
>>
>> I did an analysis of the use of API Lock in the develop branch. I
>> think right now the situation is that using it has made the API safe,
>> but it is also not optimal for performance.  Because API Lock is used
>> in csoundPerformKsmps, there's a potential delay in response to
>> calling API methods that is proportional to the value of ksmps and
>> amount of work done per sample.
>>
>> Two things that lead me to this:
>>
>> 1. In Blue, using its Virtual Keyboard, I saw that when I played it
>> and dragged the mouse it would be sluggish when using ksmps = 64, but
>> near instantaneous when ksmps = 1.
>>
>> 2. Using Csound in server mode and live coding from Vim, I saw that
>> changing ksmps changed the response of code evaluations.  (Some
>> evaluations would sometimes get very far delayed by many seconds when
>> ksmps > 1.)
>>
>> My thought is that the API should be redone to remove the lock and
>> instead use messages and an atomic message queue.  A ring buffer with
>> pre-allocated messages (aka, the Disruptor pattern:
>> https://www.slideshare.net/trishagee/introduction-to-the-disruptor)
>> would, I think be good for dealing with this.  (At least, I did an
>> implementation in my system Pink recently and it's been very good:
>> zero allocations at runtime, lock-free, wait-free).
>>
>> The way it works is externally, API users call API methods which in
>> turn queue up actions as messages.  There's a process of checking out
>> a message, modifying the message, then pushing it back onto the queue.
>> Internally, in the perfKsmps, Csound would check out the current read
>> index, read and process all messages from the ring buffer, then update
>> the read index for the next pass.
>>
>> One change this would introduce is a maximum number of incoming
>> messages.  It's a tradeoff so that we can get the other features (zero
>> allocations at runtime, lock-free, wait-free). Because it'd introduce
>> a kind of behavior change for the API, I think it might be more
>> appropriate for CS7 at this point.
>>
>> Thoughts?
>>
>> Thanks!

Hi Andres,

Always good to hear from you. :)  I took a look at the
circularbuffer.c file and do wonder if the atomic usage is maybe off,
but it's good to look at this code.  Perhaps it might be possible to
share the same data structure between the two areas of code.

All best!
steven

On Wed, Oct 11, 2017 at 12:31 AM, Andres Cabrera  wrote:
> The debugger uses a system like this for messaging. It's messaging needs are
> simple, so the included lock free ring buffer was enough.
>
> Cheers,
> Andrés
>
> On Tue, Oct 10, 2017 at 2:24 PM, Steven Yi  wrote:
>>
>> Sounds like we're on the same page so far, thanks all!  I've filed an
>> issue to track this for CS7:
>>
>> https://github.com/csound/csound/issues/859
>>
>>
>>
>> On Tue, Oct 10, 2017 at 4:27 PM, Victor Lazzarini
>>  wrote:
>> > Sounds good to me.
>> >
>> > Victor Lazzarini
>> > Dean of Arts, Celtic Studies, and Philosophy
>> > Maynooth University
>> > Ireland
>> >
>> >> On 10 Oct 2017, at 20:31, Steven Yi  wrote:
>> >>
>> >> Hi All,
>> >>
>> >> I did an analysis of the use of API Lock in the develop branch. I
>> >> think right now the situation is that using it has made the API safe,
>> >> but it is also not optimal for performance.  Because API Lock is used
>> >> in csoundPerformKsmps, there's a potential delay in response to
>> >> calling API methods that is proportional to the value of ksmps and
>> >> amount of work done per sample.
>> >>
>> >> Two things that lead me to this:
>> >>
>> >> 1. In Blue, using its Virtual Keyboard, I saw that when I played it
>> >> and dragged the mouse it would be sluggish when using ksmps = 64, but
>> >> near instantaneous when ksmps = 1.
>> >>
>> >> 2. Using Csound in server mode and live coding from Vim, I saw that
>> >> changing ksmps changed the response of code evaluations.  (Some
>> >> evaluations would sometimes get very far delayed by many seconds when
>> >> ksmps > 1.)
>> >>
>> >> My thought is that the API should be redone to remove the lock and
>> >> instead use messages and an atomic message queue.  A ring buffer with
>> >> pre-allocated messages (aka, the Disruptor pattern:
>> >> https://www.slideshare.net/trishagee/introduction-to-the-disruptor)
>> >> would, I think be good for dealing with this.  (At least, I did an
>> >> implementation in my system Pink recently and it's been very good:
>> >> zero allocations at runtime, lock-free, wait-free).
>> >>
>> >> The way it works is externally, API users call API methods which in
>> >> turn queue up actions as messages.  There's a process of checking out
>> >> a message, modifying the message, then pushing it back onto the queue.
>> >> Internally, in the perfKsmps, Csound would check out the current read
>> >> index, read and process all messages from the ring buffer, then update
>> >> the read index for the next pass.
>> >>
>> >> One change this would introduce is a maximum number of incoming
>> >> messages.  It's a tradeoff so that we can get the other features (zero
>> >> allocations at runtime, lock-free, wait-free). Because it'd introduce
>> >> a kind of behavior change for the API, I think it might be more
>> >> appropriate for CS7 at this point.
>> >>
>> >> Thoughts?
>> >>
>> >> Thanks!
>> >> steven
>