Hi folks, I'm planning a pretty big project that I hope to be able to
use in live gigging situations and want to use the Python API for it. I
am thinking that I could write some sphinx docs with the help of others
( ie I pester the collective you for answers and write the docs ) if I
can make the time. Before deciding whether this is realistic though, I'd
like to get a good idea of whether I will even be able to do what I want
using only the python api. Some questions to those who have used it a
lot.

- If you seperate your audio rendering thread from your gui/controller
thread, have you been able to achieve low latency under high load using
only python?

- Did you need to do any GIL avoidance hackery?

- Has anyone tried doing the audio thread in Cython to release the GIL
while audio renders?

- What situations have not been possible with Python only?

thanks
Iain



Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

On Wed, 2009-10-21 at 11:35 -0700, Iain Duncan wrote:
> Hi folks, I'm planning a pretty big project that I hope to be able to
> use in live gigging situations and want to use the Python API for it. I
> am thinking that I could write some sphinx docs with the help of others
> ( ie I pester the collective you for answers and write the docs ) if I
> can make the time. Before deciding whether this is realistic though, I'd
> like to get a good idea of whether I will even be able to do what I want
> using only the python api. Some questions to those who have used it a
> lot.
> 
> - If you seperate your audio rendering thread from your gui/controller
> thread, have you been able to achieve low latency under high load using
> only python?
> 
> - Did you need to do any GIL avoidance hackery?
> 
> - Has anyone tried doing the audio thread in Cython to release the GIL
> while audio renders?
> 
> - What situations have not been possible with Python only?

Also, anyone tried using the python api with a host app that uses Jack
for audio input and output? Any success?

Thanks
Iain




Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

On 21 Oct 2009, at 19:35, Iain Duncan wrote:

> Hi folks, I'm planning a pretty big project that I hope to be able to
> use in live gigging situations and want to use the Python API for  
> it. I
> am thinking that I could write some sphinx docs with the help of  
> others
> ( ie I pester the collective you for answers and write the docs ) if I
> can make the time. Before deciding whether this is realistic though,  
> I'd
> like to get a good idea of whether I will even be able to do what I  
> want
> using only the python api. Some questions to those who have used it a
> lot.
>
> - If you seperate your audio rendering thread from your gui/controller
> thread, have you been able to achieve low latency under high load  
> using
> only python?

Because Python just wraps  C/C++ code, there is no performance penalty.
If you start moving audio data into Python numerical objects and  
manipulate
it using Python code, then things can slow down.

> - Did you need to do any GIL avoidance hackery?

No, just use CsoundPerformanceThread objects and there are no GIL  
issues.

> - Has anyone tried doing the audio thread in Cython to release the GIL
> while audio renders?

I don't think there is any need for that.
GIL issues have been addressed in the wrapper interface when they arise
(mostly for enabling Python-coded callbacks)

> - What situations have not been possible with Python only?

Most things are possible now; I can't think of any issues.

Two guys you should talk to in this list: Oeyvind Brandtsegg and Chuck  
Hubbard,
they have fully-working systems written in Python using the API.

> thanks
> Iain
>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body  
> "unsubscribe csound"



Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Since Csound has a Jack IO module, you can just select it.

Victor
On 21 Oct 2009, at 19:36, Iain Duncan wrote:

> On Wed, 2009-10-21 at 11:35 -0700, Iain Duncan wrote:
>> Hi folks, I'm planning a pretty big project that I hope to be able to
>> use in live gigging situations and want to use the Python API for  
>> it. I
>> am thinking that I could write some sphinx docs with the help of  
>> others
>> ( ie I pester the collective you for answers and write the docs )  
>> if I
>> can make the time. Before deciding whether this is realistic  
>> though, I'd
>> like to get a good idea of whether I will even be able to do what I  
>> want
>> using only the python api. Some questions to those who have used it a
>> lot.
>>
>> - If you seperate your audio rendering thread from your gui/ 
>> controller
>> thread, have you been able to achieve low latency under high load  
>> using
>> only python?
>>
>> - Did you need to do any GIL avoidance hackery?
>>
>> - Has anyone tried doing the audio thread in Cython to release the  
>> GIL
>> while audio renders?
>>
>> - What situations have not been possible with Python only?
>
> Also, anyone tried using the python api with a host app that uses Jack
> for audio input and output? Any success?
>
> Thanks
> Iain
>
>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body  
> "unsubscribe csound"



Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

I have been able to use the Python API as is to create programs with
user interfaces for interactive performances in real time. Both of
these programs are in the Csound examples directory (Orbifold.py and
drone.py).

I would not say I was terribly satisfied with the behavior of these
applications, but I'm not sure I spent enough time on them to iron out
all accessible wrinkles. The main problem was dropouts that occurred
intermittently during manipulation of the GUI. Dropouts were not
constant, and that gives hope that the problem is surmountable. In
other words, I could play with the things without dropouts at times.

The Orbifold.py program uses CsoundPerformanceThread, and the GUI in
this example is doing a LOT more work than would typically be the case
in a music application. This also is grounds for hope. Probably the
GUI is the cause of the dropouts (it uses a pretty demanding
3-dimensional real-time graphics package, Visual Python).

Note that the CsoundPerformanceThread class already does what you are
asking for in terms of threading.

The drone.py program does not use CsoundPerformanceThread (because
that doesn't seem to work with Tkinter), it computes 1 ksmps at a time
in Python. This program rewrites function tables in response to user
input, and that seems to be the cause of the dropouts; if you don't do
that, you can play without dropouts.

Bottom line: I am not aware of any underlying reason why Python
programs that are not doing a whole lot of extra work couldn't
co-exist with a smoothly running instance of Csound. Stuff that I did
for other reasons is close to that, and you could probably get it to
work.

Just to make sure, you might want to try a "proof of concept" app in
your framework of choice, that runs a fairly demanding csound
orchestra in a CsoundPerformanceThread at 20 ksmps or so, while you
fiddle with knobs and sliders, and draw pictures, and such.

Hope this helps,
Mike

On 10/21/09, Iain Duncan  wrote:
> Hi folks, I'm planning a pretty big project that I hope to be able to
> use in live gigging situations and want to use the Python API for it. I
> am thinking that I could write some sphinx docs with the help of others
> ( ie I pester the collective you for answers and write the docs ) if I
> can make the time. Before deciding whether this is realistic though, I'd
> like to get a good idea of whether I will even be able to do what I want
> using only the python api. Some questions to those who have used it a
> lot.
>
> - If you seperate your audio rendering thread from your gui/controller
> thread, have you been able to achieve low latency under high load using
> only python?
>
> - Did you need to do any GIL avoidance hackery?
>
> - Has anyone tried doing the audio thread in Cython to release the GIL
> while audio renders?
>
> - What situations have not been possible with Python only?
>
> thanks
> Iain
>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> csound"
>

On Wed, 2009-10-21 at 14:57 -0400, Michael Gogins wrote:
> I have been able to use the Python API as is to create programs with
> user interfaces for interactive performances in real time. Both of
> these programs are in the Csound examples directory (Orbifold.py and
> drone.py).
> 
> I would not say I was terribly satisfied with the behavior of these
> applications, but I'm not sure I spent enough time on them to iron out
> all accessible wrinkles. The main problem was dropouts that occurred
> intermittently during manipulation of the GUI. Dropouts were not
> constant, and that gives hope that the problem is surmountable. In
> other words, I could play with the things without dropouts at times.
> 
> The Orbifold.py program uses CsoundPerformanceThread, and the GUI in
> this example is doing a LOT more work than would typically be the case
> in a music application. This also is grounds for hope. Probably the
> GUI is the cause of the dropouts (it uses a pretty demanding
> 3-dimensional real-time graphics package, Visual Python).
> 
> Note that the CsoundPerformanceThread class already does what you are
> asking for in terms of threading.
> 
> The drone.py program does not use CsoundPerformanceThread (because
> that doesn't seem to work with Tkinter), it computes 1 ksmps at a time
> in Python. This program rewrites function tables in response to user
> input, and that seems to be the cause of the dropouts; if you don't do
> that, you can play without dropouts.
> 
> Bottom line: I am not aware of any underlying reason why Python
> programs that are not doing a whole lot of extra work couldn't
> co-exist with a smoothly running instance of Csound. Stuff that I did
> for other reasons is close to that, and you could probably get it to
> work.
> 
> Just to make sure, you might want to try a "proof of concept" app in
> your framework of choice, that runs a fairly demanding csound
> orchestra in a CsoundPerformanceThread at 20 ksmps or so, while you
> fiddle with knobs and sliders, and draw pictures, and such.

Thanks Mike. Did you try putting the gui in it's own self contained
thread and communicating with the rendering thread over a queue? I am
thinking the way to prevent the gui from causing drop outs will be to
have it pass events on a queue that the performance thread only reads X
times per ksmp, where X is user controlled to figure it out. I guess as
you say I should try a proof of concept before anything else.

Thanks
Iain

> 
> Hope this helps,
> Mike
> 
> On 10/21/09, Iain Duncan  wrote:
> > Hi folks, I'm planning a pretty big project that I hope to be able to
> > use in live gigging situations and want to use the Python API for it. I
> > am thinking that I could write some sphinx docs with the help of others
> > ( ie I pester the collective you for answers and write the docs ) if I
> > can make the time. Before deciding whether this is realistic though, I'd
> > like to get a good idea of whether I will even be able to do what I want
> > using only the python api. Some questions to those who have used it a
> > lot.
> >
> > - If you seperate your audio rendering thread from your gui/controller
> > thread, have you been able to achieve low latency under high load using
> > only python?
> >
> > - Did you need to do any GIL avoidance hackery?
> >
> > - Has anyone tried doing the audio thread in Cython to release the GIL
> > while audio renders?
> >
> > - What situations have not been possible with Python only?
> >
> > thanks
> > Iain
> >
> >
> >
> > Send bugs reports to this list.
> > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> > csound"
> >
> 
> 



Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

There are several applications that have been developed to use
the Python API and these seem to demonstrate that it works quite
well. Python+Csound has been used in the olpc laptop, for instance
(TamTam and other apps).

Regarding CsoundPerformanceThread and Tkinter, the shapes.py
example demonstrates that the two can work together. I don't get
dropouts running it (well, of course, it's a simple application).

Victor


On 21 Oct 2009, at 19:57, Michael Gogins wrote:

> I have been able to use the Python API as is to create programs with
> user interfaces for interactive performances in real time. Both of
> these programs are in the Csound examples directory (Orbifold.py and
> drone.py).
>
> I would not say I was terribly satisfied with the behavior of these
> applications, but I'm not sure I spent enough time on them to iron out
> all accessible wrinkles. The main problem was dropouts that occurred
> intermittently during manipulation of the GUI. Dropouts were not
> constant, and that gives hope that the problem is surmountable. In
> other words, I could play with the things without dropouts at times.
>
> The Orbifold.py program uses CsoundPerformanceThread, and the GUI in
> this example is doing a LOT more work than would typically be the case
> in a music application. This also is grounds for hope. Probably the
> GUI is the cause of the dropouts (it uses a pretty demanding
> 3-dimensional real-time graphics package, Visual Python).
>
> Note that the CsoundPerformanceThread class already does what you are
> asking for in terms of threading.
>
> The drone.py program does not use CsoundPerformanceThread (because
> that doesn't seem to work with Tkinter), it computes 1 ksmps at a time
> in Python. This program rewrites function tables in response to user
> input, and that seems to be the cause of the dropouts; if you don't do
> that, you can play without dropouts.
>
> Bottom line: I am not aware of any underlying reason why Python
> programs that are not doing a whole lot of extra work couldn't
> co-exist with a smoothly running instance of Csound. Stuff that I did
> for other reasons is close to that, and you could probably get it to
> work.
>
> Just to make sure, you might want to try a "proof of concept" app in
> your framework of choice, that runs a fairly demanding csound
> orchestra in a CsoundPerformanceThread at 20 ksmps or so, while you
> fiddle with knobs and sliders, and draw pictures, and such.
>
> Hope this helps,
> Mike
>
> On 10/21/09, Iain Duncan  wrote:
>> Hi folks, I'm planning a pretty big project that I hope to be able to
>> use in live gigging situations and want to use the Python API for  
>> it. I
>> am thinking that I could write some sphinx docs with the help of  
>> others
>> ( ie I pester the collective you for answers and write the docs )  
>> if I
>> can make the time. Before deciding whether this is realistic  
>> though, I'd
>> like to get a good idea of whether I will even be able to do what I  
>> want
>> using only the python api. Some questions to those who have used it a
>> lot.
>>
>> - If you seperate your audio rendering thread from your gui/ 
>> controller
>> thread, have you been able to achieve low latency under high load  
>> using
>> only python?
>>
>> - Did you need to do any GIL avoidance hackery?
>>
>> - Has anyone tried doing the audio thread in Cython to release the  
>> GIL
>> while audio renders?
>>
>> - What situations have not been possible with Python only?
>>
>> thanks
>> Iain
>>
>>
>>
>> Send bugs reports to this list.
>> To unsubscribe, send email sympa@lists.bath.ac.uk with body  
>> "unsubscribe
>> csound"
>>
>
>
> -- 
> Michael Gogins
> Irreducible Productions
> http://www.michael-gogins.com
> Michael dot Gogins at gmail dot com
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body  
> "unsubscribe csound"



Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Thanks, I'll take a look at the shapes example. Perhaps I can fix up
drone.py to work the same way.

Regards,
Mike

On 10/21/09, Victor Lazzarini  wrote:
> There are several applications that have been developed to use
> the Python API and these seem to demonstrate that it works quite
> well. Python+Csound has been used in the olpc laptop, for instance
> (TamTam and other apps).
>
> Regarding CsoundPerformanceThread and Tkinter, the shapes.py
> example demonstrates that the two can work together. I don't get
> dropouts running it (well, of course, it's a simple application).
>
> Victor
>
>
> On 21 Oct 2009, at 19:57, Michael Gogins wrote:
>
>> I have been able to use the Python API as is to create programs with
>> user interfaces for interactive performances in real time. Both of
>> these programs are in the Csound examples directory (Orbifold.py and
>> drone.py).
>>
>> I would not say I was terribly satisfied with the behavior of these
>> applications, but I'm not sure I spent enough time on them to iron out
>> all accessible wrinkles. The main problem was dropouts that occurred
>> intermittently during manipulation of the GUI. Dropouts were not
>> constant, and that gives hope that the problem is surmountable. In
>> other words, I could play with the things without dropouts at times.
>>
>> The Orbifold.py program uses CsoundPerformanceThread, and the GUI in
>> this example is doing a LOT more work than would typically be the case
>> in a music application. This also is grounds for hope. Probably the
>> GUI is the cause of the dropouts (it uses a pretty demanding
>> 3-dimensional real-time graphics package, Visual Python).
>>
>> Note that the CsoundPerformanceThread class already does what you are
>> asking for in terms of threading.
>>
>> The drone.py program does not use CsoundPerformanceThread (because
>> that doesn't seem to work with Tkinter), it computes 1 ksmps at a time
>> in Python. This program rewrites function tables in response to user
>> input, and that seems to be the cause of the dropouts; if you don't do
>> that, you can play without dropouts.
>>
>> Bottom line: I am not aware of any underlying reason why Python
>> programs that are not doing a whole lot of extra work couldn't
>> co-exist with a smoothly running instance of Csound. Stuff that I did
>> for other reasons is close to that, and you could probably get it to
>> work.
>>
>> Just to make sure, you might want to try a "proof of concept" app in
>> your framework of choice, that runs a fairly demanding csound
>> orchestra in a CsoundPerformanceThread at 20 ksmps or so, while you
>> fiddle with knobs and sliders, and draw pictures, and such.
>>
>> Hope this helps,
>> Mike
>>
>> On 10/21/09, Iain Duncan  wrote:
>>> Hi folks, I'm planning a pretty big project that I hope to be able to
>>> use in live gigging situations and want to use the Python API for
>>> it. I
>>> am thinking that I could write some sphinx docs with the help of
>>> others
>>> ( ie I pester the collective you for answers and write the docs )
>>> if I
>>> can make the time. Before deciding whether this is realistic
>>> though, I'd
>>> like to get a good idea of whether I will even be able to do what I
>>> want
>>> using only the python api. Some questions to those who have used it a
>>> lot.
>>>
>>> - If you seperate your audio rendering thread from your gui/
>>> controller
>>> thread, have you been able to achieve low latency under high load
>>> using
>>> only python?
>>>
>>> - Did you need to do any GIL avoidance hackery?
>>>
>>> - Has anyone tried doing the audio thread in Cython to release the
>>> GIL
>>> while audio renders?
>>>
>>> - What situations have not been possible with Python only?
>>>
>>> thanks
>>> Iain
>>>
>>>
>>>
>>> Send bugs reports to this list.
>>> To unsubscribe, send email sympa@lists.bath.ac.uk with body
>>> "unsubscribe
>>> csound"
>>>
>>
>>
>> --
>> Michael Gogins
>> Irreducible Productions
>> http://www.michael-gogins.com
>> Michael dot Gogins at gmail dot com
>>
>>
>> Send bugs reports to this list.
>> To unsubscribe, send email sympa@lists.bath.ac.uk with body
>> "unsubscribe csound"
>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> csound"
>

On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
...
> Bottom line: I am not aware of any underlying reason why Python
> programs that are not doing a whole lot of extra work couldn't
> co-exist with a smoothly running instance of Csound.

And don't be afraid to use Numpy for any extensive and/or recurring  
numerical matrix calculations.

D.

________________________________________________
Dr David Worrall.
- Experimental Polymedia:	  www.avatar.com.au
- Sonification: 			  sonification.com.au




Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Agreed, and psyco can also speed python up considerably.

Regarding Iains question about separating the gui:
(welcome back Iain;-))
Yes, it is definitely a good idea to try to untangle the gui
processing from the audio rendering. This can probably be done in many
different ways, the main issue being the separation (thread or process
or application), not the actual method used to acheive separation.
The way I've done it, I have two separate python applications (one for
gui, one for composition processes and running Csound) communication
via Pyro (Python Remote Objects). This works well.  The gui and the
audio application automatically can find each other over the local
network, and you generally don't worry much about it after the initial
setup. .... to the extent that the details of setting it up have been
forgotten by me now, so I'll have to do some digging to give you more
details. Or you could just look at the ImproSculpt4 code at
sourceforge (no recent releases, but the cvs is kind of up to date).

best
Oeyvind

2009/10/22 DavidW :
>
> On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
> ...
>>
>> Bottom line: I am not aware of any underlying reason why Python
>> programs that are not doing a whole lot of extra work couldn't
>> co-exist with a smoothly running instance of Csound.
>
> And don't be afraid to use Numpy for any extensive and/or recurring
> numerical matrix calculations.
>
> D.
>
> ________________________________________________
> Dr David Worrall.
> - Experimental Polymedia:         www.avatar.com.au
> - Sonification:                           sonification.com.au
>
>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> csound"
>


Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

I take it the csound api could change and therefore you probably want all the
files that your program requires in the distro that you are going to do??? 
I haven't had much luck with it anyway.


iain duncan wrote:
> 
> Hi folks, I'm planning a pretty big project that I hope to be able to
> use in live gigging situations and want to use the Python API for it. I
> am thinking that I could write some sphinx docs with the help of others
> ( ie I pester the collective you for answers and write the docs ) if I
> can make the time. Before deciding whether this is realistic though, I'd
> like to get a good idea of whether I will even be able to do what I want
> using only the python api. Some questions to those who have used it a
> lot.
> 
> - If you seperate your audio rendering thread from your gui/controller
> thread, have you been able to achieve low latency under high load using
> only python?
> 
> - Did you need to do any GIL avoidance hackery?
> 
> - Has anyone tried doing the audio thread in Cython to release the GIL
> while audio renders?
> 
> - What situations have not been possible with Python only?
> 
> thanks
> Iain
> 
> 
> 
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> csound"
> 
> 

On Thu, 2009-10-22 at 10:18 +0200, Oeyvind Brandtsegg wrote:
> Agreed, and psyco can also speed python up considerably.
> 
> Regarding Iains question about separating the gui:
> (welcome back Iain;-))
> Yes, it is definitely a good idea to try to untangle the gui
> processing from the audio rendering. This can probably be done in many
> different ways, the main issue being the separation (thread or process
> or application), not the actual method used to acheive separation.
> The way I've done it, I have two separate python applications (one for
> gui, one for composition processes and running Csound) communication
> via Pyro (Python Remote Objects). This works well.  The gui and the
> audio application automatically can find each other over the local
> network, and you generally don't worry much about it after the initial
> setup. .... to the extent that the details of setting it up have been
> forgotten by me now, so I'll have to do some digging to give you more
> details. Or you could just look at the ImproSculpt4 code at
> sourceforge (no recent releases, but the cvs is kind of up to date).

Thanks Oeyvind. Did you do this because you could not get what you
needed happening using threads within one python process? Can you
elucidate on where you hit problems on this?  Did you ever try using
Cython for the audio thread? ( I haven't much experience in it, but one
of my more experienced python colleagues swears by Cython for dealing
with thread speed issues. )

Do you have the code you've done this with up anywhere?

Iain

> 
> best
> Oeyvind
> 
> 2009/10/22 DavidW :
> >
> > On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
> > ...
> >>
> >> Bottom line: I am not aware of any underlying reason why Python
> >> programs that are not doing a whole lot of extra work couldn't
> >> co-exist with a smoothly running instance of Csound.
> >
> > And don't be afraid to use Numpy for any extensive and/or recurring
> > numerical matrix calculations.
> >
> > D.
> >
> > ________________________________________________
> > Dr David Worrall.
> > - Experimental Polymedia:         www.avatar.com.au
> > - Sonification:                           sonification.com.au
> >
> >
> >
> >
> > Send bugs reports to this list.
> > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> > csound"
> >
> 
> 
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"



Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

On Thu, 2009-10-22 at 10:18 +0200, Oeyvind Brandtsegg wrote:
> Agreed, and psyco can also speed python up considerably.
> 
> Regarding Iains question about separating the gui:
> (welcome back Iain;-))
> Yes, it is definitely a good idea to try to untangle the gui
> processing from the audio rendering. This can probably be done in many
> different ways, the main issue being the separation (thread or process
> or application), not the actual method used to acheive separation.
> The way I've done it, I have two separate python applications (one for
> gui, one for composition processes and running Csound) communication
> via Pyro (Python Remote Objects). This works well.  The gui and the
> audio application automatically can find each other over the local
> network, and you generally don't worry much about it after the initial
> setup. .... to the extent that the details of setting it up have been
> forgotten by me now, so I'll have to do some digging to give you more
> details. Or you could just look at the ImproSculpt4 code at
> sourceforge (no recent releases, but the cvs is kind of up to date).
> 

Oeyvind, would you mind sharing what kind of latency vs load you have
achieved using the above? 

Does Python Remove Objects make protected-queuing between the two
processes pretty straightforward? One of my main objectives this time
around is to keep out of the guts of c level threading/queuing/memory
management if at all possible.

Thanks
Iain

> best
> Oeyvind
> 
> 2009/10/22 DavidW :
> >
> > On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
> > ...
> >>
> >> Bottom line: I am not aware of any underlying reason why Python
> >> programs that are not doing a whole lot of extra work couldn't
> >> co-exist with a smoothly running instance of Csound.
> >
> > And don't be afraid to use Numpy for any extensive and/or recurring
> > numerical matrix calculations.
> >
> > D.
> >
> > ________________________________________________
> > Dr David Worrall.
> > - Experimental Polymedia:         www.avatar.com.au
> > - Sonification:                           sonification.com.au
> >
> >
> >
> >
> > Send bugs reports to this list.
> > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> > csound"
> >
> 
> 
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"



Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

I repeat, Cython for the thread is a red herring,
CsoundPerformanceThread already does what you are thinking about. That
is, CsoundPerformanceThread creates a separate thread in C++ code,
then returns immediately to Python while the Csound thread continues
to run. All monkey business about the GIL does not apply, as this is
not a Python thread, it is a C thread. If you don't believe me, look
at the code yourself.

I have used, and I very much like, Pyro, but I do not see what
advantage it could provide for this application. You should be able to
do everything you need to in a single process with a GUI thread and a
Csound performance thread.

Note that on Windows, on the Macintosh, and frequently on Linux audio
hosts applications typically load plugins as shared modules and run
them in separate threads from the host GUI. We are trying to do the
same thing here.

Why don't you try the most straightforward approach first and see how
well it works for you?

Regards,
Mike

On 10/22/09, Iain Duncan  wrote:
> On Thu, 2009-10-22 at 10:18 +0200, Oeyvind Brandtsegg wrote:
>> Agreed, and psyco can also speed python up considerably.
>>
>> Regarding Iains question about separating the gui:
>> (welcome back Iain;-))
>> Yes, it is definitely a good idea to try to untangle the gui
>> processing from the audio rendering. This can probably be done in many
>> different ways, the main issue being the separation (thread or process
>> or application), not the actual method used to acheive separation.
>> The way I've done it, I have two separate python applications (one for
>> gui, one for composition processes and running Csound) communication
>> via Pyro (Python Remote Objects). This works well.  The gui and the
>> audio application automatically can find each other over the local
>> network, and you generally don't worry much about it after the initial
>> setup. .... to the extent that the details of setting it up have been
>> forgotten by me now, so I'll have to do some digging to give you more
>> details. Or you could just look at the ImproSculpt4 code at
>> sourceforge (no recent releases, but the cvs is kind of up to date).
>
> Thanks Oeyvind. Did you do this because you could not get what you
> needed happening using threads within one python process? Can you
> elucidate on where you hit problems on this?  Did you ever try using
> Cython for the audio thread? ( I haven't much experience in it, but one
> of my more experienced python colleagues swears by Cython for dealing
> with thread speed issues. )
>
> Do you have the code you've done this with up anywhere?
>
> Iain
>
>>
>> best
>> Oeyvind
>>
>> 2009/10/22 DavidW :
>> >
>> > On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
>> > ...
>> >>
>> >> Bottom line: I am not aware of any underlying reason why Python
>> >> programs that are not doing a whole lot of extra work couldn't
>> >> co-exist with a smoothly running instance of Csound.
>> >
>> > And don't be afraid to use Numpy for any extensive and/or recurring
>> > numerical matrix calculations.
>> >
>> > D.
>> >
>> > ________________________________________________
>> > Dr David Worrall.
>> > - Experimental Polymedia:         www.avatar.com.au
>> > - Sonification:                           sonification.com.au
>> >
>> >
>> >
>> >
>> > Send bugs reports to this list.
>> > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
>> > csound"
>> >
>>
>>
>> Send bugs reports to this list.
>> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
>> csound"
>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> csound"
>

On Thu, 2009-10-22 at 15:10 -0400, Michael Gogins wrote:
> I repeat, Cython for the thread is a red herring,
> CsoundPerformanceThread already does what you are thinking about. That
> is, CsoundPerformanceThread creates a separate thread in C++ code,
> then returns immediately to Python while the Csound thread continues
> to run. All monkey business about the GIL does not apply, as this is
> not a Python thread, it is a C thread. If you don't believe me, look
> at the code yourself.

thanks, I certainly will.

> 
> I have used, and I very much like, Pyro, but I do not see what
> advantage it could provide for this application. You should be able to
> do everything you need to in a single process with a GUI thread and a
> Csound performance thread.
> 
> Note that on Windows, on the Macintosh, and frequently on Linux audio
> hosts applications typically load plugins as shared modules and run
> them in separate threads from the host GUI. We are trying to do the
> same thing here.
> 
> Why don't you try the most straightforward approach first and see how
> well it works for you?

I intend to try all approaches. My concern about CsoundPerformanceThread
is whether I can do lower level control as I as doing previously with
the C api, wherein I was manually asking for one ksmp of audio to be
rendered and then taking only X messages off a queue for write/read into
csound space. Does CsoundPerformanceThread allow that or is there a
python alternative that might?

Essentially I want to be able to drop an entire 7 minute audio file into
a csound table on the fly, so that's going to need to be a heavily
buffered operation. I was envisioning the python app would spawn a
thread for reading in the audio file and then send each sample into
csound space over a queue, taking it's time to get it in there.

Thanks
iain

> 
> Regards,
> Mike
> 
> On 10/22/09, Iain Duncan  wrote:
> > On Thu, 2009-10-22 at 10:18 +0200, Oeyvind Brandtsegg wrote:
> >> Agreed, and psyco can also speed python up considerably.
> >>
> >> Regarding Iains question about separating the gui:
> >> (welcome back Iain;-))
> >> Yes, it is definitely a good idea to try to untangle the gui
> >> processing from the audio rendering. This can probably be done in many
> >> different ways, the main issue being the separation (thread or process
> >> or application), not the actual method used to acheive separation.
> >> The way I've done it, I have two separate python applications (one for
> >> gui, one for composition processes and running Csound) communication
> >> via Pyro (Python Remote Objects). This works well.  The gui and the
> >> audio application automatically can find each other over the local
> >> network, and you generally don't worry much about it after the initial
> >> setup. .... to the extent that the details of setting it up have been
> >> forgotten by me now, so I'll have to do some digging to give you more
> >> details. Or you could just look at the ImproSculpt4 code at
> >> sourceforge (no recent releases, but the cvs is kind of up to date).
> >
> > Thanks Oeyvind. Did you do this because you could not get what you
> > needed happening using threads within one python process? Can you
> > elucidate on where you hit problems on this?  Did you ever try using
> > Cython for the audio thread? ( I haven't much experience in it, but one
> > of my more experienced python colleagues swears by Cython for dealing
> > with thread speed issues. )
> >
> > Do you have the code you've done this with up anywhere?
> >
> > Iain
> >
> >>
> >> best
> >> Oeyvind
> >>
> >> 2009/10/22 DavidW :
> >> >
> >> > On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
> >> > ...
> >> >>
> >> >> Bottom line: I am not aware of any underlying reason why Python
> >> >> programs that are not doing a whole lot of extra work couldn't
> >> >> co-exist with a smoothly running instance of Csound.
> >> >
> >> > And don't be afraid to use Numpy for any extensive and/or recurring
> >> > numerical matrix calculations.
> >> >
> >> > D.
> >> >
> >> > ________________________________________________
> >> > Dr David Worrall.
> >> > - Experimental Polymedia:         www.avatar.com.au
> >> > - Sonification:                           sonification.com.au
> >> >
> >> >
> >> >
> >> >
> >> > Send bugs reports to this list.
> >> > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> >> > csound"
> >> >
> >>
> >>
> >> Send bugs reports to this list.
> >> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> >> csound"
> >
> >
> >
> > Send bugs reports to this list.
> > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> > csound"
> >
> 
> 



Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Yes, that's reasonable about sending a file through a queue. I assume
that the reader of the queue would block if nothing were in the queue,
and the writer to the queue would block if the queue were filled to a
certain point. That way the two threads sort of turn each other off
and on. That way you don't need to get fancy about how many samples at
a time you do. You just fill up the queue and wait for it to be
emptied, and when it's empty you fill it up some more. This is the
standard pattern for streaming data between threads.

The overhead of setting up condition variables to signal when to wait
and when to proceed is negligible in this context. But such code would
better be done in C++, partly because of the threading issues in
Python.

You might even be able to use OSC to do some of this stuff. It has
Python bindings, I think.

Regards,
Mike

On 10/22/09, Iain Duncan  wrote:
> On Thu, 2009-10-22 at 15:10 -0400, Michael Gogins wrote:
>> I repeat, Cython for the thread is a red herring,
>> CsoundPerformanceThread already does what you are thinking about. That
>> is, CsoundPerformanceThread creates a separate thread in C++ code,
>> then returns immediately to Python while the Csound thread continues
>> to run. All monkey business about the GIL does not apply, as this is
>> not a Python thread, it is a C thread. If you don't believe me, look
>> at the code yourself.
>
> thanks, I certainly will.
>
>>
>> I have used, and I very much like, Pyro, but I do not see what
>> advantage it could provide for this application. You should be able to
>> do everything you need to in a single process with a GUI thread and a
>> Csound performance thread.
>>
>> Note that on Windows, on the Macintosh, and frequently on Linux audio
>> hosts applications typically load plugins as shared modules and run
>> them in separate threads from the host GUI. We are trying to do the
>> same thing here.
>>
>> Why don't you try the most straightforward approach first and see how
>> well it works for you?
>
> I intend to try all approaches. My concern about CsoundPerformanceThread
> is whether I can do lower level control as I as doing previously with
> the C api, wherein I was manually asking for one ksmp of audio to be
> rendered and then taking only X messages off a queue for write/read into
> csound space. Does CsoundPerformanceThread allow that or is there a
> python alternative that might?
>
> Essentially I want to be able to drop an entire 7 minute audio file into
> a csound table on the fly, so that's going to need to be a heavily
> buffered operation. I was envisioning the python app would spawn a
> thread for reading in the audio file and then send each sample into
> csound space over a queue, taking it's time to get it in there.
>
> Thanks
> iain
>
>>
>> Regards,
>> Mike
>>
>> On 10/22/09, Iain Duncan  wrote:
>> > On Thu, 2009-10-22 at 10:18 +0200, Oeyvind Brandtsegg wrote:
>> >> Agreed, and psyco can also speed python up considerably.
>> >>
>> >> Regarding Iains question about separating the gui:
>> >> (welcome back Iain;-))
>> >> Yes, it is definitely a good idea to try to untangle the gui
>> >> processing from the audio rendering. This can probably be done in many
>> >> different ways, the main issue being the separation (thread or process
>> >> or application), not the actual method used to acheive separation.
>> >> The way I've done it, I have two separate python applications (one for
>> >> gui, one for composition processes and running Csound) communication
>> >> via Pyro (Python Remote Objects). This works well.  The gui and the
>> >> audio application automatically can find each other over the local
>> >> network, and you generally don't worry much about it after the initial
>> >> setup. .... to the extent that the details of setting it up have been
>> >> forgotten by me now, so I'll have to do some digging to give you more
>> >> details. Or you could just look at the ImproSculpt4 code at
>> >> sourceforge (no recent releases, but the cvs is kind of up to date).
>> >
>> > Thanks Oeyvind. Did you do this because you could not get what you
>> > needed happening using threads within one python process? Can you
>> > elucidate on where you hit problems on this?  Did you ever try using
>> > Cython for the audio thread? ( I haven't much experience in it, but one
>> > of my more experienced python colleagues swears by Cython for dealing
>> > with thread speed issues. )
>> >
>> > Do you have the code you've done this with up anywhere?
>> >
>> > Iain
>> >
>> >>
>> >> best
>> >> Oeyvind
>> >>
>> >> 2009/10/22 DavidW :
>> >> >
>> >> > On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
>> >> > ...
>> >> >>
>> >> >> Bottom line: I am not aware of any underlying reason why Python
>> >> >> programs that are not doing a whole lot of extra work couldn't
>> >> >> co-exist with a smoothly running instance of Csound.
>> >> >
>> >> > And don't be afraid to use Numpy for any extensive and/or recurring
>> >> > numerical matrix calculations.
>> >> >
>> >> > D.
>> >> >
>> >> > ________________________________________________
>> >> > Dr David Worrall.
>> >> > - Experimental Polymedia:         www.avatar.com.au
>> >> > - Sonification:                           sonification.com.au
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > Send bugs reports to this list.
>> >> > To unsubscribe, send email sympa@lists.bath.ac.uk with body
>> >> > "unsubscribe
>> >> > csound"
>> >> >
>> >>
>> >>
>> >> Send bugs reports to this list.
>> >> To unsubscribe, send email sympa@lists.bath.ac.uk with body
>> >> "unsubscribe
>> >> csound"
>> >
>> >
>> >
>> > Send bugs reports to this list.
>> > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
>> > csound"
>> >
>>
>>
>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> csound"
>

If it's the integration of csound +  python with GUI you're concerned  
about, I have had some success using the threading techniques  
discussed in Rapin & Dunn's "wxPython in Action". Haven't really  
pushed its limits but have been happy w. the results so far.

David
On 23/10/2009, at 6:10 AM, Michael Gogins wrote:

> I repeat, Cython for the thread is a red herring,
> CsoundPerformanceThread already does what you are thinking about. That
> is, CsoundPerformanceThread creates a separate thread in C++ code,
> then returns immediately to Python while the Csound thread continues
> to run. All monkey business about the GIL does not apply, as this is
> not a Python thread, it is a C thread. If you don't believe me, look
> at the code yourself.
>
> I have used, and I very much like, Pyro, but I do not see what
> advantage it could provide for this application. You should be able to
> do everything you need to in a single process with a GUI thread and a
> Csound performance thread.
>
> Note that on Windows, on the Macintosh, and frequently on Linux audio
> hosts applications typically load plugins as shared modules and run
> them in separate threads from the host GUI. We are trying to do the
> same thing here.
>
> Why don't you try the most straightforward approach first and see how
> well it works for you?
>
> Regards,
> Mike
...


________________________________________________
Dr David Worrall.
- Experimental Polymedia:	  www.avatar.com.au
- Sonification: www.sonification.com.au






Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

One of the reasons I did it was that I needed to read chn output
values from Csound in a synchronized manner, and I also wanted to have
my python timing functions be slaved to Csound ksmps to ensure sample
accurate (well... ksmps accurate anyway) timing between my sequencer
(Python timing) and the audio engine (Csound). some of this may be
accomplished by using the performance thread and callbacks but I
haven't gotten around to nailing it down. I know that using the
performance thread is the way to go and will look into it again as
soon as I can. I would definaely reccommend using the performance
thread unless you have explicit reasons for not doing so.

Another reason for the Pyro split is to enable multicore processing.
so the Gui can run on one core while Csound runs on another. This is
about to change, as work is being done on multicore Csound and
probably likewise on Python. My design was made 2 years ago and there
was no other way of putting my second core to work than doing a "stone
age, brute force" split like this.

The code is at http://sourceforge.net/projects/improsculpt/, as
mentioned the "release" is quite old, but the cvs is quite up to date.
Even in the "release" version, the stuff we talked about here is
pretty much up to date IIRC.

I currently run at -b100 -B200, with ksmps=50 or 100. This is not
perfect, but works fairly well.

Oeyvind

2009/10/22 Iain Duncan :
> On Thu, 2009-10-22 at 10:18 +0200, Oeyvind Brandtsegg wrote:
>> Agreed, and psyco can also speed python up considerably.
>>
>> Regarding Iains question about separating the gui:
>> (welcome back Iain;-))
>> Yes, it is definitely a good idea to try to untangle the gui
>> processing from the audio rendering. This can probably be done in many
>> different ways, the main issue being the separation (thread or process
>> or application), not the actual method used to acheive separation.
>> The way I've done it, I have two separate python applications (one for
>> gui, one for composition processes and running Csound) communication
>> via Pyro (Python Remote Objects). This works well.  The gui and the
>> audio application automatically can find each other over the local
>> network, and you generally don't worry much about it after the initial
>> setup. .... to the extent that the details of setting it up have been
>> forgotten by me now, so I'll have to do some digging to give you more
>> details. Or you could just look at the ImproSculpt4 code at
>> sourceforge (no recent releases, but the cvs is kind of up to date).
>
> Thanks Oeyvind. Did you do this because you could not get what you
> needed happening using threads within one python process? Can you
> elucidate on where you hit problems on this?  Did you ever try using
> Cython for the audio thread? ( I haven't much experience in it, but one
> of my more experienced python colleagues swears by Cython for dealing
> with thread speed issues. )
>
> Do you have the code you've done this with up anywhere?
>
> Iain
>
>>
>> best
>> Oeyvind
>>
>> 2009/10/22 DavidW :
>> >
>> > On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
>> > ...
>> >>
>> >> Bottom line: I am not aware of any underlying reason why Python
>> >> programs that are not doing a whole lot of extra work couldn't
>> >> co-exist with a smoothly running instance of Csound.
>> >
>> > And don't be afraid to use Numpy for any extensive and/or recurring
>> > numerical matrix calculations.
>> >
>> > D.
>> >
>> > ________________________________________________
>> > Dr David Worrall.
>> > - Experimental Polymedia:         www.avatar.com.au
>> > - Sonification:                           sonification.com.au
>> >
>> >
>> >
>> >
>> > Send bugs reports to this list.
>> > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
>> > csound"
>> >
>>
>>
>> Send bugs reports to this list.
>> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"
>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"
>


Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"