Csound Csound-dev Csound-tekno Search About

[Csnd] Re: Re: Re: Re: Re: Re: python api and speed ( and docs )

Date2009-10-23 02:41
Frommichael.gogins@gmail.com
Subject[Csnd] Re: Re: Re: Re: Re: Re: python api and speed ( and docs )
Muticore Csound may happen at some point, it is not ready for prime time 
yet, but I or someone will make sure that it does happen.

About Python, I know of no prospect for multicore Python. The Python code 
base is resolutely single-threaded. This is a common situation with respect 
to interpreted languages.

You can run multiple Python processes, each on its own core, and if you have 
an efficient way of sharing data or sending messages between processes, that 
is how you can do multicore with Python. Pyro could be used to communicate 
between such processes, so Oeyvind's approach would make sense if you needed 
to do your parallel processing in Python. But you don't need this just to 
get Csound going in its own thread.

Regards,
Mike

----- Original Message ----- 
From: "Oeyvind Brandtsegg" 
To: 
Sent: Thursday, October 22, 2009 8:11 PM
Subject: [Csnd] Re: Re: Re: Re: Re: python api and speed ( and docs )


One of the reasons I did it was that I needed to read chn output
values from Csound in a synchronized manner, and I also wanted to have
my python timing functions be slaved to Csound ksmps to ensure sample
accurate (well... ksmps accurate anyway) timing between my sequencer
(Python timing) and the audio engine (Csound). some of this may be
accomplished by using the performance thread and callbacks but I
haven't gotten around to nailing it down. I know that using the
performance thread is the way to go and will look into it again as
soon as I can. I would definaely reccommend using the performance
thread unless you have explicit reasons for not doing so.

Another reason for the Pyro split is to enable multicore processing.
so the Gui can run on one core while Csound runs on another. This is
about to change, as work is being done on multicore Csound and
probably likewise on Python. My design was made 2 years ago and there
was no other way of putting my second core to work than doing a "stone
age, brute force" split like this.

The code is at http://sourceforge.net/projects/improsculpt/, as
mentioned the "release" is quite old, but the cvs is quite up to date.
Even in the "release" version, the stuff we talked about here is
pretty much up to date IIRC.

I currently run at -b100 -B200, with ksmps=50 or 100. This is not
perfect, but works fairly well.

Oeyvind

2009/10/22 Iain Duncan :
> On Thu, 2009-10-22 at 10:18 +0200, Oeyvind Brandtsegg wrote:
>> Agreed, and psyco can also speed python up considerably.
>>
>> Regarding Iains question about separating the gui:
>> (welcome back Iain;-))
>> Yes, it is definitely a good idea to try to untangle the gui
>> processing from the audio rendering. This can probably be done in many
>> different ways, the main issue being the separation (thread or process
>> or application), not the actual method used to acheive separation.
>> The way I've done it, I have two separate python applications (one for
>> gui, one for composition processes and running Csound) communication
>> via Pyro (Python Remote Objects). This works well. The gui and the
>> audio application automatically can find each other over the local
>> network, and you generally don't worry much about it after the initial
>> setup. .... to the extent that the details of setting it up have been
>> forgotten by me now, so I'll have to do some digging to give you more
>> details. Or you could just look at the ImproSculpt4 code at
>> sourceforge (no recent releases, but the cvs is kind of up to date).
>
> Thanks Oeyvind. Did you do this because you could not get what you
> needed happening using threads within one python process? Can you
> elucidate on where you hit problems on this? Did you ever try using
> Cython for the audio thread? ( I haven't much experience in it, but one
> of my more experienced python colleagues swears by Cython for dealing
> with thread speed issues. )
>
> Do you have the code you've done this with up anywhere?
>
> Iain
>
>>
>> best
>> Oeyvind
>>
>> 2009/10/22 DavidW :
>> >
>> > On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
>> > ...
>> >>
>> >> Bottom line: I am not aware of any underlying reason why Python
>> >> programs that are not doing a whole lot of extra work couldn't
>> >> co-exist with a smoothly running instance of Csound.
>> >
>> > And don't be afraid to use Numpy for any extensive and/or recurring
>> > numerical matrix calculations.
>> >
>> > D.
>> >
>> > ________________________________________________
>> > Dr David Worrall.
>> > - Experimental Polymedia: www.avatar.com.au
>> > - Sonification: sonification.com.au
>> >
>> >
>> >
>> >
>> > Send bugs reports to this list.
>> > To unsubscribe, send email sympa@lists.bath.ac.uk with body 
>> > "unsubscribe
>> > csound"
>> >
>>
>>
>> Send bugs reports to this list.
>> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe 
>> csound"
>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe 
> csound"
>


Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe 
csound" 



Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Date2009-10-23 03:43
Fromthorin kerr
Subject[Csnd] Re: Re: Re: Re: Re: Re: Re: python api and speed ( and docs )
Fo those interested, there are a lot of parallel processing options
out there for python.
Here's a good starting point:
http://wiki.python.org/moin/ParallelProcessing
not to mention the Multiprocessing python standard library module
http://docs.python.org/dev/library/multiprocessing.html

TK

On Fri, Oct 23, 2009 at 11:41 AM,   wrote:
> Muticore Csound may happen at some point, it is not ready for prime time
> yet, but I or someone will make sure that it does happen.
>
> About Python, I know of no prospect for multicore Python. The Python code
> base is resolutely single-threaded. This is a common situation with respect
> to interpreted languages.
>
> You can run multiple Python processes, each on its own core, and if you have
> an efficient way of sharing data or sending messages between processes, that
> is how you can do multicore with Python. Pyro could be used to communicate
> between such processes, so Oeyvind's approach would make sense if you needed
> to do your parallel processing in Python. But you don't need this just to
> get Csound going in its own thread.
>
> Regards,
> Mike
>
> ----- Original Message ----- From: "Oeyvind Brandtsegg" 
> To: 
> Sent: Thursday, October 22, 2009 8:11 PM
> Subject: [Csnd] Re: Re: Re: Re: Re: python api and speed ( and docs )
>
>
> One of the reasons I did it was that I needed to read chn output
> values from Csound in a synchronized manner, and I also wanted to have
> my python timing functions be slaved to Csound ksmps to ensure sample
> accurate (well... ksmps accurate anyway) timing between my sequencer
> (Python timing) and the audio engine (Csound). some of this may be
> accomplished by using the performance thread and callbacks but I
> haven't gotten around to nailing it down. I know that using the
> performance thread is the way to go and will look into it again as
> soon as I can. I would definaely reccommend using the performance
> thread unless you have explicit reasons for not doing so.
>
> Another reason for the Pyro split is to enable multicore processing.
> so the Gui can run on one core while Csound runs on another. This is
> about to change, as work is being done on multicore Csound and
> probably likewise on Python. My design was made 2 years ago and there
> was no other way of putting my second core to work than doing a "stone
> age, brute force" split like this.
>
> The code is at http://sourceforge.net/projects/improsculpt/, as
> mentioned the "release" is quite old, but the cvs is quite up to date.
> Even in the "release" version, the stuff we talked about here is
> pretty much up to date IIRC.
>
> I currently run at -b100 -B200, with ksmps=50 or 100. This is not
> perfect, but works fairly well.
>
> Oeyvind
>
> 2009/10/22 Iain Duncan :
>>
>> On Thu, 2009-10-22 at 10:18 +0200, Oeyvind Brandtsegg wrote:
>>>
>>> Agreed, and psyco can also speed python up considerably.
>>>
>>> Regarding Iains question about separating the gui:
>>> (welcome back Iain;-))
>>> Yes, it is definitely a good idea to try to untangle the gui
>>> processing from the audio rendering. This can probably be done in many
>>> different ways, the main issue being the separation (thread or process
>>> or application), not the actual method used to acheive separation.
>>> The way I've done it, I have two separate python applications (one for
>>> gui, one for composition processes and running Csound) communication
>>> via Pyro (Python Remote Objects). This works well. The gui and the
>>> audio application automatically can find each other over the local
>>> network, and you generally don't worry much about it after the initial
>>> setup. .... to the extent that the details of setting it up have been
>>> forgotten by me now, so I'll have to do some digging to give you more
>>> details. Or you could just look at the ImproSculpt4 code at
>>> sourceforge (no recent releases, but the cvs is kind of up to date).
>>
>> Thanks Oeyvind. Did you do this because you could not get what you
>> needed happening using threads within one python process? Can you
>> elucidate on where you hit problems on this? Did you ever try using
>> Cython for the audio thread? ( I haven't much experience in it, but one
>> of my more experienced python colleagues swears by Cython for dealing
>> with thread speed issues. )
>>
>> Do you have the code you've done this with up anywhere?
>>
>> Iain
>>
>>>
>>> best
>>> Oeyvind
>>>
>>> 2009/10/22 DavidW :
>>> >
>>> > On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
>>> > ...
>>> >>
>>> >> Bottom line: I am not aware of any underlying reason why Python
>>> >> programs that are not doing a whole lot of extra work couldn't
>>> >> co-exist with a smoothly running instance of Csound.
>>> >
>>> > And don't be afraid to use Numpy for any extensive and/or recurring
>>> > numerical matrix calculations.
>>> >
>>> > D.
>>> >
>>> > ________________________________________________
>>> > Dr David Worrall.
>>> > - Experimental Polymedia: www.avatar.com.au
>>> > - Sonification: sonification.com.au
>>> >
>>> >
>>> >
>>> >
>>> > Send bugs reports to this list.
>>> > To unsubscribe, send email sympa@lists.bath.ac.uk with body >
>>> > "unsubscribe
>>> > csound"
>>> >
>>>
>>>
>>> Send bugs reports to this list.
>>> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
>>> csound"
>>
>>
>>
>> Send bugs reports to this list.
>> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
>> csound"
>>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> csound"
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
> csound"
>


Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Date2009-10-23 09:39
FromOeyvind Brandtsegg
Subject[Csnd] Re: Re: Re: Re: Re: Re: Re: Re: python api and speed ( and docs )
Thanks for the links Thorin, this is interesting indeed.
Oeyvind

2009/10/23 thorin kerr :
> Fo those interested, there are a lot of parallel processing options
> out there for python.
> Here's a good starting point:
> http://wiki.python.org/moin/ParallelProcessing
> not to mention the Multiprocessing python standard library module
> http://docs.python.org/dev/library/multiprocessing.html
>
> TK
>
> On Fri, Oct 23, 2009 at 11:41 AM,   wrote:
>> Muticore Csound may happen at some point, it is not ready for prime time
>> yet, but I or someone will make sure that it does happen.
>>
>> About Python, I know of no prospect for multicore Python. The Python code
>> base is resolutely single-threaded. This is a common situation with respect
>> to interpreted languages.
>>
>> You can run multiple Python processes, each on its own core, and if you have
>> an efficient way of sharing data or sending messages between processes, that
>> is how you can do multicore with Python. Pyro could be used to communicate
>> between such processes, so Oeyvind's approach would make sense if you needed
>> to do your parallel processing in Python. But you don't need this just to
>> get Csound going in its own thread.
>>
>> Regards,
>> Mike
>>
>> ----- Original Message ----- From: "Oeyvind Brandtsegg" 
>> To: 
>> Sent: Thursday, October 22, 2009 8:11 PM
>> Subject: [Csnd] Re: Re: Re: Re: Re: python api and speed ( and docs )
>>
>>
>> One of the reasons I did it was that I needed to read chn output
>> values from Csound in a synchronized manner, and I also wanted to have
>> my python timing functions be slaved to Csound ksmps to ensure sample
>> accurate (well... ksmps accurate anyway) timing between my sequencer
>> (Python timing) and the audio engine (Csound). some of this may be
>> accomplished by using the performance thread and callbacks but I
>> haven't gotten around to nailing it down. I know that using the
>> performance thread is the way to go and will look into it again as
>> soon as I can. I would definaely reccommend using the performance
>> thread unless you have explicit reasons for not doing so.
>>
>> Another reason for the Pyro split is to enable multicore processing.
>> so the Gui can run on one core while Csound runs on another. This is
>> about to change, as work is being done on multicore Csound and
>> probably likewise on Python. My design was made 2 years ago and there
>> was no other way of putting my second core to work than doing a "stone
>> age, brute force" split like this.
>>
>> The code is at http://sourceforge.net/projects/improsculpt/, as
>> mentioned the "release" is quite old, but the cvs is quite up to date.
>> Even in the "release" version, the stuff we talked about here is
>> pretty much up to date IIRC.
>>
>> I currently run at -b100 -B200, with ksmps=50 or 100. This is not
>> perfect, but works fairly well.
>>
>> Oeyvind
>>
>> 2009/10/22 Iain Duncan :
>>>
>>> On Thu, 2009-10-22 at 10:18 +0200, Oeyvind Brandtsegg wrote:
>>>>
>>>> Agreed, and psyco can also speed python up considerably.
>>>>
>>>> Regarding Iains question about separating the gui:
>>>> (welcome back Iain;-))
>>>> Yes, it is definitely a good idea to try to untangle the gui
>>>> processing from the audio rendering. This can probably be done in many
>>>> different ways, the main issue being the separation (thread or process
>>>> or application), not the actual method used to acheive separation.
>>>> The way I've done it, I have two separate python applications (one for
>>>> gui, one for composition processes and running Csound) communication
>>>> via Pyro (Python Remote Objects). This works well. The gui and the
>>>> audio application automatically can find each other over the local
>>>> network, and you generally don't worry much about it after the initial
>>>> setup. .... to the extent that the details of setting it up have been
>>>> forgotten by me now, so I'll have to do some digging to give you more
>>>> details. Or you could just look at the ImproSculpt4 code at
>>>> sourceforge (no recent releases, but the cvs is kind of up to date).
>>>
>>> Thanks Oeyvind. Did you do this because you could not get what you
>>> needed happening using threads within one python process? Can you
>>> elucidate on where you hit problems on this? Did you ever try using
>>> Cython for the audio thread? ( I haven't much experience in it, but one
>>> of my more experienced python colleagues swears by Cython for dealing
>>> with thread speed issues. )
>>>
>>> Do you have the code you've done this with up anywhere?
>>>
>>> Iain
>>>
>>>>
>>>> best
>>>> Oeyvind
>>>>
>>>> 2009/10/22 DavidW :
>>>> >
>>>> > On 22/10/2009, at 5:57 AM, Michael Gogins wrote:
>>>> > ...
>>>> >>
>>>> >> Bottom line: I am not aware of any underlying reason why Python
>>>> >> programs that are not doing a whole lot of extra work couldn't
>>>> >> co-exist with a smoothly running instance of Csound.
>>>> >
>>>> > And don't be afraid to use Numpy for any extensive and/or recurring
>>>> > numerical matrix calculations.
>>>> >
>>>> > D.
>>>> >
>>>> > ________________________________________________
>>>> > Dr David Worrall.
>>>> > - Experimental Polymedia: www.avatar.com.au
>>>> > - Sonification: sonification.com.au
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > Send bugs reports to this list.
>>>> > To unsubscribe, send email sympa@lists.bath.ac.uk with body >
>>>> > "unsubscribe
>>>> > csound"
>>>> >
>>>>
>>>>
>>>> Send bugs reports to this list.
>>>> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
>>>> csound"
>>>
>>>
>>>
>>> Send bugs reports to this list.
>>> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
>>> csound"
>>>
>>
>>
>> Send bugs reports to this list.
>> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
>> csound"
>>
>>
>> Send bugs reports to this list.
>> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe
>> csound"
>>
>
>
> Send bugs reports to this list.
> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"
>


Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Date2009-10-23 15:17
Fromjpff@cs.bath.ac.uk
Subject[Csnd] Re: Re: Re: Re: Re: Re: Re: python api and speed ( and docs )
> Muticore Csound may happen at some point, it is not ready for prime time
> yet, but I or someone will make sure that it does happen.
>
>

Multicore csound exists in CVS but need tine to sort out some problems
==John ff



Send bugs reports to this list.
To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"