Optimization tip wanted for Csound/Python
Date | 2016-10-13 13:33 |
From | Oeyvind Brandtsegg |
Subject | Optimization tip wanted for Csound/Python |
Hi, I need to transfer the contents of an array from Csound to Python, as I want to do some peak picking and other analysis that is easier to get done in Python. The problem is that my (admittedly ad hoc) method of transferring the values from Csound to Python incurs significant performance degradation (more details on how I measure this later in this email). The basic premise is that I need Csound to be the main program (the host), and Python runs via the py opcodes. The array kAuto is of size 256, and transfer needs to be done once every 1 second, for 3 of these arrays. One should think the bandwidth requirement are not too taxing on any system, but since I only send single values, there are more than 700 pycalls happening in one k-rate pass. The python function "p.fill_array" does nothing but put the received values one by one into a numpy array. kcnt_fill = 0 while kcnt_fill < ifftsize/2 do pycall "p.fill_array", kAuto[kcnt_fill], kcnt_fill, ifftsize/2 kcnt_fill += 1 od By performance degradation I mean that the operation seems to let the different processes to wait for each other a lot, utlimately leading to dropouts in realtime audio. This does not show up as CPU spikes (running at around 50%), but shows in Reaper's performence meter as "RT longest-block". This meter shows the number of milliseconds used/available per audio processing block. The *used* must generally be less than the *available* time to avoid dropouts. Some spikes are not necessarily bad, but in my case the *used* is generally quite stable and higher than the *available*, unless I comment out the code excerpt given above. Is there any way I can send an array from Csound to Python in one go? I seem to remember some black magic about letting Python running under Csound get access to the (host) csound instance and thus accessing the Csound API. Would that be the best way to do it, and would it be considered a safe way of doing it? best Oeyvind |
Date | 2016-10-13 20:04 |
From | Tarmo Johannes |
Subject | Re: Optimization tip wanted for Csound/Python |
Not sure, but would --realtime switch on command line help? I don't know if that puts python to other thread from audio work... 13.10.2016 14:34 kirjutas kuupäeval "Oeyvind Brandtsegg" <oyvind.brandtsegg@ntnu.no>: Hi, |
Date | 2016-10-13 20:29 |
From | Steven Yi |
Subject | Re: Optimization tip wanted for Csound/Python |
Have you tried ftables? I.e. Filling the table in Csound, then reading from the table in the python code. I seem to remember the python opcodes having access to ftables and that there was example code for this. On Thu, Oct 13, 2016 at 3:04 PM Tarmo Johannes <tarmo.johannes@otsakool.edu.ee> wrote:
|
Date | 2016-10-13 20:44 |
From | joachim heintz |
Subject | Re: Optimization tip wanted for Csound/Python |
another option might be to use OSC. john recently implemented the possibility to send/receive arrays and tables via OSC. not sure if this can be an option for your case though. ciao - joachim On 13/10/16 21:29, Steven Yi wrote: > Have you tried ftables? I.e. Filling the table in Csound, then reading from > the table in the python code. I seem to remember the python opcodes having > access to ftables and that there was example code for this. > On Thu, Oct 13, 2016 at 3:04 PM Tarmo Johannes < > tarmo.johannes@otsakool.edu.ee> wrote: > >> Not sure, but would --realtime switch on command line help? I don't know >> if that puts python to other thread from audio work... >> >> >> Tarmo >> >> 13.10.2016 14:34 kirjutas kuupäeval "Oeyvind Brandtsegg" < >> oyvind.brandtsegg@ntnu.no>: >> >> Hi, >> I need to transfer the contents of an array from Csound to Python, as >> I want to do some peak picking and other analysis that is easier to >> get done in Python. The problem is that my (admittedly ad hoc) method >> of transferring the values from Csound to Python incurs significant >> performance degradation (more details on how I measure this later in >> this email). >> >> The basic premise is that I need Csound to be the main program (the >> host), and Python runs via the py opcodes. >> >> The array kAuto is of size 256, and transfer needs to be done once >> every 1 second, for 3 of these arrays. One should think the bandwidth >> requirement are not too taxing on any system, but since I only send >> single values, there are more than 700 pycalls happening in one k-rate >> pass. >> The python function "p.fill_array" does nothing but put the received >> values one by one into a numpy array. >> >> kcnt_fill = 0 >> while kcnt_fill < ifftsize/2 do >> pycall "p.fill_array", kAuto[kcnt_fill], kcnt_fill, ifftsize/2 >> kcnt_fill += 1 >> od >> >> >> By performance degradation I mean that the operation seems to let the >> different processes to wait for each other a lot, utlimately leading >> to dropouts in realtime audio. This does not show up as CPU spikes >> (running at around 50%), but shows in Reaper's performence meter as >> "RT longest-block". This meter shows the number of milliseconds >> used/available per audio processing block. The *used* must generally >> be less than the *available* time to avoid dropouts. Some spikes are >> not necessarily bad, but in my case the *used* is generally quite >> stable and higher than the *available*, unless I comment out the code >> excerpt given above. >> >> Is there any way I can send an array from Csound to Python in one go? >> I seem to remember some black magic about letting Python running under >> Csound get access to the (host) csound instance and thus accessing the >> Csound API. Would that be the best way to do it, and would it be >> considered a safe way of doing it? >> >> best >> Oeyvind >> >> -- >> >> Oeyvind Brandtsegg >> Professor of Music Technology >> NTNU >> 7491 Trondheim >> Norway >> Cell: +47 92 203 205 >> >> http://www.partikkelaudio.com/ >> http://crossadaptive.hf.ntnu.no >> http://gdsp.hf.ntnu.no/ >> http://soundcloud.com/brandtsegg >> http://flyndresang.no/ >> http://soundcloud.com/t-emp >> >> Csound mailing list >> Csound@listserv.heanet.ie >> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND >> Send bugs reports to >> https://github.com/csound/csound/issues >> Discussions of bugs and features can be posted here >> >> Csound mailing list Csound@listserv.heanet.ie >> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to >> https://github.com/csound/csound/issues Discussions of bugs and features >> can be posted here > > Csound mailing list > Csound@listserv.heanet.ie > https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND > Send bugs reports to > https://github.com/csound/csound/issues > Discussions of bugs and features can be posted here > Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here |
Date | 2016-10-13 22:07 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
Hi all, thanks for the suggestions. If I can read from a csound table directly in python code, that would be the best. Steven, do you mean this document :http://pythonsound.sourceforge.net/pycsound.pdf ? There were several things in there that I did not know about, is the csound module as described there currently supported? 2016-10-13 21:29 GMT+02:00 Steven Yi |
Date | 2016-10-13 22:18 |
From | Steven Yi |
Subject | Re: Optimization tip wanted for Csound/Python |
Yes I think that was what I was thinking of, but I'm unsure whether it is supported now or not. On Thu, Oct 13, 2016, 5:07 PM Oeyvind Brandtsegg <oyvind.brandtsegg@ntnu.no> wrote: Hi all, |
Date | 2016-10-13 22:49 |
From | Victor Lazzarini |
Subject | Re: Optimization tip wanted for Csound/Python |
It is possible to read a Csound ftable via the API.
I'd also suggest migrating to ctcsound if you
are using csnd6. It's excellent.
Victor Lazzarini Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland
|
Date | 2016-10-14 07:39 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
Thanks, Yes, ctcsound looks very nice. So far I've just followed the info on the list with interest, waiting to dive in. Is there some instructions on how to migrate somewhere? Do I need to install Anaconda and Jupyter, or can I just replace some modules ? 2016-10-13 23:49 GMT+02:00 Victor Lazzarini |
Date | 2016-10-14 07:54 |
From | Francois PINOT |
Subject | Re: Optimization tip wanted for Csound/Python |
Only numpy is mandatory. FrançoisYou can use ctcsound with Python 2.x and/or Python 3.x. You need an environment set properly for each version of Python you will use. Anaconda facilitates this, and it provides a lot of modules for scientific computing. Jupyter is a plus allowing you to use Csound within notebooks. 2016-10-14 8:39 GMT+02:00 Oeyvind Brandtsegg <oyvind.brandtsegg@ntnu.no>: Thanks, Yes, ctcsound looks very nice. So far I've just followed the |
Date | 2016-10-14 08:38 |
From | Victor Lazzarini |
Subject | Re: Optimization tip wanted for Csound/Python |
Here’s the method from ctcsound: Help on method tableCopyIn in module ctcsound: tableCopyIn(self, table, src) unbound ctcsound.Csound method Copy the contents of an ndarray src into a given function table. The table number is assumed to be valid, and the table needs to have sufficient space to receive all the array contents. > On 14 Oct 2016, at 07:39, Oeyvind Brandtsegg |
Date | 2016-10-14 13:08 |
From | Steven Yi |
Subject | Re: Optimization tip wanted for Csound/Python |
But does ctcsound work within the python opcodes, as Oeyvind mentioned in the original email? On Fri, Oct 14, 2016 at 3:38 AM Victor Lazzarini <Victor.Lazzarini@nuim.ie> wrote: Here’s the method from ctcsound: |
Date | 2016-10-14 13:19 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
Ah, now I remember, I tried this earlier and did not complete installing Anaconda, as it seemed like it could interfere with other system variables so I backed out. Is there a way for me to configure the environment without installing Anaconda? I'm in the middle of several productions, so I am hesitant to let an installer make automatic changes to my system. 2016-10-14 8:54 GMT+02:00 Francois PINOT |
Date | 2016-10-14 13:26 |
From | Victor Lazzarini |
Subject | Re: Optimization tip wanted for Csound/Python |
You can just install numpy. Victor Lazzarini Dean of Arts, Celtic Studies, and Philosophy Maynooth University Ireland > On 14 Oct 2016, at 13:19, Oeyvind Brandtsegg |
Date | 2016-10-14 13:38 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
Yes, but how to I tell Csound (the py opcodes within Csound) to use ctcsound ? 2016-10-14 14:26 GMT+02:00 Victor Lazzarini |
Date | 2016-10-14 13:56 |
From | Francois PINOT |
Subject | Re: Optimization tip wanted for Csound/Python |
pyruni "import ctcsound" in global part? François2016-10-14 14:38 GMT+02:00 Oeyvind Brandtsegg <oyvind.brandtsegg@ntnu.no>: Yes, but how to I tell Csound (the py opcodes within Csound) to use ctcsound ? |
Date | 2016-10-14 14:23 |
From | Victor Lazzarini |
Subject | Re: Optimization tip wanted for Csound/Python |
We might need a csoundGetInstance() function or similar in ctcsound so that the opcodes can communicate with the running Csound. http://csound.github.io/docs/manual/pyinit.html > On 14 Oct 2016, at 13:56, Francois PINOT |
Date | 2016-10-15 11:04 |
From | Francois PINOT |
Subject | Re: Optimization tip wanted for Csound/Python |
I just pushed to my github repo an example showing how to use ctcsound from the py- opcodes within a running instance of Csound: Notice that actually Python2 is used by default due the hardcoded choice of Python when building Csound.https://github.com/fggp/ctcsound/blob/master/cookbook/examples/embeddedCtcsound.csd 2016-10-14 15:23 GMT+02:00 Victor Lazzarini <Victor.Lazzarini@nuim.ie>: We might need a csoundGetInstance() function or similar in ctcsound so that the |
Date | 2016-11-05 21:01 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
Thanks, this was left cold for a while, but now I'm back on track. I get >>> import ctcsound Traceback (most recent call last): File " |
Date | 2016-11-05 21:31 |
From | Victor Lazzarini |
Subject | Re: Optimization tip wanted for Csound/Python |
It's csound64.dll, which should be in the DLL path. Victor Lazzarini Dean of Arts, Celtic Studies, and Philosophy Maynooth University Ireland > On 5 Nov 2016, at 21:02, Oeyvind Brandtsegg |
Date | 2016-11-05 21:45 |
From | Michael Gogins |
Subject | Re: Optimization tip wanted for Csound/Python |
Try passing the complete pathname. Regards, On Nov 5, 2016 5:31 PM, "Victor Lazzarini" <Victor.Lazzarini@nuim.ie> wrote: It's csound64.dll, which should be in the DLL path. |
Date | 2016-11-06 11:21 |
From | Francois PINOT |
Subject | Re: Optimization tip wanted for Csound/Python |
csound64.dll should be accessible through your PATH. Normally it is located in "C:\Program Files\Csound_x64\bin". You can verify that this path is referenced in your PATH environment variable with this command:2016-11-05 22:01 GMT+01:00 Oeyvind Brandtsegg <oyvind.brandtsegg@ntnu.no>: Thanks, |
Date | 2016-11-07 18:20 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
Ok. It is in my path, but now I realize that this was not the culprit, ...my bad. I had not copied in csnd.dll from my build dir. Now I get test_ctcsound.py to this stage: (...snip) writing 4096 sample blks of 64-bit floats to dac SECTION 1: F ====================================================================== FAIL: test_inputOutputbuffer (__main__.TestGeneralIO) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_ctcsound.py", line 78, in test_inputOutputbuffer self.assertTrue(np.array_equal(obuf, ibuf/3.0)) AssertionError: False is not true ---------------------------------------------------------------------- Ran 6 tests in 8.216s FAILED (failures=1) 2016-11-06 3:21 GMT-08:00 Francois PINOT |
Date | 2016-11-07 18:27 |
From | Francois PINOT |
Subject | Re: Optimization tip wanted for Csound/Python |
Is the file "bufferInOut.csd" present in the directory you ran test_ctcsound.py from? François2016-11-07 19:20 GMT+01:00 Oeyvind Brandtsegg <oyvind.brandtsegg@ntnu.no>: Ok. It is in my path, |
Date | 2016-11-07 18:50 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
Yes, I downloaded the full repo as a zip. 2016-11-07 10:27 GMT-08:00 Francois PINOT |
Date | 2016-11-07 19:08 |
From | Francois PINOT |
Subject | Re: Optimization tip wanted for Csound/Python |
I think this is due to small floating point differences in the order of 1e-04. The test for equality is not appropriate here. You should consider that ctcsound is working on your system. François2016-11-07 19:50 GMT+01:00 Oeyvind Brandtsegg <oyvind.brandtsegg@ntnu.no>: Yes, I downloaded the full repo as a zip. |
Date | 2016-11-08 17:52 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
Great, thankl you so much for the help. I'll probably be back with more questions shortly :-) 2016-11-07 11:08 GMT-08:00 Francois PINOT |
Date | 2016-11-08 18:31 |
From | Rory Walsh |
Subject | Re: Optimization tip wanted for Csound/Python |
Btw Oeyvind, is ctcsound a possible solution to the issues we are having with Python opcodes and Live on OSX? On 8 November 2016 at 17:52, Oeyvind Brandtsegg <oyvind.brandtsegg@ntnu.no> wrote: Great, thankl you so much for the help. |
Date | 2016-11-08 19:28 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
That will be one thing to look into very soon, maybe those problems will just vanish... haha, we'll see. I have a few more workshops and presentations this week, after that I should be able to spend some more time on it. 2016-11-08 10:31 GMT-08:00 Rory Walsh |
Date | 2016-12-01 20:45 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
I'm looking at this in Jupyter Notebook now, and it looks and works great. Nice work, Francios! 2016-11-08 9:52 GMT-08:00 Oeyvind Brandtsegg |
Date | 2016-12-01 20:56 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
One minor problem remaining though, when I try to run embeddedCtcsound.csd, I get errors. The error seems to be related to import ctcsound as run within the csd. I can run import ctcsound in a python interpreter, or in IPython or Jupyter. But when I run it within a csd, like this: pyinit pyruni {{ import ctcsound }} ... I get: pyruni: python exception Traceback (most recent call last): File " |
Date | 2016-12-01 21:02 |
From | Oeyvind Brandtsegg |
Subject | Re: Optimization tip wanted for Csound/Python |
... oh, sorry for the noise, I found it. I edited embeddedCtcsound.csd so that it reads: pyruni {{ import sys sys.path.append('c:\\python27\\DLLs') import ctcsound instead of pyruni {{ import ctcsound import sys and it works nicely. I guess this is a Windows only issue, but I've sometimes encountered the need for a similar sys.path.append(os.getcwd()) on OSX (when running python in a csd under Cabbage) Should perhaps the embeddedCtcsound.csd in the ctcsound repo be edited to somehow include these platform-specific additions? best Oeyvind 2016-12-01 12:56 GMT-08:00 Oeyvind Brandtsegg |