[Cs-dev] multithreading py opcodes
| Date | 2014-02-11 23:09 |
| From | Oeyvind Brandtsegg |
| Subject | [Cs-dev] multithreading py opcodes |
Hi, I am trying to make some algoritmic composition techniques available for realtime use as midi VST plugins. I can make the VST plugin with Cabbage, and simple operations on midi data works well. I process the data in python, using the py opcodes. This also works well inside Cabbage. All good so far However, some of the more advanced processing I'd like to will in many cases not be able to finish in one ksmps block and it creates dropouts, and this of course affect processing in the VST host program too, so not a good situation. To avoid audio dropouts, I can live with some algorithmically generated notes being late. So I tried setting up the python processing of note data in its own thread so that: 1. Csound instrument 1 sends note data to python, put the data in a input queue and returns immediately 2. Python has a separate thread that consumes items from the input queue, process it, and put the result in a separate output queue 3. Csound instrument 2 use python opcodes to poll the output queue, and generate new Csound instrument events for any items found. I've tested the threading and it works well when I run it in Python, however, it behaves more strangely when I run it from Csound. It simply seems as if the worker thread is running extremely slow, when python runs under Csound. So the technical question is: Is the py engine in Csound implemented so that multithreading is not available? I know that Python does not allow true multithreading due to the global interpreter lock, but this will be ok for my use, I just want Python to be able to do a relaxed processing of input data without blocking Csound. There is plenty of free time between midi events, where it can complete a calculation and then offer the data to Csound when ready. I have used the Python threading class so far. Perhaps I should try with the multiprocessing module, which apparantly can bypass the global interpreter lock. But I figured I should ask if the whole thing is futile before going further. best Oeyvind -- Oeyvind Brandtsegg Professor of Music Technology NTNU 7491 Trondheim Norway Cell: +47 92 203 205 http://flyndresang.no/ http://www.partikkelaudio.com/ http://soundcloud.com/brandtsegg http://soundcloud.com/t-emp ------------------------------------------------------------------------------ Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk _______________________________________________ Csound-devel mailing list Csound-devel@lists.sourceforge.net |
| Date | 2014-02-11 23:27 |
| From | Michael Gogins |
| Subject | Re: [Cs-dev] multithreading py opcodes |
| Attachments | None None |
First of all Python itself is not truly multi-threaded thanks to its global interpreter lock (GIL). The "thread" facilities and the GIL enable co-operative multitasking, not true multi-threading. Only one thread at a time may access the Python interpreter. Period. If you want two Python tasks to run concurrently, then you must either (a) run two processes, one for each task, or (b) use Python only to interface to some library that does support concurrent processing. Once Python calls into such a library, that library can run as many threads as it likes and the call may immediately return to Python.
The same things exactly are true of Lua, LuaJIT, and other dynamic languages with which I am familiar. You may wish to consider switching to Lua because, although also single-threaded, its single thread can be roughly as fast as C whereas Python is going to be roughly 1/30th as fast as C.
Or you may consider using PyProcessing or some other library for running Python tasks in concurrent processes. But then you will have to use some kind of networking and queues to tie things together.
The problems you are having may be due to your Python code waiting on the GIL when it does not really need to. Hope this helps, Mike ----------------------------------------------------- Michael GoginsIrreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com On Tue, Feb 11, 2014 at 6:09 PM, Oeyvind Brandtsegg <oyvind.brandtsegg@ntnu.no> wrote: Hi, |
| Date | 2014-02-12 00:09 |
| From | Victor Lazzarini |
| Subject | Re: [Cs-dev] multithreading py opcodes |
I was experimenting with pyuni/pyrun and it appears that thread is only executing while the opcode
runs. As soon as it exits, the thread is blocked, so maybe that’s the problem.
I think this makes sense, because the interpreter is only invoked to run the code and then exits,
so it can’t keep running any background thread.
pyinit
pyruni "import thread, time"
schedule 1, 0, 25
instr 1
pyruni {{
def test(text):
n = 0
g += 1
while(n < 10):
print "%s: %d" % (text, n)
time.sleep(1)
n+=1
a = thread.start_new(test, ("thread",))
time.sleep(5)
}}
printk2 int(times:k())
endin
On 11 Feb 2014, at 23:09, Oeyvind Brandtsegg |
| Date | 2014-02-12 05:36 |
| From | thorin kerr |
| Subject | Re: [Cs-dev] multithreading py opcodes |
| Attachments | None None |
Just speculating: Does pyruni run at i-time? Does --realtime-mode run i-time code in a separate thread? Could then the thread be kept alive in Csound without dropouts? Thorin On Wed, Feb 12, 2014 at 10:09 AM, Victor Lazzarini <Victor.Lazzarini@nuim.ie> wrote: I was experimenting with pyuni/pyrun and it appears that thread is only executing while the opcode |
| Date | 2014-02-12 08:44 |
| From | Oeyvind Brandtsegg |
| Subject | Re: [Cs-dev] multithreading py opcodes |
Ah, ok, thanks for all hints, I do understand the problem a bit better now.
So, it might actually be a good idea to try using the multiprocessing
module, bypassing the GIL., it seems.
But, when I I try to use it I get an error:
AttributeError: 'module' object has no attribute 'argv'
which might stem from the same issue raised when I do
instr 1
pyruni {{
import sys
print sys.argv
}}
endin
... so does the Python interpreter as invoked by Csound not have an
arg attribute?
2014-02-12 1:09 GMT+01:00 Victor Lazzarini |