Csound Csound-dev Csound-tekno Search About

Re: [Cs-dev] Possible Optimization

Date2006-11-03 08:33
FromVictor Lazzarini
SubjectRe: [Cs-dev] Possible Optimization
That is actually the solution that is given by
CsoundPerforamnceThread: this is a C++ class
(with Python bindings) that allows a csound performance
to be run on a different thread. I use it all the
time.

Victor

>
> from the Pythonmac-SIG guys re whether or not the python
> threading   module can make use of multiple processors:
>
> > It can, but not always optimally. Python has a global
> > interpreter   lock (GIL) that is held whenever a thread
> > is executing python byte   code. This means only one
> > thread at a time can be executing byte   code, however
> multiple threads can be in C code at the same time. >
> > Therefore it is possible to use multiple CPUs, but only
> > if you have   C code that  does some calculation without
> > holding the GIL.  Note   that several GUI API's from
> > Apple, such as CoreImage make automatic   use of
> > multiple processors, and even if you don't use multiple
>  processors yourself the other processor won't be
> > completely idle,   that can and will be used for
> background tasks and other programs. >
> > If you want to make full use of multiple processors
> > using pure   python code you'll have to use multiple
> processes.
>
> > Ronald
>
>
> On 30/10/2006, at 8:19 PM, Steven Yi wrote:
>
> > Hi Michael,
> >
> > I can understand to make things as simple as possible. I
> > don't know yet how much this might affect performance
> > but I have the feeling at the moment that it would not
> > be negligible.  But also, I don't know if certain things
> > will be possible; there were messages here about
> threading in Python and that it can't take advantage of
> > multiple processors, which if the threaded handling was
> > done in C, it would be able to do so (I don't know much
> > about Python's threading model so please correct me if
> > I'm wrong).  So, this would complicate things, but it
> also might be the only way to implement this in a fashion
> > that all languages can really take advantage of it.
> >
> > Basically, I'm thinking that Csound should be as
> > performant when used by the API in a non-C/C++ language
> > as it is in C or C++, and I think that one of the ways
> > that happens to slow down now is in the messaging.
> >
> > However, I *do* want to verify this all first with tests
> > before doing anything. If messaging performance is
> > reasonably performant using a threaded solution in a
> > host language (specifically python I'm curious about
> here) or host program or in the end doesn't gain enough to
> > make a difference, then this whole thread (pun intended!
> > =) is a moot question.
> >
> > steven
> >
> >
> > On 10/30/06, Michael Gogins  wrote:
> >> I think it should stay the same as it is now -- hosts
> should set   >> up async logging if they want. It's not
> hard to do. >>
> >> One of the things I am constantly worrying about with
> Csound is   >> introducing unnecessary complexity. We need
> to balance introducing   >> new features with their cost
> in terms of maintenance and external   >> dependencies and
> so on, with their benefits. >>
> >> I think Csound is simpler in some ways now than it was,
> and more,   >> much more complex than it used to be in
> other ways. I would like   >> to simplify certain things.
> I bet Istvan has simplified some things. >>
> >> Regards,
> >> Mike
> >>
> >> -----Original Message-----
> >>> From: Steven Yi 
> >>> Sent: Oct 29, 2006 6:08 PM
> >>> To: Developer discussions
>  >>> Subject: Re:
> [Cs-dev] Possible Optimization >>>
> >>> Hi Matt,
> >>>
> >>> What you said was basically what I was thinking, so
> we're   >>> somewhat on
> >>> the same page. As far as I understand, right now
> within Csound the >>> handling is synchronous, so when an
> opcode or the engine calls >>> csound->Message, by default
> if no message callback is set by a host, >>> the default
> message callback is used and that calls vfprintf to   >>>
> output >>> to stderr directly.
> >>>
> >>> In MacCsound or Csound5GUI, the callback given to
> Csound is then   >>> used
> >>> for Message, but the asynchronous part is then handled
> in the host. >>> The opcodes call Message, the host
> callback adds the message to its >>> buffer, and the lower
> priority thread in the host is what does   >>> the end
> >>> printing.
> >>>
> >>> I was suggesting a flag because i thought that if
> Csound was changed >>> to do the low priority thread thing
> inside, it might be less than >>> optimal if that's going
> on and you already have the same thing in   >>> the
> >>> host.
> >>>
> >>> So, if the flag is off, the asynchronous handling via
> thread and >>> buffer is implemented by a host.  If flag
> is on, it gets a little >>> complicated, but there would
> have to be an extra message callback >>> stored somewhere.
>  All of the opcodes and internal code call   >>> Message,
> >>> but what would have to happen would be that Message
> would need to   >>> be a
> >>> function that adds to the buffer, then the low
> priority thread would >>> have to grab from the buffer and
> call the second stored Message >>> callback (either the
> default vfprintf one or a host provided one). >>> The low
> priority thread would also have to be started very early
> in >>> the csound compilation process to get all messages
> (I guess whenever >>> setMessageCallback is normally set,
> at that same step in the >>> compilation process).
> >>>
> >>> So, take for instance you're in python. The host sets
> a callback   >>> to do
> >>> something like append to a text field.  With the flag,
> csound would >>> store messages in a buffer internally and
> the low priority thread >>> would call the host callback.
> If that callback is slow (which it >>> might be with
> python or other scripting language with all the method >>>
> overheads involved), the it'll only slowdown the low
> priority   >>> thread,
> >>> not the csound main loop.
> >>>
> >>> If you don't use the flag and you're in a scripting
> language,   >>> then you
> >>> can set a callback and do the whole threading thing in
> the scripting >>> language, but that callback, even if
> it's just to add to a linked >>> list, isn't going to be
> as fast as in C, and perhaps neither will   >>> the
> >>> thread in the scripting language be processing as
> quickly as in   >>> C.  So
> >>> it tightens up everything to do as much in C as
> possible I think to >>> let the main audio loop go as
> quickly as possible. >>>
> >>> But I was thinking that the situation like csound5gui
> and MacCsound, >>> you're already handling it in an
> optimal way, and just adding it   >>> into
> >>> csound would only slow it down, which is why I was
> suggesting a   >>> flag.
> >>>
> >>> Does that sound alright or is there something I'm
> missing that this >>> wouldn't work out?
> >>> steven
> >>>
> >>>
> >>> On 10/29/06, matt ingalls  wrote:
> >>>> i probably don't really understand, but
> >>>> i would assume the asynchronous handling be in the
> >>>> default message routine, which would then no be used
> >>>> if a host app registers a message callback?
> >>>> [ so no need for a commandline flag ]
> >>>>
> >>>> On Oct 29, 2006, at 2:44 AM, Steven Yi wrote:
> >>>>
> >>>>> Hi Matt,
> >>>>>
> >>>>> Thanks for the reply!  I'm beginning to think that
> it's pretty >>>>> important to have asynchronous handling
> for the Message callback, >>>>> especially when rendering
> in realtime.  I think I'll experiment >>>>> sometime with
> the commandline to see if implementing a threaded   >>>>>
> system >>>>> there will help out for blue and also in
> realtime CSD's.  If it   >>>>> does
> >>>>> indeed improve performance, would it be an issue to
> you or   >>>>> ayone else
> >>>>> here if it was put into the core and disabled by
> default but >>>>> enable-able (I've just coined a new word
> , I know :P ) with a   >>>>> flag?  It
> >>>>> seems like it would be tedious to implement for API
> users in   >>>>> non-C/C++
> >>>>> languages.
> >>>>>
> >>>>> BTW: Just as an experiment in Java, I found that the
> way to   >>>>> implement
> >>>>> a MessageCallback is to use the
> CsoundCallbackWrapper as in below: >>>>>
> >>>>>         Csound csound = new Csound();
> >>>>>         CsoundCallbackWrapper cw = new
> CsoundCallbackWrapper  >>>>> (csound) {
> >>>>>             public void MessageCallback(int attr,
> String msg) { >>>>>
> System.out.println(">>" + msg); >>>>>             }
> >>>>>         };
> >>>>>         cw.SetMessageCallback();
> >>>>>
> >>>>>         csound.Compile("/work/blue/sineTest.csd",
> "-Wdo", "dac"); >>>>>         csound.Perform();
> >>>>>
> >>>>> Learning something every day! =)
> >>>>>
> >>>>> steven
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 10/29/06, matt ingalls  wrote:
> >>>>>>
> >>>>>>
> >>>>>> On Oct 28, 2006, at 4:12 PM, Steven Yi wrote:
> >>>>>>
> >>>>>>
> >>>>>> Do any hosts currently implement setting their own
> >>>>>>
> >>>>>> message callbacks to work in an asynchronous
> manner? >>>>>>
> >>>>>> yes - i have a circular buffer the csoundMessage
> callback   >>>>>> writes to,
> >>>>>> and then a lower priority thread handles printing
> the buffer. >>>>>>
> >>>>>>
> >>>>>>
> ----------------------------------------------------------
> -------  >>>>>> ----
> >>>>>> ----
> >>>>>> Using Tomcat but need to do more? Need to support
> web services, >>>>>> security?
> >>>>>> Get stuff done quickly with pre-integrated
> technology to make   >>>>>> your
> >>>>>> job
> >>>>>> easier
> >>>>>> Download IBM WebSphere Application Server v.1.0.1
> based on Apache >>>>>> Geronimo
> >>>>>> http://sel.as-us.falkag.net/sel?
> >>>>>> cmd=lnk&kid=120709&bid=263057&dat=121642
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Csound-devel mailing list
> >>>>>> Csound-devel@lists.sourceforge.net
> >>>>>>
> https://lists.sourceforge.net/lists/listinfo/csound-devel
> >>>>>> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> ----------------------------------------------------------
> --------  >>>>> ----
> >>>>> ---
> >>>>> Using Tomcat but need to do more? Need to support
> web services, >>>>> security?
> >>>>> Get stuff done quickly with pre-integrated
> technology to make your >>>>> job easier
> >>>>> Download IBM WebSphere Application Server v.1.0.1
> based on Apache >>>>> Geronimo
> >>>>> http://sel.as-us.falkag.net/sel?
> >>>>> cmd=lnk&kid=120709&bid=263057&dat=121642
> >>>>> _______________________________________________
> >>>>> Csound-devel mailing list
> >>>>> Csound-devel@lists.sourceforge.net
> >>>>>
> https://lists.sourceforge.net/lists/listinfo/csound-devel
> >>>>> >>>>
> >>>>
> >>>>
> ----------------------------------------------------------
> ---------  >>>> ------
> >>>> Using Tomcat but need to do more? Need to support web
> services,   >>>> security?
> >>>> Get stuff done quickly with pre-integrated technology
> to make   >>>> your job easier
> >>>> Download IBM WebSphere Application Server v.1.0.1
> based on   >>>> Apache Geronimo
> >>>> http://sel.as-us.falkag.net/sel?
> >>>> cmd=lnk&kid=120709&bid=263057&dat=121642
> >>>> _______________________________________________
> >>>> Csound-devel mailing list
> >>>> Csound-devel@lists.sourceforge.net
> >>>>
> https://lists.sourceforge.net/lists/listinfo/csound-devel
> >>>> >>>
> >>>
> ----------------------------------------------------------
> ----------  >>> -----
> >>> Using Tomcat but need to do more? Need to support web
> services,   >>> security?
> >>> Get stuff done quickly with pre-integrated technology
> to make   >>> your job easier
> >>> Download IBM WebSphere Application Server v.1.0.1
> based on Apache   >>> Geronimo
> >>> http://sel.as-us.falkag.net/sel?
> >>> cmd=lnk&kid=120709&bid=263057&dat=121642
> >>> _______________________________________________
> >>> Csound-devel mailing list
> >>> Csound-devel@lists.sourceforge.net
> >>>
> https://lists.sourceforge.net/lists/listinfo/csound-devel
> >> >>
> >>
> >>
> >>
> ----------------------------------------------------------
> -----------  >> ----
> >> Using Tomcat but need to do more? Need to support web
> services,   >> security?
> >> Get stuff done quickly with pre-integrated technology
> to make your   >> job easier
> >> Download IBM WebSphere Application Server v.1.0.1 based
> on Apache   >> Geronimo
> >> http://sel.as-us.falkag.net/sel?
> >> cmd=lnk&kid=120709&bid=263057&dat=121642
> >> _______________________________________________
> >> Csound-devel mailing list
> >> Csound-devel@lists.sourceforge.net
> >>
> https://lists.sourceforge.net/lists/listinfo/csound-devel
> >> >
> >
> ----------------------------------------------------------
> > ------------  ---
> > Using Tomcat but need to do more? Need to support web
> > services,   security?
> > Get stuff done quickly with pre-integrated technology to
> > make your   job easier
> > Download IBM WebSphere Application Server v.1.0.1 based
> > on Apache   Geronimo
> > http://sel.as-us.falkag.net/sel?
> > cmd=lnk&kid=120709&bid=263057&dat=121642
> > _______________________________________________
> > Csound-devel mailing list
> > Csound-devel@lists.sourceforge.net
> >
> https://lists.sourceforge.net/lists/listinfo/csound-devel
> >
>
> _______________________________________
> experimental polymedia:    www.avatar.com.au
> Sonic Communications Research Group,
> University of Canberra:     www.canberra.edu.au
>
>
>
>
>
> ----------------------------------------------------------
> --------------- Using Tomcat but need to do more? Need to
> support web services, security? Get stuff done quickly
> with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on
> Apache Geronimo
>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net