[Cs-dev] JavaScript API
Date | 2015-07-02 18:58 |
From | Michael Gogins |
Subject | [Cs-dev] JavaScript API |
Attachments | None None |
I need input regarding the JavaScript interfaces to Csound. My goal is to unify the signatures of the most important JavaScript functions across all of the interfaces, and also make sure that the Csound object is named "csound" in all the interfaces. This will enable a piece developed for one JavaScript interface to run the same way in all the other JavaScript interfaces. I am aware that different interfaces may do somewhat different things using the same names, or use different names for doing the same things, or that callbacks are used in some interfaces but not in others. Again, my objective here is to ensure that all the interfaces support the functions in the "Objective" column (without removing code that already exists), and the contents of this column are open to change. These functions should all be synchronous and should all use only elementary data types. (1) Currently, the "csound" object is instantiated by the host and injected into the user's JavaScript context. In other words, "csound" is a static singleton, as far as the JavaScript context is concerned. Is it important to be able to create a new instance of Csound FROM JavaScript? (2) In the Csound C++ API we have the naming convention that all functions start with an initial capital letter, e.g. ReadScore. In most of the JavaScript interfaces, most functions start with an initial lower-case letter, e.g. readScore. What is your preference: (a) lower case, (b) upper case, (c) both lower and upper case? (3) I am assuming that any audio streams and/or MIDI messages communicated between the JavaScript context and Csound itself will be accomplished via Web Audio as in Emscripten and/or Web MIDI, not by use of lower level Csound APIs. Is this reasonable? Of course Csound itself may still be able to directly access audio and MIDI ports on the computer. (4) Please let me know what functions you would like to add to or remove from the list in the "Objective" column, i.e. from the common JavaScript interface. (5) The "Perform" function in the JavaScript interfaces runs in a separate thread, similar to CsoundPerformanceThread. This function is called after the orchestra is compiled to actually perform the score and/or handle real-time events. No code whatsoever in this thread communicates directly with the JavaScript context, all such communication is via thread-safe FIFOs. As background for your thoughts, please consider that in some but not all HTML/JavaScript environments (e.g. the Chromium Embedded Framework used in CsoundQt), the browser runs in one process ("browser"), and the actual user interface and JavaScript virtual machine run in another process ("renderer"); these processes must communicate using an IPC channel and this contributes overhead to Csound API function calls. Also consider that JavaScript is single-threaded, and can simulate multi-threading only with Web Workers or libuv threads that also need IPC to communicate. Of course, Csound itself, isolated from the JavaScript VM, can run multiple threads. Regards, Mike ----------------------------------------------------- Michael GoginsIrreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com |
Date | 2015-07-03 16:08 |
From | Rory Walsh |
Subject | Re: [Cs-dev] JavaScript API |
Hi Mike. This is quite extensive. I agree that a common naming system would make life easier for everyone. I've my own preferences for naming convention, which more or less follow the ones used in the C++ wrapper. However, I'm fine with any convention so long as it's consistent across platforms. While we are discussing this, I wonder if the same conventions and renaming could be applied to all wrappers? Java, C++, Python etc. Having a consistent naming scheme also means simplified docs for each wrapped interface, but perhaps this is part of a much larger project. People would have to update their code, but as far as I understand it, it would not need an API bump as the underlying C functions would remain the same. On 2 July 2015 at 18:58, Michael Gogins |
Date | 2015-07-03 17:00 |
From | Michael Gogins |
Subject | Re: [Cs-dev] JavaScript API |
Attachments | None None |
Thanks for your response. This is a complex issue. In my experience it is not possible to design an interface without experience using that interface. Catch-22. People use the interface in different ways on different platforms, so an interface defined by one developer on one platform for one set of purposes will not be suitable for everyone on all platforms. In addition, it is all too common for developers to re-invent the wheel if something that already works is not quite to their taste, and Csound developers are no exception. That said, in my view, the best way to define an interface is to leave the existing interfaces in place and define an abstract interface (in C++, that would be a class with all virtual functions none of which are defined). The csound.hpp file comes close to this, even though it is a concrete class. Then this file would be inherited by a concrete class on each platform, and would be exposed by SWIG for Python and Lua, by ctypes or cffi for Python or Lisp, or by being used to implement native JavaScript functions of the same type. My objective here is simply to provide the core of the Csound API in a way that usable in the same way across all the JavaScript platforms. And it should be usable by inexperienced programmers. That means as far as possible, it should avoid complex data types and callbacks. I have no objection to also supporting more complex data types (channel pointer, CsoundBindings, etc.) which are more useful for writing finished applications, but I think this would be cleaner if it were in a subclass of this base API interface. The csound.hpp interface comes close, but the JavaScript interface needs the Csound performance loop to run in a separate thread or Csound will hog the browser process. I could introduce a new function to this class to run this thread, and maybe I should. The easiest way to implement it uses c++11 or c++14 threads and boost::lockfree::queue, which are a bit more recent that the coding standard used by most of Csound. Another addition that I will definitely make is to run a csd file as a string, in addition to being able to run a filename. Last time I checked, Cabbage uses Csound.hpp and csPerformanceThread.hpp, correct? How much does Cabbage use callbacks to interact with Csound? Or do you either poll, or use the csPerformanceThread channels etc.? Regards, Mikr ----------------------------------------------------- Michael GoginsIrreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com On Fri, Jul 3, 2015 at 11:08 AM, Rory Walsh <rorywalsh@ear.ie> wrote: Hi Mike. This is quite extensive. I agree that a common naming system |
Date | 2015-07-03 18:21 |
From | Steven Yi |
Subject | Re: [Cs-dev] JavaScript API |
I've added comments to issue #328 that tracks unifying the API's [1]. I have proposed we implement the CsoundObj API design in JS so that it matches up with Android and iOS, which would simplify cross-platform application development that targets these platforms. CsoundObj is by design platform-specific, wrapping the core libcsound API, which is by design portable. CsoundObj's design addresses the problem of platforms that do not offer runtime dynamic library loading of plugins. With desktop Csound, we encapsulate hardware and other I/O within plugins. Without the ability to load plugins, the onus to handle at least audio I/O is put on the user of libcsound. Both of the web versions (Emscripten/PNACL) of Csound currently start off with libcsound's API as a base. From there they wrap with their own API's. In essence, they do the same things that CsoundObj does on Android and iOS, handling gluing libcsound to the audio system as well as other I/O. CsoundObj also provides facilities for adding synchronously executed Bindings for values over channels. These are however optional, and users can do standard message-based asynchronous read/writes to Csound using CsoundObj. CsoundObj also provides a fallback mechanism to expose the underlying libcsound API. To note, Bindings would currently work with Emscripten but not PNaCl, and when AudioWorkers (or whatever they're going to call it since things are in flux again there and it's not really a Worker anymore) are implemented, Bindings won't be possible on either Emscripten or PNaCl. That part of CsoundObj may very well have to be left out, but that does not affect the other aspects of CsoundObj's design. (I suppose a message-based binding system could be put together, but it's behavior would be different than on other systems.) >From my comments on #328, I'm not sure if it's clear, but I think as long as a CsoundObj API can be implemented on top of whatever is going to be provided via Emscripten/PNaCl, then that's fine. If CsoundObj is what is provided out of the box, great, if not, as long as it can be implemented on top of what is provided, then users can avail themselves of that design across web and mobile platforms. Regarding inheritance and interfaces, I do think designing towards interfaces is generally good. However, I think things like a Csound class don't need to be interfaces and inheritance isn't the best option over object composition. For example, in Android, CsoundObj has a Csound class member, and does not subclass Csound. [1] - https://github.com/csound/csound/issues/328 On Fri, Jul 3, 2015 at 12:00 PM, Michael Gogins |
Date | 2015-07-03 21:28 |
From | Michael Gogins |
Subject | Re: [Cs-dev] JavaScript API |
Attachments | None None |
Thanks for your response. I agree that actual inheritance of a class or interface is not necessary, although it would enforce the contract that I am proposing. The only issue that I have at this point is the Csound class member. Just to make it crystal clear, I want a single interface (object or class) that includes as member functions all of the functions in the "Objectives" column of my spreadsheet, which are all synchronous class member functions that take only elementary types as arguments and return only elementary data types, and do not involve callbacks. So the obvious solution is just to add these functions to CsoundObj on all platforms that use Csound Obj. This core API has several purposes: (1) To simplify the use of the Csound API for inexperienced programmers. (2) To enable a piece that uses this core API to run in all of Csound's JavaScript environments without modification. The second purpose is why using a Csound class member in CsoundObj would not work, because on NW.js and CsoundQt there is no CsoundObj and the JavaScript functions are implemented directly in native code as v8 callbacks. So a piece written for CsoundObj's interface that depends on the Csound class member would need modification to run in CsoundQt or NW.js. This is exactly what I don't want. Best, Mike ----------------------------------------------------- Michael GoginsIrreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com On Fri, Jul 3, 2015 at 1:21 PM, Steven Yi <stevenyi@gmail.com> wrote: I've added comments to issue #328 that tracks unifying the API's [1]. |
Date | 2015-07-04 00:56 |
From | Steven Yi |
Subject | Re: [Cs-dev] JavaScript API |
I'm not sure I understand some aspects of this. Can you not load regular JS files when in CsoundQT or NW.js? If you can, then you can just load a CsoundObj.js file that wraps whatever is exposed from V8 side. That way, you could use what is exposed from V8, or use CsoundObj.js. If you match up all of the functions across JS implementations, then you can run your HTML/CSD across them. If you write against CsoundObj, you also get clear guidance on what to do if you want to then port your applications to Android, iOS, or OSX. As I mentioned, I think this can be two separate things: CsoundObj written on top of what is exposed by implementations. What you're talking about seems to be matching up all of the implementations. I'm saying we should offer CsoundObj on top of that. The thing to note about CsoundObj is that it's built on top of the portable Csound API that's offered for that platform. For Android, it's built on top of the Java Csound class, and on iOS, it's built on top of the Csound C API. Users *could* just use the lower-level Csound API, but then they'd have to do all of the platform-specific stuff themselves (i.e., I/O for audio between Csound and audio system, MIDI, how to bind GUI items to channels, etc.). Instead, CsoundObj includes the common cases for the platform-specific code, and delegates the portable stuff to the lower level Csound API's. So in that regards, the portable stuff (the items in the Objectives column) would be in the classes provided by Emscripten, PNaCl, NW.js, etc. And again, CsoundObj builds on top that, and lets the user do what they want with the portable lower-level API. Question: why don't we just have a Csound class that has all of the same named methods as that found in the SWIG interface wrappers? That would address your objectives, yes? If that happens, adding CsoundObj on top would be fairly easy. On Fri, Jul 3, 2015 at 4:28 PM, Michael Gogins |
Date | 2015-07-04 20:53 |
From | Steven Yi |
Subject | Re: [Cs-dev] JavaScript API |
Attachments | None None |
Hi Michael, Just to add another datapoint, I pasted at the end of this email a python session that list of methods currently available on the Csound class from Python. I went through the spreadsheet again and the methods listed in the Objectives column seem good to me for a target set of functions to have with one exception in that I would like reset() to be added. (With usage we can add more methods if requested.) For CsoundObj, it can sit on top of this just fine. Going back to your first email, there's a question about singletons. I think from doing the Csound Notebook, with PNaCl I had some issues in that there was no reset function. Because of that, when I switched from one project to another, I wasn't able to start from a clean project state to run the new project. I could do this with Emscripten though (I can't recall what function I used though). For a singleton, I'm split on that. Part of me think it shouldn't be a singleton on principle, but in real world usage it's probably just fine. To note, one of the goals for a merged API was to be able to load a single csound.js file that would choose whether to use PNaCl or Emscripten, depending on what capabilities are present (i.e., on Chrome or not). I imagine that having the singletons shouldn't affect that goal, but just wanted to note that. Thanks, steven Steven-Yis-MacBook-Pro-249:cs6 stevenyi$ python Python 2.7.6 (default, Sep 9 2014, 15:04:36) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import csnd6 >>> a = csnd6.Csound() virtual_keyboard real time MIDI plugin for Csound 0dBFS level = 32768.0 Csound version 6.05 (double samples) Jun 28 2015 libsndfile-1.0.25 >>> dir(a) ['AddSpinSample', 'AppendOpcode', 'Cleanup', 'Compile', 'CompileArgs', 'CompileCsd', 'CompileOrc', 'CompileTree', 'CreateGlobalVariable', 'CreateMessageBuffer', 'DeleteChannelList', 'DeleteTree', 'DeleteUtilityList', 'DestroyGlobalVariable', 'DestroyMessageBuffer', 'DisposeOpcodeList', 'EvalCode', 'Get0dBFS', 'GetAPIVersion', 'GetAudioChannel', 'GetChannel', 'GetChannelPtr', 'GetControlChannelHints', 'GetCsound', 'GetDebug', 'GetEnv', 'GetFirstMessage', 'GetFirstMessageAttr', 'GetInputBuffer', 'GetInputBufferSize', 'GetKr', 'GetKsmps', 'GetMessageCnt', 'GetMessageLevel', 'GetNchnls', 'GetNchnlsInput', 'GetOutputBuffer', 'GetOutputBufferSize', 'GetOutputName', 'GetParams', 'GetRtPlayUserData', 'GetRtRecordUserData', 'GetScoreOffsetSeconds', 'GetScoreTime', 'GetSpin', 'GetSpout', 'GetSpoutSample', 'GetSr', 'GetStringChannel', 'GetTable', 'GetUtilityDescription', 'GetVersion', 'InitializeCscore', 'InputMessage', 'IsScorePending', 'KeyPressed', 'ListChannels', 'ListUtilities', 'Message', 'MessageS', 'NewOpcodeList', 'ParseOrc', 'Perform', 'PerformBuffer', 'PerformKsmps', 'PopFirstMessage', 'PvsinSet', 'PvsoutGet', 'QueryGlobalVariable', 'QueryGlobalVariableNoCheck', 'ReadScore', 'Reset', 'RewindScore', 'RunUtility', 'ScoreEvent', 'ScoreEventAbsolute', 'ScoreExtract', 'ScoreSort', 'SetChannel', 'SetControlChannelHints', 'SetDebug', 'SetExternalMidiInCloseCallback', 'SetExternalMidiInOpenCallback', 'SetExternalMidiReadCallback', 'SetGlobalEnv', 'SetHostImplementedAudioIO', 'SetHostImplementedMIDIIO', 'SetInput', 'SetInputChannelCallback', 'SetMIDIFileInput', 'SetMIDIFileOutput', 'SetMIDIInput', 'SetMIDIOutput', 'SetMessageCallback', 'SetMessageLevel', 'SetOption', 'SetOutput', 'SetOutputChannelCallback', 'SetParams', 'SetScoreOffsetSeconds', 'SetScorePending', 'Start', 'Stop', 'TableCopyIn', 'TableCopyOut', 'TableGet', 'TableLength', 'TableSet', '__class__', '__del__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattr__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__swig_destroy__', '__swig_getmethods__', '__swig_setmethods__', '__weakref__', 'pydata', 'this'] On Fri, Jul 3, 2015 at 7:56 PM, Steven Yi <stevenyi@gmail.com> wrote: I'm not sure I understand some aspects of this. Can you not load |
Date | 2015-07-04 21:45 |
From | Michael Gogins |
Subject | Re: [Cs-dev] JavaScript API |
Attachments | None None |
That's great. It's not at all impossible or even that difficult to not use singletons, but we can start with a singleton and add creators and destructors. "For CsoundObj, it can sit on top of this just fine." This seems ambiguous to me. Do you mean that CsoundObj can be implemented using the "Objectives" functions, or that the "Objectives" functions can simply be added to CsoundObj? I raise this issue because the assumption I am making is that there will be an object name "csound" in the JavaScript namespace that has the "Objectives" functions, but it seems to me that the default instance of CsoundObj might already be named "csound." Best, Mike ----------------------------------------------------- Michael GoginsIrreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com On Sat, Jul 4, 2015 at 3:53 PM, Steven Yi <stevenyi@gmail.com> wrote:
|
Date | 2015-07-04 21:58 |
From | Steven Yi |
Subject | Re: [Cs-dev] JavaScript API |
Attachments | None None |
At this point, I'm thinking: * Csound w/ Objectives Function * CsoundObj => has Csound So the same setup as what is in Android/iOS with the Csound member. The CsoundObj class itself isn't the whole API, there's also the CsoundMIDI, CsoundUI, and CsoundMotion classes too. There may be something that can be done for Bindings, just perhaps not with the same synchronous behavior. So the idea as mentioned is: * User wants to write a Project that runs across JS implementations => Just use Csound class * User wants to write a Project that goes across JS, Android, iOS, OSX => Look at CsoundObj If there's a default instance of CsoundObj, it would not take the place of a default instance of Csound, but rather wrap it (so there'd be two singletons essentially). On Sat, Jul 4, 2015 at 4:45 PM, Michael Gogins <michael.gogins@gmail.com> wrote:
|
Date | 2015-07-04 22:04 |
From | Michael Gogins |
Subject | Re: [Cs-dev] JavaScript API |
Attachments | None None |
OK, let me see if I understand you. Where there is a singleton instance of CsoundObj, there also is a singleton instance of Csound (the "Objectives" object). This is the one that is named "csound," not CsoundObj. That can be named "csoundObj" or something. In this way, on all JS platforms, users can count on code that uses the "csound" object working the same way on all platforms without editing. Is that it? Best, Mike ----------------------------------------------------- Michael GoginsIrreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com On Sat, Jul 4, 2015 at 4:58 PM, Steven Yi <stevenyi@gmail.com> wrote:
|
Date | 2015-07-04 22:16 |
From | Steven Yi |
Subject | Re: [Cs-dev] JavaScript API |
Attachments | None None |
Yes, that is what I'm thinking. It keeps the portable API (Csound) separate from the platform-specific API (CsoundObj). I imagine there will be some crossover to accommodate issues with JS architecture (some platform-specific things may end up getting into the Csound singleton), but as long as the interface of CsoundObj can be implemented, that's fine I think. Ideally, I'd prefer to see the split be exactly down the portable/platform-specific line. That gets closer parity to the architecture of what is on the desktop platforms. But what's offered with web and JS doesn't really make that possible, so it's something of a compromise. On Sat, Jul 4, 2015 at 5:04 PM, Michael Gogins <michael.gogins@gmail.com> wrote:
|