Is there WebAssembly support for Csound in Unity yet?

Is there WebAssembly support for Csound in Unity yet?

Webassembly Csound already compiles to a single JS file that embeds
the WASM binary into it. I assume this would work with Unity but am
unsure. I think you would only need the dist/csound.js file. You could
also try using the one from 6.18.7:

https://unpkg.com/browse/@csound/browser@6.18.7/

It might be possible to rename it .jspre and create a test .jslib that
uses the Csound API.

On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
<giovanni.bedetti@gmail.com> wrote:
>
> Quick update on this: no luck yet!
>
> Sorry for the long email that follows. Any help would be much appreciated!
>
>
> Unity uses Emscripten for the WebGL build.
>
> There are 2 options using Emscripten:
>
> Compile Csound as a static library to be used as Unity plug-in
> First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
>
> Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
>
>
> I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> This should be the preferred way of creating the required files for Unity.
>
> The supported native plug-in file formats are
>
> LLVM Bitcode file format (.bc)
> GNU archive file format (.a)
>
> I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> I ran yarn build in the wasm folder.
> In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
>
> I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
>
> So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
>
> .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
>
> Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
>
> const hello = {
>     Hello: function() {
>         window.alert("Hello, world!");
>     }
> };
> mergeInto(LibraryManager.library, hello);
>
> to be called in C# like this:
>
> [DllImport("__Internal")]
>     private static extern void Hello();
>
> Bundle the C/C++ code in the Unity project
> This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> Out of curiosity, I tried to import some folders from the Csound repo:
> - Engine
> - H
> - include
> - InOut
> - interfaces
> - OOps
> - Opcodes
> - po
> - util
> - util1
> - util2
>
> When I build I get the following error, "Plugins colliding with each other.":
>
> Plugin 'mp3dec_internal.h' is used from several locations:
>  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
>  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> Plugin 'mpadec_internal.h' is used from several locations:
>  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
>  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> Plugin 'stdopcod.h' is used from several locations:
>  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>
> I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
>
> There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
>
> Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
>
> Hope someone has some tips here, I'm lost in the Web(GL)!
>
>
> Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
>>
>> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
>> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
>>
>> In the next days I will try something and report here my findings.
>>
>>
>> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
>>>
>>> Is there WebAssembly support for Csound in Unity yet?

Thanks for your replies!
Dr.B let's chat soon! I'll drop you an email ;)

I'm getting closer! I was able to load Csound! Woho!

I have created a repository with a test project and a build folder if anybody wants to take a look.

I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins. 
If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)

I tried using the other extension, .jslib, and I got this error instead:
Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',

So I renamed the file back to csound.jspre, and commented the export lines at the end.

const Csound = Csound$$module$src$index;
Csound.toString = () => 'async (options) => CsoundObj;';
//export { Csound }
//export default Csound
//# sourceMappingURL=csound.js.map

It builds and runs ok, of course no Csound yet as nothing is calling it.

So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)

mergeInto(LibraryManager.library, {

csoundInitialize: async function(flags) {

window.alert("csoundInitialize");

const csoundVariations = [

{ useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },

{ useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },

{ useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },

{ useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },

{ useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },

];

//Csound.csoundInitialize(flags); // this doesn't work

console.log("starting to await for Csound with flag: + flags")

const cs = await Csound(csoundVariations[flags]);

console.log(`Csound version: ${cs.name}`);

const startReturn = await cs.start();

console.log(startReturn);

// await cs.stop();

// cs.terminateInstance && (await cs.terminateInstance());

},

});

Then call csoundInitialize from a C# script 

void Start()

{

Debug.Log("Calling csoundInitialize");

csoundInitialize(0);

}

and bang we got it!!

TestBuild_0.framework.js:913 Calling csoundInitialize
TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:913 0

Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D 
I'll dig deeper inside this as soon as I have sound output.

So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
But of course the Start function is not waiting for it to complete before progressing.

All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence. 
Have a look at the initialization sequence of the CsoundUnityBridge
So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
Should the CsoundUnityWebGL be a completely different package?

The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).

So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound. 
I'm thinking of workarounds but have no ideas at the moment.

But for now let's focus on the fact that there was some progress! It's a win! :D

I'll keep you posted as I continue to experiment with this, 
thanks again for now!

Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:

Webassembly Csound already compiles to a single JS file that embeds
the WASM binary into it. I assume this would work with Unity but am
unsure. I think you would only need the dist/csound.js file. You could
also try using the one from 6.18.7:

https://unpkg.com/browse/@csound/browser@6.18.7/

It might be possible to rename it .jspre and create a test .jslib that
uses the Csound API.

On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
<giovanni.bedetti@gmail.com> wrote:
>
> Quick update on this: no luck yet!
>
> Sorry for the long email that follows. Any help would be much appreciated!
>
>
> Unity uses Emscripten for the WebGL build.
>
> There are 2 options using Emscripten:
>
> Compile Csound as a static library to be used as Unity plug-in
> First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
>
> Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
>
>
> I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> This should be the preferred way of creating the required files for Unity.
>
> The supported native plug-in file formats are
>
> LLVM Bitcode file format (.bc)
> GNU archive file format (.a)
>
> I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> I ran yarn build in the wasm folder.
> In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
>
> I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
>
> So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
>
> .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
>
> Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
>
> const hello = {
>     Hello: function() {
>         window.alert("Hello, world!");
>     }
> };
> mergeInto(LibraryManager.library, hello);
>
> to be called in C# like this:
>
> [DllImport("__Internal")]
>     private static extern void Hello();
>
> Bundle the C/C++ code in the Unity project
> This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> Out of curiosity, I tried to import some folders from the Csound repo:
> - Engine
> - H
> - include
> - InOut
> - interfaces
> - OOps
> - Opcodes
> - po
> - util
> - util1
> - util2
>
> When I build I get the following error, "Plugins colliding with each other.":
>
> Plugin 'mp3dec_internal.h' is used from several locations:
>  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
>  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> Plugin 'mpadec_internal.h' is used from several locations:
>  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
>  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> Plugin 'stdopcod.h' is used from several locations:
>  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>
> I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
>
> There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
>
> Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
>
> Hope someone has some tips here, I'm lost in the Web(GL)!
>
>
> Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
>>
>> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
>> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
>>
>> In the next days I will try something and report here my findings.
>>
>>
>> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
>>>
>>> Is there WebAssembly support for Csound in Unity yet?

Do they plan to add support for the OnAudioFilterRead, is it on any roadmaps? That's a fairly major part of things. 

On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <giovanni.bedetti@gmail.com> wrote:

Thanks for your replies!
Dr.B let's chat soon! I'll drop you an email ;)

I'm getting closer! I was able to load Csound! Woho!

I have created a repository with a test project and a build folder if anybody wants to take a look.

I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins. 
If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)

I tried using the other extension, .jslib, and I got this error instead:
Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',

So I renamed the file back to csound.jspre, and commented the export lines at the end.

const Csound = Csound$$module$src$index;
Csound.toString = () => 'async (options) => CsoundObj;';
//export { Csound }
//export default Csound
//# sourceMappingURL=csound.js.map

It builds and runs ok, of course no Csound yet as nothing is calling it.

So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)

mergeInto(LibraryManager.library, {

csoundInitialize: async function(flags) {

window.alert("csoundInitialize");

const csoundVariations = [

{ useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },

{ useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },

{ useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },

{ useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },

{ useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },

];

//Csound.csoundInitialize(flags); // this doesn't work

console.log("starting to await for Csound with flag: + flags")

const cs = await Csound(csoundVariations[flags]);

console.log(`Csound version: ${cs.name}`);

const startReturn = await cs.start();

console.log(startReturn);

// await cs.stop();

// cs.terminateInstance && (await cs.terminateInstance());

},

});

Then call csoundInitialize from a C# script 

void Start()

{

Debug.Log("Calling csoundInitialize");

csoundInitialize(0);

}

and bang we got it!!

TestBuild_0.framework.js:913 Calling csoundInitialize
TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:913 0

Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D 
I'll dig deeper inside this as soon as I have sound output.

So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
But of course the Start function is not waiting for it to complete before progressing.

All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence. 
Have a look at the initialization sequence of the CsoundUnityBridge
So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
Should the CsoundUnityWebGL be a completely different package?

The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).

So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound. 
I'm thinking of workarounds but have no ideas at the moment.

But for now let's focus on the fact that there was some progress! It's a win! :D

I'll keep you posted as I continue to experiment with this, 
thanks again for now!

Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:

Webassembly Csound already compiles to a single JS file that embeds
the WASM binary into it. I assume this would work with Unity but am
unsure. I think you would only need the dist/csound.js file. You could
also try using the one from 6.18.7:

https://unpkg.com/browse/@csound/browser@6.18.7/

It might be possible to rename it .jspre and create a test .jslib that
uses the Csound API.

On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
<giovanni.bedetti@gmail.com> wrote:
>
> Quick update on this: no luck yet!
>
> Sorry for the long email that follows. Any help would be much appreciated!
>
>
> Unity uses Emscripten for the WebGL build.
>
> There are 2 options using Emscripten:
>
> Compile Csound as a static library to be used as Unity plug-in
> First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
>
> Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
>
>
> I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> This should be the preferred way of creating the required files for Unity.
>
> The supported native plug-in file formats are
>
> LLVM Bitcode file format (.bc)
> GNU archive file format (.a)
>
> I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> I ran yarn build in the wasm folder.
> In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
>
> I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
>
> So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
>
> .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
>
> Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
>
> const hello = {
>     Hello: function() {
>         window.alert("Hello, world!");
>     }
> };
> mergeInto(LibraryManager.library, hello);
>
> to be called in C# like this:
>
> [DllImport("__Internal")]
>     private static extern void Hello();
>
> Bundle the C/C++ code in the Unity project
> This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> Out of curiosity, I tried to import some folders from the Csound repo:
> - Engine
> - H
> - include
> - InOut
> - interfaces
> - OOps
> - Opcodes
> - po
> - util
> - util1
> - util2
>
> When I build I get the following error, "Plugins colliding with each other.":
>
> Plugin 'mp3dec_internal.h' is used from several locations:
>  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
>  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> Plugin 'mpadec_internal.h' is used from several locations:
>  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
>  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> Plugin 'stdopcod.h' is used from several locations:
>  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>
> I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
>
> There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
>
> Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
>
> Hope someone has some tips here, I'm lost in the Web(GL)!
>
>
> Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
>>
>> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
>> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
>>
>> In the next days I will try something and report here my findings.
>>
>>
>> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
>>>
>>> Is there WebAssembly support for Csound in Unity yet?

I have no idea, this is what the docs say about this, no mention about future plans anywhere:

Because Unity uses FMODto manage audio for platforms, the Web platform supports limited audio functionality, which only includes the basic features. FMOD relies on threads, which the WebGL API doesn’t support. For this reason, Unity uses an implementation based on the internal Web Audio API, which enables the browser to handle audio playback and mixing.

Il Mar 18 Giu 2024, 13:41 Rory Walsh <rorywalsh@ear.ie> ha scritto:

Do they plan to add support for the OnAudioFilterRead, is it on any roadmaps? That's a fairly major part of things. 

On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <giovanni.bedetti@gmail.com> wrote:

Thanks for your replies!
Dr.B let's chat soon! I'll drop you an email ;)

I'm getting closer! I was able to load Csound! Woho!

I have created a repository with a test project and a build folder if anybody wants to take a look.

I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins. 
If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)

I tried using the other extension, .jslib, and I got this error instead:
Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',

So I renamed the file back to csound.jspre, and commented the export lines at the end.

const Csound = Csound$$module$src$index;
Csound.toString = () => 'async (options) => CsoundObj;';
//export { Csound }
//export default Csound
//# sourceMappingURL=csound.js.map

It builds and runs ok, of course no Csound yet as nothing is calling it.

So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)

mergeInto(LibraryManager.library, {

csoundInitialize: async function(flags) {

window.alert("csoundInitialize");

const csoundVariations = [

{ useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },

{ useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },

{ useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },

{ useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },

{ useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },

];

//Csound.csoundInitialize(flags); // this doesn't work

console.log("starting to await for Csound with flag: + flags")

const cs = await Csound(csoundVariations[flags]);

console.log(`Csound version: ${cs.name}`);

const startReturn = await cs.start();

console.log(startReturn);

// await cs.stop();

// cs.terminateInstance && (await cs.terminateInstance());

},

});

Then call csoundInitialize from a C# script 

void Start()

{

Debug.Log("Calling csoundInitialize");

csoundInitialize(0);

}

and bang we got it!!

TestBuild_0.framework.js:913 Calling csoundInitialize
TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:913 0

Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D 
I'll dig deeper inside this as soon as I have sound output.

So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
But of course the Start function is not waiting for it to complete before progressing.

All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence. 
Have a look at the initialization sequence of the CsoundUnityBridge
So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
Should the CsoundUnityWebGL be a completely different package?

The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).

So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound. 
I'm thinking of workarounds but have no ideas at the moment.

But for now let's focus on the fact that there was some progress! It's a win! :D

I'll keep you posted as I continue to experiment with this, 
thanks again for now!

Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:

Webassembly Csound already compiles to a single JS file that embeds
the WASM binary into it. I assume this would work with Unity but am
unsure. I think you would only need the dist/csound.js file. You could
also try using the one from 6.18.7:

https://unpkg.com/browse/@csound/browser@6.18.7/

It might be possible to rename it .jspre and create a test .jslib that
uses the Csound API.

On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
<giovanni.bedetti@gmail.com> wrote:
>
> Quick update on this: no luck yet!
>
> Sorry for the long email that follows. Any help would be much appreciated!
>
>
> Unity uses Emscripten for the WebGL build.
>
> There are 2 options using Emscripten:
>
> Compile Csound as a static library to be used as Unity plug-in
> First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
>
> Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
>
>
> I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> This should be the preferred way of creating the required files for Unity.
>
> The supported native plug-in file formats are
>
> LLVM Bitcode file format (.bc)
> GNU archive file format (.a)
>
> I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> I ran yarn build in the wasm folder.
> In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
>
> I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
>
> So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
>
> .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
>
> Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
>
> const hello = {
>     Hello: function() {
>         window.alert("Hello, world!");
>     }
> };
> mergeInto(LibraryManager.library, hello);
>
> to be called in C# like this:
>
> [DllImport("__Internal")]
>     private static extern void Hello();
>
> Bundle the C/C++ code in the Unity project
> This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> Out of curiosity, I tried to import some folders from the Csound repo:
> - Engine
> - H
> - include
> - InOut
> - interfaces
> - OOps
> - Opcodes
> - po
> - util
> - util1
> - util2
>
> When I build I get the following error, "Plugins colliding with each other.":
>
> Plugin 'mp3dec_internal.h' is used from several locations:
>  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
>  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> Plugin 'mpadec_internal.h' is used from several locations:
>  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
>  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> Plugin 'stdopcod.h' is used from several locations:
>  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>
> I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
>
> There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
>
> Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
>
> Hope someone has some tips here, I'm lost in the Web(GL)!
>
>
> Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
>>
>> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
>> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
>>
>> In the next days I will try something and report here my findings.
>>
>>
>> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
>>>
>>> Is there WebAssembly support for Csound in Unity yet?

I made some progress, we have sound!
Check it out in the repo.
At this link you can test the working build! (it's safe I swear)

Il giorno mar 18 giu 2024 alle ore 13:56 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:

I have no idea, this is what the docs say about this, no mention about future plans anywhere:

Because Unity uses FMODto manage audio for platforms, the Web platform supports limited audio functionality, which only includes the basic features. FMOD relies on threads, which the WebGL API doesn’t support. For this reason, Unity uses an implementation based on the internal Web Audio API, which enables the browser to handle audio playback and mixing.

Il Mar 18 Giu 2024, 13:41 Rory Walsh <rorywalsh@ear.ie> ha scritto:

Do they plan to add support for the OnAudioFilterRead, is it on any roadmaps? That's a fairly major part of things. 

On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <giovanni.bedetti@gmail.com> wrote:

Thanks for your replies!
Dr.B let's chat soon! I'll drop you an email ;)

I'm getting closer! I was able to load Csound! Woho!

I have created a repository with a test project and a build folder if anybody wants to take a look.

I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins. 
If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)

I tried using the other extension, .jslib, and I got this error instead:
Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',

So I renamed the file back to csound.jspre, and commented the export lines at the end.

const Csound = Csound$$module$src$index;
Csound.toString = () => 'async (options) => CsoundObj;';
//export { Csound }
//export default Csound
//# sourceMappingURL=csound.js.map

It builds and runs ok, of course no Csound yet as nothing is calling it.

So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)

mergeInto(LibraryManager.library, {

csoundInitialize: async function(flags) {

window.alert("csoundInitialize");

const csoundVariations = [

{ useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },

{ useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },

{ useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },

{ useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },

{ useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },

];

//Csound.csoundInitialize(flags); // this doesn't work

console.log("starting to await for Csound with flag: + flags")

const cs = await Csound(csoundVariations[flags]);

console.log(`Csound version: ${cs.name}`);

const startReturn = await cs.start();

console.log(startReturn);

// await cs.stop();

// cs.terminateInstance && (await cs.terminateInstance());

},

});

Then call csoundInitialize from a C# script 

void Start()

{

Debug.Log("Calling csoundInitialize");

csoundInitialize(0);

}

and bang we got it!!

TestBuild_0.framework.js:913 Calling csoundInitialize
TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:913 0

Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D 
I'll dig deeper inside this as soon as I have sound output.

So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
But of course the Start function is not waiting for it to complete before progressing.

All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence. 
Have a look at the initialization sequence of the CsoundUnityBridge
So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
Should the CsoundUnityWebGL be a completely different package?

The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).

So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound. 
I'm thinking of workarounds but have no ideas at the moment.

But for now let's focus on the fact that there was some progress! It's a win! :D

I'll keep you posted as I continue to experiment with this, 
thanks again for now!

Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:

Webassembly Csound already compiles to a single JS file that embeds
the WASM binary into it. I assume this would work with Unity but am
unsure. I think you would only need the dist/csound.js file. You could
also try using the one from 6.18.7:

https://unpkg.com/browse/@csound/browser@6.18.7/

It might be possible to rename it .jspre and create a test .jslib that
uses the Csound API.

On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
<giovanni.bedetti@gmail.com> wrote:
>
> Quick update on this: no luck yet!
>
> Sorry for the long email that follows. Any help would be much appreciated!
>
>
> Unity uses Emscripten for the WebGL build.
>
> There are 2 options using Emscripten:
>
> Compile Csound as a static library to be used as Unity plug-in
> First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
>
> Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
>
>
> I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> This should be the preferred way of creating the required files for Unity.
>
> The supported native plug-in file formats are
>
> LLVM Bitcode file format (.bc)
> GNU archive file format (.a)
>
> I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> I ran yarn build in the wasm folder.
> In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
>
> I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
>
> So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
>
> .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
>
> Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
>
> const hello = {
>     Hello: function() {
>         window.alert("Hello, world!");
>     }
> };
> mergeInto(LibraryManager.library, hello);
>
> to be called in C# like this:
>
> [DllImport("__Internal")]
>     private static extern void Hello();
>
> Bundle the C/C++ code in the Unity project
> This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> Out of curiosity, I tried to import some folders from the Csound repo:
> - Engine
> - H
> - include
> - InOut
> - interfaces
> - OOps
> - Opcodes
> - po
> - util
> - util1
> - util2
>
> When I build I get the following error, "Plugins colliding with each other.":
>
> Plugin 'mp3dec_internal.h' is used from several locations:
>  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
>  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> Plugin 'mpadec_internal.h' is used from several locations:
>  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
>  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> Plugin 'stdopcod.h' is used from several locations:
>  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>
> I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
>
> There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
>
> Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
>
> Hope someone has some tips here, I'm lost in the Web(GL)!
>
>
> Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
>>
>> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
>> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
>>
>> In the next days I will try something and report here my findings.
>>
>>
>> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
>>>
>>> Is there WebAssembly support for Csound in Unity yet?

I made some progress, we have sound!
Check it out in the repo.
At this link you can test the working build! (it's safe I swear)

Il giorno mar 18 giu 2024 alle ore 13:56 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:

I have no idea, this is what the docs say about this, no mention about future plans anywhere:

Because Unity uses FMODto manage audio for platforms, the Web platform supports limited audio functionality, which only includes the basic features. FMOD relies on threads, which the WebGL API doesn’t support. For this reason, Unity uses an implementation based on the internal Web Audio API, which enables the browser to handle audio playback and mixing.

Il Mar 18 Giu 2024, 13:41 Rory Walsh <rorywalsh@ear.ie> ha scritto:

Do they plan to add support for the OnAudioFilterRead, is it on any roadmaps? That's a fairly major part of things. 

On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <giovanni.bedetti@gmail.com> wrote:

Thanks for your replies!
Dr.B let's chat soon! I'll drop you an email ;)

I'm getting closer! I was able to load Csound! Woho!

I have created a repository with a test project and a build folder if anybody wants to take a look.

I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins. 
If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)

I tried using the other extension, .jslib, and I got this error instead:
Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',

So I renamed the file back to csound.jspre, and commented the export lines at the end.

const Csound = Csound$$module$src$index;
Csound.toString = () => 'async (options) => CsoundObj;';
//export { Csound }
//export default Csound
//# sourceMappingURL=csound.js.map

It builds and runs ok, of course no Csound yet as nothing is calling it.

So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)

mergeInto(LibraryManager.library, {

csoundInitialize: async function(flags) {

window.alert("csoundInitialize");

const csoundVariations = [

{ useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },

{ useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },

{ useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },

{ useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },

{ useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },

];

//Csound.csoundInitialize(flags); // this doesn't work

console.log("starting to await for Csound with flag: + flags")

const cs = await Csound(csoundVariations[flags]);

console.log(`Csound version: ${cs.name}`);

const startReturn = await cs.start();

console.log(startReturn);

// await cs.stop();

// cs.terminateInstance && (await cs.terminateInstance());

},

});

Then call csoundInitialize from a C# script 

void Start()

{

Debug.Log("Calling csoundInitialize");

csoundInitialize(0);

}

and bang we got it!!

TestBuild_0.framework.js:913 Calling csoundInitialize
TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:913 0

Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D 
I'll dig deeper inside this as soon as I have sound output.

So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
But of course the Start function is not waiting for it to complete before progressing.

All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence. 
Have a look at the initialization sequence of the CsoundUnityBridge
So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
Should the CsoundUnityWebGL be a completely different package?

The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).

So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound. 
I'm thinking of workarounds but have no ideas at the moment.

But for now let's focus on the fact that there was some progress! It's a win! :D

I'll keep you posted as I continue to experiment with this, 
thanks again for now!

Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:

Webassembly Csound already compiles to a single JS file that embeds
the WASM binary into it. I assume this would work with Unity but am
unsure. I think you would only need the dist/csound.js file. You could
also try using the one from 6.18.7:

https://unpkg.com/browse/@csound/browser@6.18.7/

It might be possible to rename it .jspre and create a test .jslib that
uses the Csound API.

On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
<giovanni.bedetti@gmail.com> wrote:
>
> Quick update on this: no luck yet!
>
> Sorry for the long email that follows. Any help would be much appreciated!
>
>
> Unity uses Emscripten for the WebGL build.
>
> There are 2 options using Emscripten:
>
> Compile Csound as a static library to be used as Unity plug-in
> First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
>
> Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
>
>
> I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> This should be the preferred way of creating the required files for Unity.
>
> The supported native plug-in file formats are
>
> LLVM Bitcode file format (.bc)
> GNU archive file format (.a)
>
> I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> I ran yarn build in the wasm folder.
> In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
>
> I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
>
> So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
>
> .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
>
> Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
>
> const hello = {
>     Hello: function() {
>         window.alert("Hello, world!");
>     }
> };
> mergeInto(LibraryManager.library, hello);
>
> to be called in C# like this:
>
> [DllImport("__Internal")]
>     private static extern void Hello();
>
> Bundle the C/C++ code in the Unity project
> This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> Out of curiosity, I tried to import some folders from the Csound repo:
> - Engine
> - H
> - include
> - InOut
> - interfaces
> - OOps
> - Opcodes
> - po
> - util
> - util1
> - util2
>
> When I build I get the following error, "Plugins colliding with each other.":
>
> Plugin 'mp3dec_internal.h' is used from several locations:
>  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
>  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> Plugin 'mpadec_internal.h' is used from several locations:
>  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
>  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> Plugin 'stdopcod.h' is used from several locations:
>  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>
> I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
>
> There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
>
> Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
>
> Hope someone has some tips here, I'm lost in the Web(GL)!
>
>
> Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
>>
>> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
>> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
>>
>> In the next days I will try something and report here my findings.
>>
>>
>> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
>>>
>>> Is there WebAssembly support for Csound in Unity yet?

wow - you are getting there.  this will be awesome too - an important next step.

- Dr.B

Dr. Richard Boulanger

Professor

Electronic Production and Design

Berklee College of Music

Professional Writing & Technology Division

On Tue, Jun 18, 2024 at 1:36 PM Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:

I made some progress, we have sound!
Check it out in the repo.
At this link you can test the working build! (it's safe I swear)

Il giorno mar 18 giu 2024 alle ore 13:56 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:

I have no idea, this is what the docs say about this, no mention about future plans anywhere:

Because Unity uses FMODto manage audio for platforms, the Web platform supports limited audio functionality, which only includes the basic features. FMOD relies on threads, which the WebGL API doesn’t support. For this reason, Unity uses an implementation based on the internal Web Audio API, which enables the browser to handle audio playback and mixing.

Il Mar 18 Giu 2024, 13:41 Rory Walsh <rorywalsh@ear.ie> ha scritto:

Do they plan to add support for the OnAudioFilterRead, is it on any roadmaps? That's a fairly major part of things. 

On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <giovanni.bedetti@gmail.com> wrote:

Thanks for your replies!
Dr.B let's chat soon! I'll drop you an email ;)

I'm getting closer! I was able to load Csound! Woho!

I have created a repository with a test project and a build folder if anybody wants to take a look.

I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins. 
If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)

I tried using the other extension, .jslib, and I got this error instead:
Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',

So I renamed the file back to csound.jspre, and commented the export lines at the end.

const Csound = Csound$$module$src$index;
Csound.toString = () => 'async (options) => CsoundObj;';
//export { Csound }
//export default Csound
//# sourceMappingURL=csound.js.map

It builds and runs ok, of course no Csound yet as nothing is calling it.

So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)

mergeInto(LibraryManager.library, {

csoundInitialize: async function(flags) {

window.alert("csoundInitialize");

const csoundVariations = [

{ useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },

{ useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },

{ useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },

{ useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },

{ useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },

];

//Csound.csoundInitialize(flags); // this doesn't work

console.log("starting to await for Csound with flag: + flags")

const cs = await Csound(csoundVariations[flags]);

console.log(`Csound version: ${cs.name}`);

const startReturn = await cs.start();

console.log(startReturn);

// await cs.stop();

// cs.terminateInstance && (await cs.terminateInstance());

},

});

Then call csoundInitialize from a C# script 

void Start()

{

Debug.Log("Calling csoundInitialize");

csoundInitialize(0);

}

and bang we got it!!

TestBuild_0.framework.js:913 Calling csoundInitialize
TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:913 0

Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D 
I'll dig deeper inside this as soon as I have sound output.

So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
But of course the Start function is not waiting for it to complete before progressing.

All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence. 
Have a look at the initialization sequence of the CsoundUnityBridge
So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
Should the CsoundUnityWebGL be a completely different package?

The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).

So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound. 
I'm thinking of workarounds but have no ideas at the moment.

But for now let's focus on the fact that there was some progress! It's a win! :D

I'll keep you posted as I continue to experiment with this, 
thanks again for now!

Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:

Webassembly Csound already compiles to a single JS file that embeds
the WASM binary into it. I assume this would work with Unity but am
unsure. I think you would only need the dist/csound.js file. You could
also try using the one from 6.18.7:

https://unpkg.com/browse/@csound/browser@6.18.7/

It might be possible to rename it .jspre and create a test .jslib that
uses the Csound API.

On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
<giovanni.bedetti@gmail.com> wrote:
>
> Quick update on this: no luck yet!
>
> Sorry for the long email that follows. Any help would be much appreciated!
>
>
> Unity uses Emscripten for the WebGL build.
>
> There are 2 options using Emscripten:
>
> Compile Csound as a static library to be used as Unity plug-in
> First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
>
> Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
>
>
> I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> This should be the preferred way of creating the required files for Unity.
>
> The supported native plug-in file formats are
>
> LLVM Bitcode file format (.bc)
> GNU archive file format (.a)
>
> I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> I ran yarn build in the wasm folder.
> In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
>
> I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
>
> So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
>
> .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
>
> Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
>
> const hello = {
>     Hello: function() {
>         window.alert("Hello, world!");
>     }
> };
> mergeInto(LibraryManager.library, hello);
>
> to be called in C# like this:
>
> [DllImport("__Internal")]
>     private static extern void Hello();
>
> Bundle the C/C++ code in the Unity project
> This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> Out of curiosity, I tried to import some folders from the Csound repo:
> - Engine
> - H
> - include
> - InOut
> - interfaces
> - OOps
> - Opcodes
> - po
> - util
> - util1
> - util2
>
> When I build I get the following error, "Plugins colliding with each other.":
>
> Plugin 'mp3dec_internal.h' is used from several locations:
>  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
>  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> Plugin 'mpadec_internal.h' is used from several locations:
>  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
>  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> Plugin 'stdopcod.h' is used from several locations:
>  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>
> I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
>
> There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
>
> Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
>
> Hope someone has some tips here, I'm lost in the Web(GL)!
>
>
> Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
>>
>> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
>> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
>>
>> In the next days I will try something and report here my findings.
>>
>>
>> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
>>>
>>> Is there WebAssembly support for Csound in Unity yet?

I too heard the sine tone in my browser - nice.

- now - a button and a fader in the browser will be something!

- Dr.B

Dr. Richard Boulanger

Professor

Electronic Production and Design

Berklee College of Music

Professional Writing & Technology Division

On Tue, Jun 18, 2024 at 9:12 PM Dr. Richard Boulanger <rboulanger@berklee.edu> wrote:

wow - you are getting there.  this will be awesome too - an important next step.

- Dr.B

Dr. Richard Boulanger

Professor

Electronic Production and Design

Berklee College of Music

Professional Writing & Technology Division

On Tue, Jun 18, 2024 at 1:36 PM Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:

I made some progress, we have sound!
Check it out in the repo.
At this link you can test the working build! (it's safe I swear)

Il giorno mar 18 giu 2024 alle ore 13:56 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:

I have no idea, this is what the docs say about this, no mention about future plans anywhere:

Because Unity uses FMODto manage audio for platforms, the Web platform supports limited audio functionality, which only includes the basic features. FMOD relies on threads, which the WebGL API doesn’t support. For this reason, Unity uses an implementation based on the internal Web Audio API, which enables the browser to handle audio playback and mixing.

Il Mar 18 Giu 2024, 13:41 Rory Walsh <rorywalsh@ear.ie> ha scritto:

Do they plan to add support for the OnAudioFilterRead, is it on any roadmaps? That's a fairly major part of things. 

On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <giovanni.bedetti@gmail.com> wrote:

Thanks for your replies!
Dr.B let's chat soon! I'll drop you an email ;)

I'm getting closer! I was able to load Csound! Woho!

I have created a repository with a test project and a build folder if anybody wants to take a look.

I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins. 
If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)

I tried using the other extension, .jslib, and I got this error instead:
Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',

So I renamed the file back to csound.jspre, and commented the export lines at the end.

const Csound = Csound$$module$src$index;
Csound.toString = () => 'async (options) => CsoundObj;';
//export { Csound }
//export default Csound
//# sourceMappingURL=csound.js.map

It builds and runs ok, of course no Csound yet as nothing is calling it.

So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)

mergeInto(LibraryManager.library, {

csoundInitialize: async function(flags) {

window.alert("csoundInitialize");

const csoundVariations = [

{ useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },

{ useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },

{ useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },

{ useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },

{ useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },

];

//Csound.csoundInitialize(flags); // this doesn't work

console.log("starting to await for Csound with flag: + flags")

const cs = await Csound(csoundVariations[flags]);

console.log(`Csound version: ${cs.name}`);

const startReturn = await cs.start();

console.log(startReturn);

// await cs.stop();

// cs.terminateInstance && (await cs.terminateInstance());

},

});

Then call csoundInitialize from a C# script 

void Start()

{

Debug.Log("Calling csoundInitialize");

csoundInitialize(0);

}

and bang we got it!!

TestBuild_0.framework.js:913 Calling csoundInitialize
TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:913 0

Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D 
I'll dig deeper inside this as soon as I have sound output.

So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
But of course the Start function is not waiting for it to complete before progressing.

All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence. 
Have a look at the initialization sequence of the CsoundUnityBridge
So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
Should the CsoundUnityWebGL be a completely different package?

The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).

So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound. 
I'm thinking of workarounds but have no ideas at the moment.

But for now let's focus on the fact that there was some progress! It's a win! :D

I'll keep you posted as I continue to experiment with this, 
thanks again for now!

Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:

Webassembly Csound already compiles to a single JS file that embeds
the WASM binary into it. I assume this would work with Unity but am
unsure. I think you would only need the dist/csound.js file. You could
also try using the one from 6.18.7:

https://unpkg.com/browse/@csound/browser@6.18.7/

It might be possible to rename it .jspre and create a test .jslib that
uses the Csound API.

On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
<giovanni.bedetti@gmail.com> wrote:
>
> Quick update on this: no luck yet!
>
> Sorry for the long email that follows. Any help would be much appreciated!
>
>
> Unity uses Emscripten for the WebGL build.
>
> There are 2 options using Emscripten:
>
> Compile Csound as a static library to be used as Unity plug-in
> First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
>
> Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
>
>
> I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> This should be the preferred way of creating the required files for Unity.
>
> The supported native plug-in file formats are
>
> LLVM Bitcode file format (.bc)
> GNU archive file format (.a)
>
> I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> I ran yarn build in the wasm folder.
> In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
>
> I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
>
> So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
>
> .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
>
> Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
>
> const hello = {
>     Hello: function() {
>         window.alert("Hello, world!");
>     }
> };
> mergeInto(LibraryManager.library, hello);
>
> to be called in C# like this:
>
> [DllImport("__Internal")]
>     private static extern void Hello();
>
> Bundle the C/C++ code in the Unity project
> This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> Out of curiosity, I tried to import some folders from the Csound repo:
> - Engine
> - H
> - include
> - InOut
> - interfaces
> - OOps
> - Opcodes
> - po
> - util
> - util1
> - util2
>
> When I build I get the following error, "Plugins colliding with each other.":
>
> Plugin 'mp3dec_internal.h' is used from several locations:
>  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
>  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> Plugin 'mpadec_internal.h' is used from several locations:
>  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
>  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> Plugin 'stdopcod.h' is used from several locations:
>  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
>
> I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
>
> There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
>
> Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
>
> Hope someone has some tips here, I'm lost in the Web(GL)!
>
>
> Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
>>
>> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
>> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
>>
>> In the next days I will try something and report here my findings.
>>
>>
>> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
>>>
>>> Is there WebAssembly support for Csound in Unity yet?

Tested it here on the Mac and the interaction is working well. On the mobile, I guess the touch screen interaction does not replace
the mouse. I get the csoundInitialize dialogue, but that’s about it.

On both I get this “HTTPS response Header” message, but that goes away after a while.

========================
Prof. Victor Lazzarini
Maynooth University
Ireland

> On 20 Jun 2024, at 11:45, Giovanni Bedetti <giovanni.bedetti@GMAIL.COM> wrote:
>
> *Warning*
> This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
> I have updated the repo and the build, same links as above.
> I have been able to create multiple instances of Csound from Unity, reference a specific instance and set a channel there, so I am making good progress!
> I had to rely on callbacks so yes the method signature (and workflow) will probably be very different from the original CsoundUnity implementation.
> At the moment there is no sound at all on the editor, so you'll have to build every time after a change.
>
>
>
>
> Il giorno mer 19 giu 2024 alle ore 03:13 Dr. Richard Boulanger <rboulanger@berklee.edu> ha scritto:
> I too heard the sine tone in my browser - nice.
> - now - a button and a fader in the browser will be something!
>
> - Dr.B
>
> Dr. Richard Boulanger
> Professor
> Electronic Production and Design
> Berklee College of Music
> Professional Writing & Technology Division
>
>
> On Tue, Jun 18, 2024 at 9:12 PM Dr. Richard Boulanger <rboulanger@berklee.edu> wrote:
> wow - you are getting there.  this will be awesome too - an important next step.
>
> - Dr.B
>
> Dr. Richard Boulanger
> Professor
> Electronic Production and Design
> Berklee College of Music
> Professional Writing & Technology Division
>
>
> On Tue, Jun 18, 2024 at 1:36 PM Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:
> I made some progress, we have sound!
> Check it out in the repo.
> At this link you can test the working build! (it's safe I swear)
>
>
>
> Il giorno mar 18 giu 2024 alle ore 13:56 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
> I have no idea, this is what the docs say about this, no mention about future plans anywhere:
> Because Unity uses FMODto manage audio for platforms, the Web platform supports limited audio functionality, which only includes the basic features. FMOD relies on threads, which the WebGL API doesn’t support. For this reason, Unity uses an implementation based on the internal Web Audio API, which enables the browser to handle audio playback and mixing.
>
> Il Mar 18 Giu 2024, 13:41 Rory Walsh <rorywalsh@ear.ie> ha scritto:
> Do they plan to add support for the OnAudioFilterRead, is it on any roadmaps? That's a fairly major part of things. 
>
> On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <giovanni.bedetti@gmail.com> wrote:
> Thanks for your replies!
> Dr.B let's chat soon! I'll drop you an email ;)
>
> I'm getting closer! I was able to load Csound! Woho!
>
> I have created a repository with a test project and a build folder if anybody wants to take a look.
>
> I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins.
> If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
> Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)
>
> I tried using the other extension, .jslib, and I got this error instead:
> Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',
>
> So I renamed the file back to csound.jspre, and commented the export lines at the end.
>
> const Csound = Csound$$module$src$index;
> Csound.toString = () => 'async (options) => CsoundObj;';
> //export { Csound }
> //export default Csound
> //# sourceMappingURL=csound.js.map
>
> It builds and runs ok, of course no Csound yet as nothing is calling it.
>
> So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)
>
>
> mergeInto(LibraryManager.library, {
>
> csoundInitialize: async function(flags) {
> window.alert("csoundInitialize");
>
> const csoundVariations = [
> { useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },
> { useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },
> { useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },
> { useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },
> { useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },
> ];
>
> //Csound.csoundInitialize(flags); // this doesn't work
> console.log("starting to await for Csound with flag: + flags")
> const cs = await Csound(csoundVariations[flags]);
> console.log(`Csound version: ${cs.name}`);
> const startReturn = await cs.start();
> console.log(startReturn);
> // await cs.stop(); // cs.terminateInstance && (await cs.terminateInstance()); },
> });
>
> Then call csoundInitialize from a C# script
> void Start()
> {
> Debug.Log("Calling csoundInitialize");
> csoundInitialize(0);
> }
>
> and bang we got it!!
>
> TestBuild_0.framework.js:913 Calling csoundInitialize
> TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
> TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
> TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
> TestBuild_0.framework.js:383 [commit: HEAD]
> TestBuild_0.framework.js:383 libsndfile-1.1.0
> TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
> TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
> TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
> TestBuild_0.framework.js:383 orch now loaded
> TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
> TestBuild_0.framework.js:383 SECTION 1:
> TestBuild_0.framework.js:913 0
>
> Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
> Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D
> I'll dig deeper inside this as soon as I have sound output.
>
> So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
> But of course the Start function is not waiting for it to complete before progressing.
>
> All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence.
> Have a look at the initialization sequence of the CsoundUnityBridge
> So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
> I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
> Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
> We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
> Should the CsoundUnityWebGL be a completely different package?
>
> The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).
> So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound.
> I'm thinking of workarounds but have no ideas at the moment.
>
> But for now let's focus on the fact that there was some progress! It's a win! :D
>
> I'll keep you posted as I continue to experiment with this,
> thanks again for now!
>
>
> Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:
> Webassembly Csound already compiles to a single JS file that embeds
> the WASM binary into it. I assume this would work with Unity but am
> unsure. I think you would only need the dist/csound.js file. You could
> also try using the one from 6.18.7:
>
> https://unpkg.com/browse/@csound/browser@6.18.7/
>
> It might be possible to rename it .jspre and create a test .jslib that
> uses the Csound API.
>
> On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
> <giovanni.bedetti@gmail.com> wrote:
> >
> > Quick update on this: no luck yet!
> >
> > Sorry for the long email that follows. Any help would be much appreciated!
> >
> >
> > Unity uses Emscripten for the WebGL build.
> >
> > There are 2 options using Emscripten:
> >
> > Compile Csound as a static library to be used as Unity plug-in
> > First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
> >
> > Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
> >
> >
> > I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> > This should be the preferred way of creating the required files for Unity.
> >
> > The supported native plug-in file formats are
> >
> > LLVM Bitcode file format (.bc)
> > GNU archive file format (.a)
> >
> > I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> > I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> > I ran yarn build in the wasm folder.
> > In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
> >
> > I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> > In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
> >
> > So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> > I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> > When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> > I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> > Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
> >
> > .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> > .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
> >
> > Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
> >
> > const hello = {
> >     Hello: function() {
> >         window.alert("Hello, world!");
> >     }
> > };
> > mergeInto(LibraryManager.library, hello);
> >
> > to be called in C# like this:
> >
> > [DllImport("__Internal")]
> >     private static extern void Hello();
> >
> > Bundle the C/C++ code in the Unity project
> > This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> > Out of curiosity, I tried to import some folders from the Csound repo:
> > - Engine
> > - H
> > - include
> > - InOut
> > - interfaces
> > - OOps
> > - Opcodes
> > - po
> > - util
> > - util1
> > - util2
> >
> > When I build I get the following error, "Plugins colliding with each other.":
> >
> > Plugin 'mp3dec_internal.h' is used from several locations:
> >  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> >  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> > Plugin 'mpadec_internal.h' is used from several locations:
> >  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> >  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> > Plugin 'stdopcod.h' is used from several locations:
> >  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
> >  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
> >
> > I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> > I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
> >
> > There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
> >
> > Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
> >
> > Hope someone has some tips here, I'm lost in the Web(GL)!
> >
> >
> > Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
> >>
> >> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
> >> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
> >>
> >> In the next days I will try something and report here my findings.
> >>
> >>
> >> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
> >>>
> >>> Is there WebAssembly support for Csound in Unity yet?

yes, I got these in the past but I can't remember what I did to fix it. I know mobile phones needed some sort of interaction to wake up Webaudio, but I think you have that with the screen clicking.

My simple tutorials (https://vlazzarini.github.io/vanilla/) make sound, so I am sure you can get your code to run as well.

Prof. Victor Lazzarini

Maynooth University

Ireland

On 20 Jun 2024, at 14:41, Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:



Yes unfortunately it doesn't work on mobile, the start promise is timed out.

This is the log:

TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 1.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:374 start promise timed out

This is the stack of the promise timeout error:

(anonymous) @ TestBuild_0.framework.js:374
setTimeout (async)
(anonymous) @ TestBuild_0.framework.js:373
createStartPromise @ TestBuild_0.framework.js:373
a @ TestBuild_0.framework.js:898
_csoundInitialize @ TestBuild_0.framework.js:913
await in _csoundInitialize (async)
$func33262 @ 0670e506:0x97e895
$func34985 @ 0670e506:0x9abd39
$func36433 @ 0670e506:0x9c759b
$dynCall_iiii @ 0670e506:0x170a42a
invoke_iiii @ TestBuild_0.framework.js:913
$func36417 @ 0670e506:0x9c6632
$func1111 @ 0670e506:0xa6bf1
$func60024 @ 0670e506:0x16c471c
$func57642 @ 0670e506:0x1674cc4
$func38105 @ 0670e506:0xaabc30
$func38106 @ 0670e506:0xaabc45
$func57359 @ 0670e506:0x1664efe
$func57067 @ 0670e506:0x163f9af
$func38930 @ 0670e506:0xaf67e8
$func38930 @ 0670e506:0xaf685c
$func60744 @ 0670e506:0x16f7e08
$dynCall_v @ 0670e506:0x170a466
(anonymous) @ TestBuild_0.framework.js:913
_JS_CallAsLongAsNoExceptionsSeen @ TestBuild_0.framework.js:913
$func60497 @ 0670e506:0x16e3bc4
$dynCall_iiii @ 0670e506:0x170a42a
(anonymous) @ TestBuild_0.framework.js:913
touchEventHandlerFunc @ TestBuild_0.framework.js:913
jsEventHandler @ TestBuild_0.framework.js:913

What could be the cause?

About “HTTPS response Header”, I will investigate the cause, it should have no bad effect though.

Il giorno gio 20 giu 2024 alle ore 14:36 Victor Lazzarini <000010b17ddd988e-dmarc-request@listserv.heanet.ie> ha scritto:

Tested it here on the Mac and the interaction is working well. On the mobile, I guess the touch screen interaction does not replace
the mouse. I get the csoundInitialize dialogue, but that’s about it.

On both I get this “HTTPS response Header” message, but that goes away after a while.

========================
Prof. Victor Lazzarini
Maynooth University
Ireland

> On 20 Jun 2024, at 11:45, Giovanni Bedetti <giovanni.bedetti@GMAIL.COM> wrote:
>
> *Warning*
> This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
> I have updated the repo and the build, same links as above.
> I have been able to create multiple instances of Csound from Unity, reference a specific instance and set a channel there, so I am making good progress!
> I had to rely on callbacks so yes the method signature (and workflow) will probably be very different from the original CsoundUnity implementation.
> At the moment there is no sound at all on the editor, so you'll have to build every time after a change.
>
>
>
>
> Il giorno mer 19 giu 2024 alle ore 03:13 Dr. Richard Boulanger <rboulanger@berklee.edu> ha scritto:
> I too heard the sine tone in my browser - nice.
> - now - a button and a fader in the browser will be something!
>
> - Dr.B
>
> Dr. Richard Boulanger
> Professor
> Electronic Production and Design
> Berklee College of Music
> Professional Writing & Technology Division
>
>
> On Tue, Jun 18, 2024 at 9:12 PM Dr. Richard Boulanger <rboulanger@berklee.edu> wrote:
> wow - you are getting there.  this will be awesome too - an important next step.
>
> - Dr.B
>
> Dr. Richard Boulanger
> Professor
> Electronic Production and Design
> Berklee College of Music
> Professional Writing & Technology Division
>
>
> On Tue, Jun 18, 2024 at 1:36 PM Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:
> I made some progress, we have sound!
> Check it out in the repo.
> At this link you can test the working build! (it's safe I swear)
>
>
>
> Il giorno mar 18 giu 2024 alle ore 13:56 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
> I have no idea, this is what the docs say about this, no mention about future plans anywhere:
> Because Unity uses FMODto manage audio for platforms, the Web platform supports limited audio functionality, which only includes the basic features. FMOD relies on threads, which the WebGL API doesn’t support. For this reason, Unity uses an implementation based on the internal Web Audio API, which enables the browser to handle audio playback and mixing.
>
> Il Mar 18 Giu 2024, 13:41 Rory Walsh <rorywalsh@ear.ie> ha scritto:
> Do they plan to add support for the OnAudioFilterRead, is it on any roadmaps? That's a fairly major part of things. 
>
> On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <giovanni.bedetti@gmail.com> wrote:
> Thanks for your replies!
> Dr.B let's chat soon! I'll drop you an email ;)
>
> I'm getting closer! I was able to load Csound! Woho!
>
> I have created a repository with a test project and a build folder if anybody wants to take a look.
>
> I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins.
> If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
> Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)
>
> I tried using the other extension, .jslib, and I got this error instead:
> Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',
>
> So I renamed the file back to csound.jspre, and commented the export lines at the end.
>
> const Csound = Csound$$module$src$index;
> Csound.toString = () => 'async (options) => CsoundObj;';
> //export { Csound }
> //export default Csound
> //# sourceMappingURL=csound.js.map
>
> It builds and runs ok, of course no Csound yet as nothing is calling it.
>
> So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)
>
>
> mergeInto(LibraryManager.library, {
>
> csoundInitialize: async function(flags) {
> window.alert("csoundInitialize");
>
> const csoundVariations = [
> { useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },
> { useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },
> { useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },
> { useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },
> { useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },
> ];
>
> //Csound.csoundInitialize(flags); // this doesn't work
> console.log("starting to await for Csound with flag: + flags")
> const cs = await Csound(csoundVariations[flags]);
> console.log(`Csound version: ${cs.name}`);
> const startReturn = await cs.start();
> console.log(startReturn);
> // await cs.stop(); // cs.terminateInstance && (await cs.terminateInstance()); },
> });
>
> Then call csoundInitialize from a C# script
> void Start()
> {
> Debug.Log("Calling csoundInitialize");
> csoundInitialize(0);
> }
>
> and bang we got it!!
>
> TestBuild_0.framework.js:913 Calling csoundInitialize
> TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
> TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
> TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
> TestBuild_0.framework.js:383 [commit: HEAD]
> TestBuild_0.framework.js:383 libsndfile-1.1.0
> TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
> TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
> TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
> TestBuild_0.framework.js:383 orch now loaded
> TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
> TestBuild_0.framework.js:383 SECTION 1:
> TestBuild_0.framework.js:913 0
>
> Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
> Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D
> I'll dig deeper inside this as soon as I have sound output.
>
> So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
> But of course the Start function is not waiting for it to complete before progressing.
>
> All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence.
> Have a look at the initialization sequence of the CsoundUnityBridge
> So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
> I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
> Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
> We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
> Should the CsoundUnityWebGL be a completely different package?
>
> The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).
> So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound.
> I'm thinking of workarounds but have no ideas at the moment.
>
> But for now let's focus on the fact that there was some progress! It's a win! :D
>
> I'll keep you posted as I continue to experiment with this,
> thanks again for now!
>
>
> Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:
> Webassembly Csound already compiles to a single JS file that embeds
> the WASM binary into it. I assume this would work with Unity but am
> unsure. I think you would only need the dist/csound.js file. You could
> also try using the one from 6.18.7:
>
> https://unpkg.com/browse/@csound/browser@6.18.7/
>
> It might be possible to rename it .jspre and create a test .jslib that
> uses the Csound API.
>
> On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
> <giovanni.bedetti@gmail.com> wrote:
> >
> > Quick update on this: no luck yet!
> >
> > Sorry for the long email that follows. Any help would be much appreciated!
> >
> >
> > Unity uses Emscripten for the WebGL build.
> >
> > There are 2 options using Emscripten:
> >
> > Compile Csound as a static library to be used as Unity plug-in
> > First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
> >
> > Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
> >
> >
> > I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> > This should be the preferred way of creating the required files for Unity.
> >
> > The supported native plug-in file formats are
> >
> > LLVM Bitcode file format (.bc)
> > GNU archive file format (.a)
> >
> > I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> > I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> > I ran yarn build in the wasm folder.
> > In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
> >
> > I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> > In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
> >
> > So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> > I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> > When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> > I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> > Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
> >
> > .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> > .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
> >
> > Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
> >
> > const hello = {
> >     Hello: function() {
> >         window.alert("Hello, world!");
> >     }
> > };
> > mergeInto(LibraryManager.library, hello);
> >
> > to be called in C# like this:
> >
> > [DllImport("__Internal")]
> >     private static extern void Hello();
> >
> > Bundle the C/C++ code in the Unity project
> > This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> > Out of curiosity, I tried to import some folders from the Csound repo:
> > - Engine
> > - H
> > - include
> > - InOut
> > - interfaces
> > - OOps
> > - Opcodes
> > - po
> > - util
> > - util1
> > - util2
> >
> > When I build I get the following error, "Plugins colliding with each other.":
> >
> > Plugin 'mp3dec_internal.h' is used from several locations:
> >  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> >  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> > Plugin 'mpadec_internal.h' is used from several locations:
> >  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> >  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> > Plugin 'stdopcod.h' is used from several locations:
> >  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
> >  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
> >
> > I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> > I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
> >
> > There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
> >
> > Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
> >
> > Hope someone has some tips here, I'm lost in the Web(GL)!
> >
> >
> > Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
> >>
> >> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
> >> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
> >>
> >> In the next days I will try something and report here my findings.
> >>
> >>
> >> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
> >>>
> >>> Is there WebAssembly support for Csound in Unity yet?

Works on iphone, I get sound and I can shift the pitch around. I think it doesn't like more than one Csound instance though, there's dropouts.

Prof. Victor Lazzarini

Maynooth University

Ireland

On 21 Jun 2024, at 11:50, Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:



Thanks for the tutorials, they are of great help!
The fix to make it run on mobile was very easy and it was exactly what you suggested. 
I needed one more click to wake up the audio context, apparently it takes some more time on Unity-mobile, so when Csound was being started the audio context was not unlocked yet.
The build should now work on all mobile devices. If someone can test it on an iPhone it would be great!
I also fixed the MIME warning on the server adding an .htaccess file with this content:


# This configuration file should be uploaded to the server as "<Application Folder>/Build/.htaccess" -> THIS IS WRONG, I HAD TO PUT THE FILE IN THE APPLICATION FOLDER NOT IN THE BUILD FOLDER
# NOTE: "mod_mime" Apache module must be enabled for this configuration to work.
<IfModule mod_mime.c>

# The following lines are required for builds without decompression fallback, compressed with gzip
RemoveType .gz
AddEncoding gzip .gz
AddType application/gzip .data.gz # The correct MIME type here would be application/octet-stream, but due to Safari bug https://bugs.webkit.org/show_bug.cgi?id=247421, it's preferable to use MIME Type application/gzip instead.
AddType application/wasm .wasm.gz
AddType application/javascript .js.gz
AddType application/octet-stream .symbols.json.gz

# The following lines are required for builds without decompression fallback, compressed with Brotli
RemoveType .br
RemoveLanguage .br
AddEncoding br .br
AddType application/octet-stream .data.br
AddType application/wasm .wasm.br
AddType application/javascript .js.br
AddType application/octet-stream .symbols.json.br

# The following line improves loading performance for uncompressed builds
AddType application/wasm .wasm

# Uncomment the following line to improve loading performance for gzip-compressed builds with decompression fallback
# AddEncoding gzip .unityweb

# Uncomment the following line to improve loading performance for brotli-compressed builds with decompression fallback
# AddEncoding br .unityweb

</IfModule>


More info on servers at this link: https://docs.unity3d.com/Manual/webgl-server-configuration-code-samples.html

Il giorno gio 20 giu 2024 alle ore 18:15 Victor Lazzarini <000010b17ddd988e-dmarc-request@listserv.heanet.ie> ha scritto:

yes, I got these in the past but I can't remember what I did to fix it. I know mobile phones needed some sort of interaction to wake up Webaudio, but I think you have that with the screen clicking.

My simple tutorials (https://vlazzarini.github.io/vanilla/) make sound, so I am sure you can get your code to run as well.

Prof. Victor Lazzarini

Maynooth University

Ireland

On 20 Jun 2024, at 14:41, Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:



Yes unfortunately it doesn't work on mobile, the start promise is timed out.

This is the log:

TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 1.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:374 start promise timed out

This is the stack of the promise timeout error:

(anonymous) @ TestBuild_0.framework.js:374
setTimeout (async)
(anonymous) @ TestBuild_0.framework.js:373
createStartPromise @ TestBuild_0.framework.js:373
a @ TestBuild_0.framework.js:898
_csoundInitialize @ TestBuild_0.framework.js:913
await in _csoundInitialize (async)
$func33262 @ 0670e506:0x97e895
$func34985 @ 0670e506:0x9abd39
$func36433 @ 0670e506:0x9c759b
$dynCall_iiii @ 0670e506:0x170a42a
invoke_iiii @ TestBuild_0.framework.js:913
$func36417 @ 0670e506:0x9c6632
$func1111 @ 0670e506:0xa6bf1
$func60024 @ 0670e506:0x16c471c
$func57642 @ 0670e506:0x1674cc4
$func38105 @ 0670e506:0xaabc30
$func38106 @ 0670e506:0xaabc45
$func57359 @ 0670e506:0x1664efe
$func57067 @ 0670e506:0x163f9af
$func38930 @ 0670e506:0xaf67e8
$func38930 @ 0670e506:0xaf685c
$func60744 @ 0670e506:0x16f7e08
$dynCall_v @ 0670e506:0x170a466
(anonymous) @ TestBuild_0.framework.js:913
_JS_CallAsLongAsNoExceptionsSeen @ TestBuild_0.framework.js:913
$func60497 @ 0670e506:0x16e3bc4
$dynCall_iiii @ 0670e506:0x170a42a
(anonymous) @ TestBuild_0.framework.js:913
touchEventHandlerFunc @ TestBuild_0.framework.js:913
jsEventHandler @ TestBuild_0.framework.js:913

What could be the cause?

About “HTTPS response Header”, I will investigate the cause, it should have no bad effect though.

Il giorno gio 20 giu 2024 alle ore 14:36 Victor Lazzarini <000010b17ddd988e-dmarc-request@listserv.heanet.ie> ha scritto:

Tested it here on the Mac and the interaction is working well. On the mobile, I guess the touch screen interaction does not replace
the mouse. I get the csoundInitialize dialogue, but that’s about it.

On both I get this “HTTPS response Header” message, but that goes away after a while.

========================
Prof. Victor Lazzarini
Maynooth University
Ireland

> On 20 Jun 2024, at 11:45, Giovanni Bedetti <giovanni.bedetti@GMAIL.COM> wrote:
>
> *Warning*
> This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
> I have updated the repo and the build, same links as above.
> I have been able to create multiple instances of Csound from Unity, reference a specific instance and set a channel there, so I am making good progress!
> I had to rely on callbacks so yes the method signature (and workflow) will probably be very different from the original CsoundUnity implementation.
> At the moment there is no sound at all on the editor, so you'll have to build every time after a change.
>
>
>
>
> Il giorno mer 19 giu 2024 alle ore 03:13 Dr. Richard Boulanger <rboulanger@berklee.edu> ha scritto:
> I too heard the sine tone in my browser - nice.
> - now - a button and a fader in the browser will be something!
>
> - Dr.B
>
> Dr. Richard Boulanger
> Professor
> Electronic Production and Design
> Berklee College of Music
> Professional Writing & Technology Division
>
>
> On Tue, Jun 18, 2024 at 9:12 PM Dr. Richard Boulanger <rboulanger@berklee.edu> wrote:
> wow - you are getting there.  this will be awesome too - an important next step.
>
> - Dr.B
>
> Dr. Richard Boulanger
> Professor
> Electronic Production and Design
> Berklee College of Music
> Professional Writing & Technology Division
>
>
> On Tue, Jun 18, 2024 at 1:36 PM Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:
> I made some progress, we have sound!
> Check it out in the repo.
> At this link you can test the working build! (it's safe I swear)
>
>
>
> Il giorno mar 18 giu 2024 alle ore 13:56 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
> I have no idea, this is what the docs say about this, no mention about future plans anywhere:
> Because Unity uses FMODto manage audio for platforms, the Web platform supports limited audio functionality, which only includes the basic features. FMOD relies on threads, which the WebGL API doesn’t support. For this reason, Unity uses an implementation based on the internal Web Audio API, which enables the browser to handle audio playback and mixing.
>
> Il Mar 18 Giu 2024, 13:41 Rory Walsh <rorywalsh@ear.ie> ha scritto:
> Do they plan to add support for the OnAudioFilterRead, is it on any roadmaps? That's a fairly major part of things. 
>
> On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <giovanni.bedetti@gmail.com> wrote:
> Thanks for your replies!
> Dr.B let's chat soon! I'll drop you an email ;)
>
> I'm getting closer! I was able to load Csound! Woho!
>
> I have created a repository with a test project and a build folder if anybody wants to take a look.
>
> I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins.
> If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
> Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)
>
> I tried using the other extension, .jslib, and I got this error instead:
> Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',
>
> So I renamed the file back to csound.jspre, and commented the export lines at the end.
>
> const Csound = Csound$$module$src$index;
> Csound.toString = () => 'async (options) => CsoundObj;';
> //export { Csound }
> //export default Csound
> //# sourceMappingURL=csound.js.map
>
> It builds and runs ok, of course no Csound yet as nothing is calling it.
>
> So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)
>
>
> mergeInto(LibraryManager.library, {
>
> csoundInitialize: async function(flags) {
> window.alert("csoundInitialize");
>
> const csoundVariations = [
> { useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },
> { useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },
> { useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },
> { useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },
> { useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },
> ];
>
> //Csound.csoundInitialize(flags); // this doesn't work
> console.log("starting to await for Csound with flag: + flags")
> const cs = await Csound(csoundVariations[flags]);
> console.log(`Csound version: ${cs.name}`);
> const startReturn = await cs.start();
> console.log(startReturn);
> // await cs.stop(); // cs.terminateInstance && (await cs.terminateInstance()); },
> });
>
> Then call csoundInitialize from a C# script
> void Start()
> {
> Debug.Log("Calling csoundInitialize");
> csoundInitialize(0);
> }
>
> and bang we got it!!
>
> TestBuild_0.framework.js:913 Calling csoundInitialize
> TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
> TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
> TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
> TestBuild_0.framework.js:383 [commit: HEAD]
> TestBuild_0.framework.js:383 libsndfile-1.1.0
> TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
> TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
> TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
> TestBuild_0.framework.js:383 orch now loaded
> TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
> TestBuild_0.framework.js:383 SECTION 1:
> TestBuild_0.framework.js:913 0
>
> Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
> Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D
> I'll dig deeper inside this as soon as I have sound output.
>
> So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
> But of course the Start function is not waiting for it to complete before progressing.
>
> All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence.
> Have a look at the initialization sequence of the CsoundUnityBridge
> So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
> I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
> Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
> We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
> Should the CsoundUnityWebGL be a completely different package?
>
> The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).
> So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound.
> I'm thinking of workarounds but have no ideas at the moment.
>
> But for now let's focus on the fact that there was some progress! It's a win! :D
>
> I'll keep you posted as I continue to experiment with this,
> thanks again for now!
>
>
> Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:
> Webassembly Csound already compiles to a single JS file that embeds
> the WASM binary into it. I assume this would work with Unity but am
> unsure. I think you would only need the dist/csound.js file. You could
> also try using the one from 6.18.7:
>
> https://unpkg.com/browse/@csound/browser@6.18.7/
>
> It might be possible to rename it .jspre and create a test .jslib that
> uses the Csound API.
>
> On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
> <giovanni.bedetti@gmail.com> wrote:
> >
> > Quick update on this: no luck yet!
> >
> > Sorry for the long email that follows. Any help would be much appreciated!
> >
> >
> > Unity uses Emscripten for the WebGL build.
> >
> > There are 2 options using Emscripten:
> >
> > Compile Csound as a static library to be used as Unity plug-in
> > First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
> >
> > Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
> >
> >
> > I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> > This should be the preferred way of creating the required files for Unity.
> >
> > The supported native plug-in file formats are
> >
> > LLVM Bitcode file format (.bc)
> > GNU archive file format (.a)
> >
> > I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> > I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> > I ran yarn build in the wasm folder.
> > In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
> >
> > I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> > In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
> >
> > So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> > I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> > When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> > I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> > Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
> >
> > .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> > .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
> >
> > Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
> >
> > const hello = {
> >     Hello: function() {
> >         window.alert("Hello, world!");
> >     }
> > };
> > mergeInto(LibraryManager.library, hello);
> >
> > to be called in C# like this:
> >
> > [DllImport("__Internal")]
> >     private static extern void Hello();
> >
> > Bundle the C/C++ code in the Unity project
> > This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> > Out of curiosity, I tried to import some folders from the Csound repo:
> > - Engine
> > - H
> > - include
> > - InOut
> > - interfaces
> > - OOps
> > - Opcodes
> > - po
> > - util
> > - util1
> > - util2
> >
> > When I build I get the following error, "Plugins colliding with each other.":
> >
> > Plugin 'mp3dec_internal.h' is used from several locations:
> >  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> >  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> > Plugin 'mpadec_internal.h' is used from several locations:
> >  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> >  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> > Plugin 'stdopcod.h' is used from several locations:
> >  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
> >  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
> >
> > I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> > I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
> >
> > There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
> >
> > Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
> >
> > Hope someone has some tips here, I'm lost in the Web(GL)!
> >
> >
> > Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
> >>
> >> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
> >> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
> >>
> >> In the next days I will try something and report here my findings.
> >>
> >>
> >> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
> >>>
> >>> Is there WebAssembly support for Csound in Unity yet?

That's great!

I tried on a low end Samsung it works, but I get this error on the second instantiation:

TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
011e293a:0x48091 Uncaught (in promise) RuntimeError: unreachable
    at __wasi_proc_exit (011e293a:0x48091)
    at _Exit (011e293a:0x480c2)
    at __wasilibc_populate_preopens (011e293a:0xb88c9)
    at __wasm_call_ctors (011e293a:0x7b11)
    at _start (011e293a:0x5cb4e)
    at WASI$$module$src$filesystem$wasi.start (__compiled.worklet.singlethread.worker.js:663:140)
    at $jscompDefaultExport$$module$src$module (__compiled.worklet.singlethread.worker.js:752:428)
$__wasi_proc_exit @ 011e293a:0x48091
$_Exit @ 011e293a:0x480c2
$__wasilibc_populate_preopens @ 011e293a:0xb88c9
$__wasm_call_ctors @ 011e293a:0x7b11
$_start @ 011e293a:0x5cb4e
WASI$$module$src$filesystem$wasi.start @ __compiled.worklet.singlethread.worker.js:663
$jscompDefaultExport$$module$src$module @ __compiled.worklet.singlethread.worker.js:752
Promise.then (async)
initialize @ __compiled.worklet.singlethread.worker.js:758
e @ __compiled.worklet.singlethread.worker.js:67

Instead with a Google Pixel 4a I can create 10 Csound instances before the start promise is timed out.

So I think we're good ;)

Il giorno ven 21 giu 2024 alle ore 14:37 Victor Lazzarini <000010b17ddd988e-dmarc-request@listserv.heanet.ie> ha scritto:

Works on iphone, I get sound and I can shift the pitch around. I think it doesn't like more than one Csound instance though, there's dropouts.

Prof. Victor Lazzarini

Maynooth University

Ireland

On 21 Jun 2024, at 11:50, Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:



Thanks for the tutorials, they are of great help!
The fix to make it run on mobile was very easy and it was exactly what you suggested. 
I needed one more click to wake up the audio context, apparently it takes some more time on Unity-mobile, so when Csound was being started the audio context was not unlocked yet.
The build should now work on all mobile devices. If someone can test it on an iPhone it would be great!
I also fixed the MIME warning on the server adding an .htaccess file with this content:


# This configuration file should be uploaded to the server as "<Application Folder>/Build/.htaccess" -> THIS IS WRONG, I HAD TO PUT THE FILE IN THE APPLICATION FOLDER NOT IN THE BUILD FOLDER
# NOTE: "mod_mime" Apache module must be enabled for this configuration to work.
<IfModule mod_mime.c>

# The following lines are required for builds without decompression fallback, compressed with gzip
RemoveType .gz
AddEncoding gzip .gz
AddType application/gzip .data.gz # The correct MIME type here would be application/octet-stream, but due to Safari bug https://bugs.webkit.org/show_bug.cgi?id=247421, it's preferable to use MIME Type application/gzip instead.
AddType application/wasm .wasm.gz
AddType application/javascript .js.gz
AddType application/octet-stream .symbols.json.gz

# The following lines are required for builds without decompression fallback, compressed with Brotli
RemoveType .br
RemoveLanguage .br
AddEncoding br .br
AddType application/octet-stream .data.br
AddType application/wasm .wasm.br
AddType application/javascript .js.br
AddType application/octet-stream .symbols.json.br

# The following line improves loading performance for uncompressed builds
AddType application/wasm .wasm

# Uncomment the following line to improve loading performance for gzip-compressed builds with decompression fallback
# AddEncoding gzip .unityweb

# Uncomment the following line to improve loading performance for brotli-compressed builds with decompression fallback
# AddEncoding br .unityweb

</IfModule>


More info on servers at this link: https://docs.unity3d.com/Manual/webgl-server-configuration-code-samples.html

Il giorno gio 20 giu 2024 alle ore 18:15 Victor Lazzarini <000010b17ddd988e-dmarc-request@listserv.heanet.ie> ha scritto:

yes, I got these in the past but I can't remember what I did to fix it. I know mobile phones needed some sort of interaction to wake up Webaudio, but I think you have that with the screen clicking.

My simple tutorials (https://vlazzarini.github.io/vanilla/) make sound, so I am sure you can get your code to run as well.

Prof. Victor Lazzarini

Maynooth University

Ireland

On 20 Jun 2024, at 14:41, Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:



Yes unfortunately it doesn't work on mobile, the start promise is timed out.

This is the log:

TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 1.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:374 start promise timed out

This is the stack of the promise timeout error:

(anonymous) @ TestBuild_0.framework.js:374
setTimeout (async)
(anonymous) @ TestBuild_0.framework.js:373
createStartPromise @ TestBuild_0.framework.js:373
a @ TestBuild_0.framework.js:898
_csoundInitialize @ TestBuild_0.framework.js:913
await in _csoundInitialize (async)
$func33262 @ 0670e506:0x97e895
$func34985 @ 0670e506:0x9abd39
$func36433 @ 0670e506:0x9c759b
$dynCall_iiii @ 0670e506:0x170a42a
invoke_iiii @ TestBuild_0.framework.js:913
$func36417 @ 0670e506:0x9c6632
$func1111 @ 0670e506:0xa6bf1
$func60024 @ 0670e506:0x16c471c
$func57642 @ 0670e506:0x1674cc4
$func38105 @ 0670e506:0xaabc30
$func38106 @ 0670e506:0xaabc45
$func57359 @ 0670e506:0x1664efe
$func57067 @ 0670e506:0x163f9af
$func38930 @ 0670e506:0xaf67e8
$func38930 @ 0670e506:0xaf685c
$func60744 @ 0670e506:0x16f7e08
$dynCall_v @ 0670e506:0x170a466
(anonymous) @ TestBuild_0.framework.js:913
_JS_CallAsLongAsNoExceptionsSeen @ TestBuild_0.framework.js:913
$func60497 @ 0670e506:0x16e3bc4
$dynCall_iiii @ 0670e506:0x170a42a
(anonymous) @ TestBuild_0.framework.js:913
touchEventHandlerFunc @ TestBuild_0.framework.js:913
jsEventHandler @ TestBuild_0.framework.js:913

What could be the cause?

About “HTTPS response Header”, I will investigate the cause, it should have no bad effect though.

Il giorno gio 20 giu 2024 alle ore 14:36 Victor Lazzarini <000010b17ddd988e-dmarc-request@listserv.heanet.ie> ha scritto:

Tested it here on the Mac and the interaction is working well. On the mobile, I guess the touch screen interaction does not replace
the mouse. I get the csoundInitialize dialogue, but that’s about it.

On both I get this “HTTPS response Header” message, but that goes away after a while.

========================
Prof. Victor Lazzarini
Maynooth University
Ireland

> On 20 Jun 2024, at 11:45, Giovanni Bedetti <giovanni.bedetti@GMAIL.COM> wrote:
>
> *Warning*
> This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
> I have updated the repo and the build, same links as above.
> I have been able to create multiple instances of Csound from Unity, reference a specific instance and set a channel there, so I am making good progress!
> I had to rely on callbacks so yes the method signature (and workflow) will probably be very different from the original CsoundUnity implementation.
> At the moment there is no sound at all on the editor, so you'll have to build every time after a change.
>
>
>
>
> Il giorno mer 19 giu 2024 alle ore 03:13 Dr. Richard Boulanger <rboulanger@berklee.edu> ha scritto:
> I too heard the sine tone in my browser - nice.
> - now - a button and a fader in the browser will be something!
>
> - Dr.B
>
> Dr. Richard Boulanger
> Professor
> Electronic Production and Design
> Berklee College of Music
> Professional Writing & Technology Division
>
>
> On Tue, Jun 18, 2024 at 9:12 PM Dr. Richard Boulanger <rboulanger@berklee.edu> wrote:
> wow - you are getting there.  this will be awesome too - an important next step.
>
> - Dr.B
>
> Dr. Richard Boulanger
> Professor
> Electronic Production and Design
> Berklee College of Music
> Professional Writing & Technology Division
>
>
> On Tue, Jun 18, 2024 at 1:36 PM Giovanni Bedetti <giovanni.bedetti@gmail.com> wrote:
> I made some progress, we have sound!
> Check it out in the repo.
> At this link you can test the working build! (it's safe I swear)
>
>
>
> Il giorno mar 18 giu 2024 alle ore 13:56 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
> I have no idea, this is what the docs say about this, no mention about future plans anywhere:
> Because Unity uses FMODto manage audio for platforms, the Web platform supports limited audio functionality, which only includes the basic features. FMOD relies on threads, which the WebGL API doesn’t support. For this reason, Unity uses an implementation based on the internal Web Audio API, which enables the browser to handle audio playback and mixing.
>
> Il Mar 18 Giu 2024, 13:41 Rory Walsh <rorywalsh@ear.ie> ha scritto:
> Do they plan to add support for the OnAudioFilterRead, is it on any roadmaps? That's a fairly major part of things. 
>
> On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <giovanni.bedetti@gmail.com> wrote:
> Thanks for your replies!
> Dr.B let's chat soon! I'll drop you an email ;)
>
> I'm getting closer! I was able to load Csound! Woho!
>
> I have created a repository with a test project and a build folder if anybody wants to take a look.
>
> I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it to .jspre and placed it inside Assets/Plugins.
> If I build without any changes it builds ok, but as soon as the game starts on the browser I get this message:
> Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or compression was misconfigured? (check Content-Encoding HTTP Response Header on web server)
>
> I tried using the other extension, .jslib, and I got this error instead:
> Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib": SyntaxError: Unexpected token 'export',
>
> So I renamed the file back to csound.jspre, and commented the export lines at the end.
>
> const Csound = Csound$$module$src$index;
> Csound.toString = () => 'async (options) => CsoundObj;';
> //export { Csound }
> //export default Csound
> //# sourceMappingURL=csound.js.map
>
> It builds and runs ok, of course no Csound yet as nothing is calling it.
>
> So I added a .jslib file and wrote an initialize method (grabbed from the test.js in the 6.18.7 dist above)
>
>
> mergeInto(LibraryManager.library, {
>
> csoundInitialize: async function(flags) {
> window.alert("csoundInitialize");
>
> const csoundVariations = [
> { useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },
> { useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },
> { useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },
> { useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },
> { useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN, MessagePort" },
> ];
>
> //Csound.csoundInitialize(flags); // this doesn't work
> console.log("starting to await for Csound with flag: + flags")
> const cs = await Csound(csoundVariations[flags]);
> console.log(`Csound version: ${cs.name}`);
> const startReturn = await cs.start();
> console.log(startReturn);
> // await cs.stop(); // cs.terminateInstance && (await cs.terminateInstance()); },
> });
>
> Then call csoundInitialize from a C# script
> void Start()
> {
> Debug.Log("Calling csoundInitialize");
> csoundInitialize(0);
> }
>
> and bang we got it!!
>
> TestBuild_0.framework.js:913 Calling csoundInitialize
> TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
> TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet, Single-threaded
> TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17 2023
> TestBuild_0.framework.js:383 [commit: HEAD]
> TestBuild_0.framework.js:383 libsndfile-1.1.0
> TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
> TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
> TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
> TestBuild_0.framework.js:383 orch now loaded
> TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
> TestBuild_0.framework.js:383 SECTION 1:
> TestBuild_0.framework.js:913 0
>
> Only the SINGLE_THREAD csoundVariations print the Csound version, for the other 3 it says undefined, but the rest of the logs are still printed (sr and kr could differ).
> Unity apparently doesn't support multi-threading on WebGL yet (this is an interesting post on the subject), so workers don't work? :D
> I'll dig deeper inside this as soon as I have sound output.
>
> So let's get a bit into the details. It was "nice" that I was able to call an async method (csoundInitialize) from a non-async context (the Start() function).
> But of course the Start function is not waiting for it to complete before progressing.
>
> All of C# methods in the CsoundUnity package are synchronous, and sometimes called in a sequence.
> Have a look at the initialization sequence of the CsoundUnityBridge
> So I guess that to integrate WebGL into the existing CsoundUnity package we will have to use a lot of directives to handle that and it could become hard to maintain.
> I mean, the way we use the API in C# is very different from what we should do with the CsoundObj API.
> Furthermore, promises don't exist in the .NET environment out of the box, so we should rely on an external package to have them in C#.
> We could add callbacks on completion for the methods that need to be awaited but it doesn't look very stylish.
> Should the CsoundUnityWebGL be a completely different package?
>
> The other very important thing to keep in mind is that the OnAudioFilterRead function (what we use to do business in the Audio Thread, grabbing audio data from Csound and writing it into the AudioSource buffer - and the opposite) is not triggered at all on WebGL, and it means that it is not triggered on the editor too (I was very surprised to discover this).
> So you won't have the normal CsoundUnity behaviour on the Editor if the selected platform is WebGL. This means that the normal CsoundUnity workflow on the Editor on WebGL is not possible. I mean, the inspector with the control channels grabbed from the csd will still work, but pressing play on the Editor won't produce any sound.
> I'm thinking of workarounds but have no ideas at the moment.
>
> But for now let's focus on the fact that there was some progress! It's a win! :D
>
> I'll keep you posted as I continue to experiment with this,
> thanks again for now!
>
>
> Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <stevenyi@gmail.com> ha scritto:
> Webassembly Csound already compiles to a single JS file that embeds
> the WASM binary into it. I assume this would work with Unity but am
> unsure. I think you would only need the dist/csound.js file. You could
> also try using the one from 6.18.7:
>
> https://unpkg.com/browse/@csound/browser@6.18.7/
>
> It might be possible to rename it .jspre and create a test .jslib that
> uses the Csound API.
>
> On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
> <giovanni.bedetti@gmail.com> wrote:
> >
> > Quick update on this: no luck yet!
> >
> > Sorry for the long email that follows. Any help would be much appreciated!
> >
> >
> > Unity uses Emscripten for the WebGL build.
> >
> > There are 2 options using Emscripten:
> >
> > Compile Csound as a static library to be used as Unity plug-in
> > First thing I tried the old emscripten build for Web Csound from the master branch (6.18.1), but I couldn't build it on macOs with this error message:
> >
> > Error: Could not resolve './libcsound' from src/CsoundScriptProcessorNode.js
> >
> >
> > I also tried older versions of emscripten, but I cannot install the ones older than version 2.0, so no luck there.
> > This should be the preferred way of creating the required files for Unity.
> >
> > The supported native plug-in file formats are
> >
> > LLVM Bitcode file format (.bc)
> > GNU archive file format (.a)
> >
> > I noticed that the Csound wasm build was exporting a file with that extension so I went this route.
> > I was able to build the fix-csound-wasm branch (off CS7) and I can confirm that it works on macOS (commit bac483d).
> > I ran yarn build in the wasm folder.
> > In the wasm/lib folder I could find a "libcsound-wasm.a" file, that will be accepted by Unity as a native WebGL plug-in.
> >
> > I was hoping that the .a file would be enough, but it appears that I cannot reach its methods from C#, I guess I need some js glue code with the explicit exports.
> > In fact, If I try to build the Unity project with only the above .a file it builds ok, but as soon as I import the CsoundUnity package (with a little change to try and load the library for WebGL) I get loads of build errors (one for each native method we have, like "error: undefined symbol: csoundAddSpinSample (referenced by top-level compiled C/C++ code").
> >
> > So I started to wonder if I needed the wasm/browser part to be able to use the above library.
> > I tried to build the browser package. First of all I ran yarn link "@csound/wasm-bin" to link it with the previously built lib, then yarn build.
> > When built, it creates a "dist" folder, with several .js and .map files with the name starting with "__compiled", like "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js", "__compiled.vanilla.worker.js.map", together with "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> > I'm not sure how to import those files in Unity, I have very little experience on the WebGL platform.
> > Unity supports two JavaScript plug-in file types that let you add JavaScript code to your Unity project
> >
> > .jslib A JavaScript library to use in addition to those in Emscripten’s core libraries: emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> > .jspre A a file whose contents are added before the emitted code: emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
> >
> > Furthermore, Unity expects this syntax to retrieve the native methods that can be used in C#:
> >
> > const hello = {
> >     Hello: function() {
> >         window.alert("Hello, world!");
> >     }
> > };
> > mergeInto(LibraryManager.library, hello);
> >
> > to be called in C# like this:
> >
> > [DllImport("__Internal")]
> >     private static extern void Hello();
> >
> > Bundle the C/C++ code in the Unity project
> > This option is very undocumented and there's no mention of any build script or makefile to edit, so at the moment I have no idea how Csound will be built by Unity.
> > Out of curiosity, I tried to import some folders from the Csound repo:
> > - Engine
> > - H
> > - include
> > - InOut
> > - interfaces
> > - OOps
> > - Opcodes
> > - po
> > - util
> > - util1
> > - util2
> >
> > When I build I get the following error, "Plugins colliding with each other.":
> >
> > Plugin 'mp3dec_internal.h' is used from several locations:
> >  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> >  Assets/Plugins/H/mp3dec_internal.h would be copied to <PluginPath>/mp3dec_internal.h
> > Plugin 'mpadec_internal.h' is used from several locations:
> >  Assets/Plugins/H/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> >  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to <PluginPath>/mpadec_internal.h
> > Plugin 'stdopcod.h' is used from several locations:
> >  Assets/Plugins/Opcodes/stdopcod.h would be copied to <PluginPath>/stdopcod.h
> >  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to <PluginPath>/stdopcod.h
> >
> > I had a look at the two mp3dec_internal.h and they are identical except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h has an additional #include "mpadec_config.h".
> > I'm not sure what to try next here, I guess removing one of the two folders is too simple to be an option :D but maybe I'll try it later!
> >
> > There is also the option to use a pure Javascript plugin, but how to create a javascript version of Csound?
> >
> > Last option, Unity native audio plugins, but they can't be used for WebGL, see this comment on the official Native plugins repo
> >
> > Hope someone has some tips here, I'm lost in the Web(GL)!
> >
> >
> > Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <giovanni.bedetti@gmail.com> ha scritto:
> >>
> >> Not yet, there are limitations in the Unity FMOD implementation for the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead callback, which is the function we use to fill the Unity AudioSources content with the Csound output.
> >> Technically though it shouldn't be hard to import a Csound library for the Web, and use the API. The difficulty is sending the Csound content to the AudioSources, and so being able to move those sources in the 3D space and having them spatialized (Unity spatializer is first order ambisonic).
> >>
> >> In the next days I will try something and report here my findings.
> >>
> >>
> >> Il Ven 31 Mag 2024, 09:29 Michael Gogins <michael.gogins@gmail.com> ha scritto:
> >>>
> >>> Is there WebAssembly support for Csound in Unity yet?