Csound Csound-dev Csound-tekno Search About

[Csnd] WebAudio Csound on mobile devices

Date2019-09-11 15:01
FromTarmo Johannes
Subject[Csnd] WebAudio Csound on mobile devices
Hi,

I tried out a web application with WebAudio Csound:

On desktop it works fine but my experience with handheld devices is:

on Android/Chrome there is huge latency (but this might be something to do with the KineticJS used for canvas)
on Android/Samsung internet browser and iOs/Safari the audio is crackling (I think they both are based on Webkit?). Also
crackle with these browsers.

What is your experience? Can someone comment?

Thanks!
tarmo


Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Date2019-09-11 15:58
FromSteven Yi
SubjectRe: [Csnd] WebAudio Csound on mobile devices
I tried and also got crackles on the flute page. Sliders worked
alright here, canvas did not make sound. This is with Android Chrome
on Pixel 3.

Running https://instruments.kunstmusik.com/ (these were made a couple
of years ago, code at https://github.com/kunstmusik/instruments) on my
browser was very smooth with little latency.

I have not tested on iOS in a while, but last I checked Safari had a
not very good WebAudio experience. I think it does not support
AudioWorklets and shares issues that happen with ScriptProcessorNode
running on the main thread.  On desktop and mobile though, I've found
Firefox's SPN implementation to be quite good compared to Chrome-based
browsers. The instruments site ran here on Android and did not hiccup
even when using multitouch input.

I checked and the instruments site is using Csound 6.11. I don't know
if that is a factor, or if Victor's csound.js wrapper is a factor, or
canvas, or what. I suppose it might be good to check the instruments
site on your devices to see how that responds and maybe we can figure
out more information from there.

On Wed, Sep 11, 2019 at 10:01 AM Tarmo Johannes  wrote:
>
> Hi,
>
> I tried out a web application with WebAudio Csound:
> http://tarmo.uuu.ee/wasm/flute/flute.html
>
> On desktop it works fine but my experience with handheld devices is:
>
> on Android/Chrome there is huge latency (but this might be something to do with the KineticJS used for canvas)
> on Android/Samsung internet browser and iOs/Safari the audio is crackling (I think they both are based on Webkit?). Also
> https://vlazzarini.github.io/paw/examples/sliders.html
> https://vlazzarini.github.io/paw/examples/canvas.html
> crackle with these browsers.
>
> What is your experience? Can someone comment?
>
> Thanks!
> tarmo
>
>
> Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2019-09-13 22:11
FromTarmo Johannes
SubjectRe: [Csnd] WebAudio Csound on mobile devices
AttachmentsNone  None  

Date2019-09-14 01:26
FromSteven Yi
SubjectRe: [Csnd] WebAudio Csound on mobile devices
Hi Tarmo,

HTTPS is definitely a requirement to use Audioworklets:

"Creating an AudioWorkletNode requires at least two things: an
AudioContext object and the processor name as a string. A processor
definition can be loaded and registered by the new Audio Worklet
object's addModule() call. Worklet APIs including Audio Worklet are
only available in a secure context, thus a page using them must be
served over HTTPS, although http://localhost is considered a secure
for local testing."

(from https://developers.google.com/web/updates/2017/12/audio-worklet)

That may be the issue for your. I saw that
http://tarmo.uuu.ee/wasm/flute/flute.html loaded SPN but
https://tarmo.uuu.ee/wasm/flute/flute.html loaded AudioWorklet on
Chrome Desktop here.

Perhaps that's the issue?

Steven

On Fri, Sep 13, 2019 at 5:11 PM Tarmo Johannes  wrote:
>
> Thanks, Steven!
>
> I did some testing and remote debugging from Google Chrome on my android to see the javascript console output and as you suggested, the quality depends wheter AudioWorklet or   ScriptProcessorNode is used.
>
> The results are somewhat strange:
>
> When I load
> https://instruments.kunstmusik.com/vt/index.html
> or
> https://vlazzarini.github.io/paw/examples/canvas.html
> either on desktop  or Android, the AudioWorklet is found and used:
> CsoundObj.js:49 Using WASM + AudioWorklet Csound implementation
>
> If I load the same (or probably the same) canvas.html that I had downloaded from Csound webpage (csound-web-6.12.0) on local server then
> - on Chrome desktop AudioWorklet is used
> - on Chorme/Android ScriptProcessor:
> js/CsoundObj.js:53 Using WASM + ScriptProcessorNode Csound implementation
>
> May this have something to do with the actual Csound-web version? The same with Csound-web-6.13.0 -  loading canvas.html to Android/Chrome uses ScriptProcessorNode.
> Can you guess what is different on kunstmusik.com or vlazzarini.github.io  ? Earlier Csound version? https?
>
> greetings,
> tarmo
>
>
>
> On Sep 11 2019, at 5:58 pm, Steven Yi  wrote:
>
> I tried and also got crackles on the flute page. Sliders worked
> alright here, canvas did not make sound. This is with Android Chrome
> on Pixel 3.
>
> Running https://instruments.kunstmusik.com/ (these were made a couple
> of years ago, code at https://github.com/kunstmusik/instruments) on my
> browser was very smooth with little latency.
>
> I have not tested on iOS in a while, but last I checked Safari had a
> not very good WebAudio experience. I think it does not support
> AudioWorklets and shares issues that happen with ScriptProcessorNode
> running on the main thread. On desktop and mobile though, I've found
> Firefox's SPN implementation to be quite good compared to Chrome-based
> browsers. The instruments site ran here on Android and did not hiccup
> even when using multitouch input.
>
> I checked and the instruments site is using Csound 6.11. I don't know
> if that is a factor, or if Victor's csound.js wrapper is a factor, or
> canvas, or what. I suppose it might be good to check the instruments
> site on your devices to see how that responds and maybe we can figure
> out more information from there.
>
> On Wed, Sep 11, 2019 at 10:01 AM Tarmo Johannes  wrote:
>
>
> Hi,
>
> I tried out a web application with WebAudio Csound:
> http://tarmo.uuu.ee/wasm/flute/flute.html
>
> On desktop it works fine but my experience with handheld devices is:
>
> on Android/Chrome there is huge latency (but this might be something to do with the KineticJS used for canvas)
> on Android/Samsung internet browser and iOs/Safari the audio is crackling (I think they both are based on Webkit?). Also
> https://vlazzarini.github.io/paw/examples/sliders.html
> https://vlazzarini.github.io/paw/examples/canvas.html
> crackle with these browsers.
>
> What is your experience? Can someone comment?
>
> Thanks!
> tarmo
>
>
> Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here
>
>
> Csound mailing list
> Csound@listserv.heanet.ie
> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
> Send bugs reports to
> https://github.com/csound/csound/issues
> Discussions of bugs and features can be posted here
>
> Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here