[Csnd] Csound apps for Android in Qt/C++
Date | 2014-03-22 21:36 |
From | Tarmo Johannes |
Subject | [Csnd] Csound apps for Android in Qt/C++ |
Hi!
Sharing experience: [+ long email warning!]
Every now and then I have needed to write an app for android. I have never learnt to like or properly understand java and every time I waste huge amount of time finding out why some simple thing does not work.
On the other hand I love Qt and programming in C++ (+ QtCreator, the main IDE for developing in Qt, is really great, I feel it much more comfortable than Eclipse). Now Qt 5.2 has solid support for Android (and behold! - there is also beta support for iOs!). I was wondering if I can bring Qt for android and Csound for android together. And I succeeded!
I find the vast benefits of this approach: 1) first of all - you can write basically one app that works with small modifications on all platforms where you can find Qt and Csound - you can use the same code for desktop (linux, win, osx), android, iOs, meego and I don't know what else 2) you can create relatively easily beautiful animated frontends in QML (Qt Meta Language), write efficient, fast code in C++ and bind it with csound through the C++ api, use all the goodies of Qt like convenient threading, signals and slots system etc etc
You can write your app also using traditional widget based techinique. It works but does not look so good on touch screens.
Downsides: 1) the apps will be naturally bigger than native android code and may be problematic for devices with little memory 2) there are definitely specific android things that do not work through qt yet
A small test app is up on: http://tarmo.uuu.ee/android/QtApp-debug.apk and the source code (best to explore with QtCreator): http://tarmo.uuu.ee/android/qt-quick-proov.zip
The app demonstrates how to write a simple front-end in QML, bind it with a class in c++ (here class CsEngine that runs csound in a different thread). When you press on the button, it moves to other position, sends an score event to csound object according to the duration of the animation. During the animation it also sends a changing value (from 0 to 1) to a csound channel, according the easing line of the animation and csound plays a sound while the button moves.
And all that in less than 200 lines of code all together!
If you want to try it yourself:
PREPARATION For developing in Qt for android you need: Install and set up Android SDK, NDK and Qt development kit for android Good tutorials: https://www.kdab.com/qt-on-android-episode-1/ https://www.kdab.com/qt-on-android-episode-2/ https://www.kdab.com/qt-android-episode-3/
CODING: Run QtCreator, create new project, most suitably a Qt Quick Application, but can be also QtWiddgets application, -design your front-end (the design mode is very good - you can quickly drag and drop your interface together if you don't need much tweaking!) - create a class that handles csound. The methods that can be reached from qml must be declared with keyword Q_INVOKABLE - forward an object of that class to qmlviewer like viewer.rootContext()->setContextProperty("csound", &cs); // here cs is an object of the class and "csound" the name used int qml code
LIBRARIES AND HEADERS If you want to write your app so that it runs both on desktop and android, you need to set some specification. It is best to do with some conditions in the project (.pro) file and #ifdef sections in the code. You need to use also different headers and libraries - libcsound64 (for double precision) for desktop and libcsoundandroid for android ( the libraries for android must be built for arm architecture). It is easiest to get the prebuilt libraries from the csound-android project, folder CsoundAndroid/libs/armeabi. The section in the project file could look like:
INCLUDEPATH += /home/tarmo/src/cs6/include/csound /home/tarmo/src/csound6-git/android/CsoundAndroid/jni/
android { LIBS += -L/home/tarmo/src/csound-android-6.02.0/CsoundAndroid/libs/armeabi/ -lcsoundandroid -lsndfile HEADERS += AndroidCsound.hpp # this is probably not necessary DEFINES += FOR_ADNROID
} else: win32|unix { LIBS += -L/home/tarmo/src/cs6/lib -lcsound64 -lsndfile DEFINES += FOR_DESKTOP }
Of course you need to change the folder names according to your system.
In the code your csound object should use Csound as its class (and header csound.hpp) for desktop and AndroidCsound (AndroidCsound.hpp) for android. Also note that in android, before the Compile (or CompileOrc) method you should run setOpenSlCallbacks() method to enable the audio.
// from csengine.h -------- #ifdef FOR_ADNROID AndroidCsound cs; #elif FOR_DESKTOP Csound cs; #endif
// from csengine.cpp ------------- #ifdef FOR_ADNROID cs.setOpenSlCallbacks(); // for android audio to work #endif
DEPLOYING TO ANDROID DEVICE If you get everything built successfully and want run the app in an android or virtual device, you need to take care that you let the system copy also the cound libraries libcsoundandroid and libsndfile (built for arm architecture [or whatever architecture your android device uses]). It is easy. In QtCreator go to Projects->Android Kit -> Run -> Deploy configuration -> Additional libraries -> add libcsoundandroid and libsndfile.so
Also make sure you add audio permissions android.permission.MODIFY_AUDIO_SETTING and android.permission.RECORD_AUDIO to your AndroidManifest.xml file. To do so, click to Projects->Android Kit -> Run -> Deploy configuration -> Create AndroidManifest.xml and set the permissions there (and any other settings that you need to tell android).
There are three possibilities to deploy the Qt libs: 1) use external tool Ministro that will be downloaded from Google play first time you run this app (if it not there already). Ministro is a great too for Qt support - it downloads and installs necessary Qt libraries. If you have many Qt based apps, it is best to use minsitro - your package stays relatively small. 2) include all necessary libraries in your package. The app will become big (around 20 MB or more), but everything is there and no more questions for the user. I used now this solution. 3) let the system copy libraries to temporary directory - good for developing, since you don't need to send all the libraries to the device every time you deploy but not suitable for distribution - the libraries will be missing from apk package
And then: press Run!
Thanks to Steven Yi for hints about the CoundAndroid class and help for getting audio work!
If it seem useful, I would be happy to turn this email to a wiki page. Where?
And, of course, I have done just small testing and don't know what obstacles may come up but I am really excited! At least for me it seems to be MUCH more efficient way to write for android.
best! tarmo
|
Date | 2014-03-22 22:00 |
From | Richard van Bemmelen |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
I also discovered QML recently in combination with PySide for Python GUI development. Amazing possibilities! Richard 2014-03-22 22:36 GMT+01:00 Tarmo Johannes <tarmo.johannes@otsakool.edu.ee>:
|
Date | 2014-03-23 02:40 |
From | Andres Cabrera |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
Hi Tarmo, This is great. Thanks for the detailed writeup. This seems to be one of the best paths to write an audio application in Android.In your experience how do you find resource usage? Andrés On Sat, Mar 22, 2014 at 2:36 PM, Tarmo Johannes <tarmo.johannes@otsakool.edu.ee> wrote:
|
Date | 2014-03-23 10:56 |
From | Victor Lazzarini |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
Tarmo, this is very nice, I wonder if we should add this to the wiki, or somewhere else online. Victor ======================== Dr Victor Lazzarini Senior Lecturer NUI Maynooth, Ireland victor dot lazzarini at nuim dot ie On 23 Mar 2014, at 02:40, Andres Cabrera |
Date | 2014-03-23 11:44 |
From | Tarmo Johannes |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
Thanks!
I just did some experiments, how it to use existing csound files:
You should put csound file to resource [Andrés of course knows it but for all readers some hints:] in Qt Creator, right click on the project in project tree -> add new file -> Qt / Qt Recourse file, give it a name, open it, add prefix ( '/') , Add files, like test.csd). You can reach them in form ":/test.csd"
Csound.Copile(char * cs name) cannot read recourse files, so the file should be copied to a temporary file (and probably in Android it is also necessary due to permissions). In Qt it is only some lines:
#include <QTemporaryFile>
// in main: cs.open(":/test.csd");
// in csengine.cpp int CsEngine::open(QString csd) { QTemporaryFile *tempFile = QTemporaryFile::createNativeFile(csd); //TODO: check if not 0 //NB! createNativeFile new in Qt 5.2! before: createLocalFile if (!cs.Compile( tempFile->fileName().toLocal8Bit().data()) ){ cs.Start(); cs.Perform(); cs.Stop(); return 0; } else { qDebug()<<"Could not open csound file: "<<csd; return -1; } } // the temporary file gets destroyed here, so no worries about memory
So it is basically one line
QTemporaryFile *tempFile = QTemporaryFile::createNativeFile(csd);
that does the work! Compare it to the whole bunch of code you need in java to do the same thing.
If there are any audio files that will be used, I guess, they must be copied to the temporary files the same way. But then it needs likely more care not destroy them before csound has stopped (QTempFile gets destroyed when it gets out of scope).
greetings, tarmo
On Saturday 22 March 2014 19:40:11 Andres Cabrera wrote: > Hi Tarmo, > > This is great. Thanks for the detailed writeup. This seems to be one of the > best paths to write an audio application in Android. > > In your experience how do you find resource usage? > > Cheers, > Andrés > > > On Sat, Mar 22, 2014 at 2:36 PM, Tarmo Johannes < > > tarmo.johannes@otsakool.edu.ee> wrote: > > Hi! > > > > Sharing experience: > > > > [+ long email warning!] > > > > > > > > Every now and then I have needed to write an app for android. I have never > > > > learnt to like or properly understand java and every time I waste huge > > amount > > > > of time finding out why some simple thing does not work. > > > > > > > > On the other hand I love Qt and programming in C++ (+ QtCreator, the main > > IDE > > > > for developing in Qt, is really great, I feel it much more comfortable > > than > > > > Eclipse). Now Qt 5.2 has solid support for Android (and behold! - > > > > there is also beta support for iOs!). I was wondering if I can bring Qt > > for > > > > android and Csound for android together. And I succeeded! > > > > > > > > I find the vast benefits of this approach: > > > > 1) first of all - you can write basically one app that works with small > > > > modifications on all platforms where you can find Qt and Csound - you can > > use > > > > the same code for desktop (linux, win, osx), android, iOs, meego and I > > don't know what else > > > > 2) you can create relatively easily beautiful animated frontends in QML > > (Qt Meta Language), write efficient, fast code in C++ and bind it with > > csound through the C++ api, use all the goodies of Qt like convenient > > threading, signals and slots system etc etc > > > > > > > > You can write your app also using traditional widget based techinique. It > > works but does not look so good on touch screens. > > > > > > > > Downsides: > > > > 1) the apps will be naturally bigger than native android code and may be > > problematic for devices with little memory > > > > 2) there are definitely specific android things that do not work through > > qt yet > > > > > > > > > > > > A small test app is up on: > > > > http://tarmo.uuu.ee/android/QtApp-debug.apk > > > > and the source code (best to explore with QtCreator): > > > > http://tarmo.uuu.ee/android/qt-quick-proov.zip > > > > > > > > The app demonstrates how to write a simple front-end in QML, bind it > > > > with a class in c++ (here class CsEngine that runs csound in a different > > > > thread). > > > > When you press on the button, it moves to other position, sends > > > > an score event to csound object according to the duration of the > > animation. > > > > During the animation it also sends a changing value (from 0 to 1) to a > > > > csound channel, according the easing line of the animation and csound > > plays a sound while the button moves. > > > > > > > > And all that in less than 200 lines of code all together! > > > > > > > > > > > > > > > > If you want to try it yourself: > > > > > > > > PREPARATION > > > > For developing in Qt for android you need: > > > > Install and set up Android SDK, NDK and Qt development kit for android > > > > Good tutorials: > > > > https://www.kdab.com/qt-on-android-episode-1/ > > > > https://www.kdab.com/qt-on-android-episode-2/ > > > > https://www.kdab.com/qt-android-episode-3/ > > > > > > > > CODING: > > > > Run QtCreator, create new project, most suitably a Qt Quick Application, > > but can be also QtWiddgets application, > > > > -design your front-end (the design mode is very good - you can quickly > > drag and drop your interface together if you don't need much tweaking!) > > > > - create a class that handles csound. The methods that can be reached from > > qml must be declared with keyword Q_INVOKABLE > > > > - forward an object of that class to qmlviewer like > > > > viewer.rootContext()->setContextProperty("csound", &cs); // here cs is an > > object of the class and "csound" the name used int qml code > > > > > > > > > > > > LIBRARIES AND HEADERS > > > > If you want to write your app so that it runs both on desktop and android, > > you need to set some specification. It is best to do with some conditions > > in the project (.pro) file and #ifdef sections in the code. You need to > > use > > also different headers and libraries - libcsound64 (for double precision) > > for desktop and libcsoundandroid for android ( the libraries for android > > must be built for arm architecture). It is easiest to get the prebuilt > > libraries from the csound-android project, folder > > CsoundAndroid/libs/armeabi. > > > > The section in the project file could look like: > > > > > > > > INCLUDEPATH += /home/tarmo/src/cs6/include/csound > > /home/tarmo/src/csound6-git/android/CsoundAndroid/jni/ > > > > > > > > android { > > > > LIBS += > > -L/home/tarmo/src/csound-android-6.02.0/CsoundAndroid/libs/armeabi/ > > -lcsoundandroid -lsndfile > > > > HEADERS += AndroidCsound.hpp # this is probably not necessary > > > > DEFINES += FOR_ADNROID > > > > > > > > } else: win32|unix { > > > > LIBS += -L/home/tarmo/src/cs6/lib -lcsound64 -lsndfile > > > > DEFINES += FOR_DESKTOP > > > > } > > > > > > > > Of course you need to change the folder names according to your system. > > > > > > > > In the code your csound object should use Csound as its class (and header > > csound.hpp) for desktop and AndroidCsound (AndroidCsound.hpp) for android. > > > > Also note that in android, before the Compile (or CompileOrc) method you > > should run setOpenSlCallbacks() method to enable the audio. > > > > > > > > // from csengine.h -------- > > > > #ifdef FOR_ADNROID > > > > AndroidCsound cs; > > > > #elif FOR_DESKTOP > > > > Csound cs; > > > > #endif > > > > > > > > // from csengine.cpp ------------- > > > > #ifdef FOR_ADNROID > > > > cs.setOpenSlCallbacks(); // for android audio to work > > > > #endif > > > > > > > > > > > > DEPLOYING TO ANDROID DEVICE > > > > If you get everything built successfully and want run the app in an > > android or virtual device, you need to take care that you let the system > > copy also the cound libraries libcsoundandroid and libsndfile (built for > > arm architecture [or whatever architecture your android device uses]). It > > is easy. > > > > In QtCreator go to Projects->Android Kit -> Run -> Deploy configuration > > -> Additional libraries -> add libcsoundandroid and libsndfile.so > > > > > > > > Also make sure you add audio permissions > > android.permission.MODIFY_AUDIO_SETTING and > > android.permission.RECORD_AUDIO > > > > to your AndroidManifest.xml file. > > > > To do so, click to Projects->Android Kit -> Run -> Deploy configuration > > -> Create AndroidManifest.xml and set the permissions there (and any other > > settings that you need to tell android). > > > > > > > > There are three possibilities to deploy the Qt libs: > > > > 1) use external tool Ministro that will be downloaded from Google play > > first time you run this app (if it not there already). Ministro is a great > > too for Qt support - it downloads and installs necessary Qt libraries. If > > you have many Qt based apps, it is best to use minsitro - your package > > stays relatively small. > > > > 2) include all necessary libraries in your package. The app will become > > big (around 20 MB or more), but everything is there and no more questions > > for the user. I used now this solution. > > > > 3) let the system copy libraries to temporary directory - good for > > developing, since you don't need to send all the libraries to the device > > every time you deploy but not suitable for distribution - the libraries > > will be missing from apk package > > > > > > > > > > > > And then: press Run! > > > > > > > > > > > > Thanks to Steven Yi for hints about the CoundAndroid class and help for > > getting audio work! > > > > > > > > If it seem useful, I would be happy to turn this email to a wiki page. > > Where? > > > > > > > > And, of course, I have done just small testing and don't know what > > obstacles may come up but I am really excited! At least for me it seems to > > be MUCH more efficient way to write for android. > > > > > > > > > > > > best! > > > > tarmo > > Send bugs reports to the Sourceforge bug trackers > csound6: > https://sourceforge.net/p/csound/tickets/ > csound5: > https://sourceforge.net/p/csound/bugs/ > Discussions of bugs and features can be posted here > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe > csound"
|
Date | 2014-03-23 20:39 |
From | Jacques Leplat |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
Really interesting Tarmo. your experiences with CSound and QT are fascinating. It looks like the holy grail I have been seeking: cross mobile platform development is possible using c++, csound and qt. found http://grimaldi.univ-tln.fr/Qt/C++-GUI-Programming-with-Qt-4-1st-ed.pdf to help get into qt. Jacques
|
Date | 2014-03-23 20:53 |
From | Andres Cabrera |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
You will want to find something about Qt5. Although the API hasn't changed much, as Tarmo mentioned, you will want to use the newer QML GUIs instead of the previous QtWidget method.Cheers, Andrés On Sun, Mar 23, 2014 at 1:39 PM, Jacques Leplat <jleplat@j3ltd.com> wrote:
|
Date | 2014-03-24 14:06 |
From | Steven Yi |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
Hi Tarmo, This is fantastic! I think this information and project would be really nice to have as part of the Csound for Android SDK. My thought is: 1. Within csound/android in the Git repo, we add your QT project. 2. In the project folder, we add a README that has the information from your email 3. We update the Csound Android manual to discuss QT, perhaps adding the same information (and in that case, perhaps we put simplified build instructions in the README, and point the user to the manual for full information) I think it would similarly be very nice to have an iOS project. I think too that it would be good to start assembling a Csound Developer's SDK that would have a QT template project, XCode project, etc. We could make the SDK work for both host and plugin development. Thanks! steven On Sat, Mar 22, 2014 at 5:36 PM, Tarmo Johannes |
Date | 2014-03-24 14:47 |
From | Tarmo Johannes |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
Thanks, I am glad if these experiments are useful! Please let me know if I can do anything for preparing the supporting information. And again - I have done only first experiments and by no means I am no expert of QML or Qt- Android programming, but the beginning is promising! I hope to do some more experiments this week and will report about the results. best! tarmo On Monday 24 March 2014 10:06:42 Steven Yi wrote: > Hi Tarmo, > > This is fantastic! I think this information and project would be > really nice to have as part of the Csound for Android SDK. My thought > is: > > 1. Within csound/android in the Git repo, we add your QT project. > 2. In the project folder, we add a README that has the information > from your email > 3. We update the Csound Android manual to discuss QT, perhaps adding > the same information (and in that case, perhaps we put simplified > build instructions in the README, and point the user to the manual for > full information) > > I think it would similarly be very nice to have an iOS project. I > think too that it would be good to start assembling a Csound > Developer's SDK that would have a QT template project, XCode project, > etc. We could make the SDK work for both host and plugin development. > > Thanks! > steven > > > On Sat, Mar 22, 2014 at 5:36 PM, Tarmo Johannes > > |
Date | 2014-03-24 15:35 |
From | Michael Gogins |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
I'm very interested in following this. I prefer C++ to Java, so if experiments are promising, I may switch the Csound 6 Android app to the Qt approach. It would be a great bonus if that simplified building Csound for iOS as well. Thanks, Mike ----------------------------------------------------- Michael GoginsIrreducible Productions http://michaelgogins.tumblr.com Michael dot Gogins at gmail dot com On Mon, Mar 24, 2014 at 10:47 AM, Tarmo Johannes <tarmo.johannes@otsakool.edu.ee> wrote: Thanks, |
Date | 2014-03-25 21:27 |
From | Steven Yi |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
Hi Tarmo, We can proceed in a few ways: 1. If you're comfortable with it, you can use Github to fork a copy of Csound, make changes (i.e. add your project as a folder to the csound/android folder, add a README.md there), then issue a pull-request back to the main csound repository. We'll then review the pull-request and merge it back into the main line. 2. If you're not comfortable with it, you can package your project with a README.md as a zip and perhaps open an issue on the Github tracker to have your project added to the android folder. One of us will then take a look and do the merge for you. 3. If you want, I can get you started by taking the project you sent a link to and the email you sent and just setup something. You could then probably check it out from git and offer advice on how you'd like it modified. Thanks! steven On Mon, Mar 24, 2014 at 10:47 AM, Tarmo Johannes |
Date | 2014-03-26 08:32 |
From | Tarmo Johannes |
Subject | Re: [Csnd] Csound apps for Android in Qt/C++ |
Hi Steven, I think it will be no problem for me to learn how Github works and I will send a pull-request. I am working now on another example, when it will be done, I will review the code of both projects and prepare for submitting. Thanks! tarmo On Tuesday 25 March 2014 17:27:56 Steven Yi wrote: > Hi Tarmo, > > We can proceed in a few ways: > > 1. If you're comfortable with it, you can use Github to fork a copy of > Csound, make changes (i.e. add your project as a folder to the > csound/android folder, add a README.md there), then issue a > pull-request back to the main csound repository. We'll then review > the pull-request and merge it back into the main line. > > 2. If you're not comfortable with it, you can package your project > with a README.md as a zip and perhaps open an issue on the Github > tracker to have your project added to the android folder. One of us > will then take a look and do the merge for you. > > 3. If you want, I can get you started by taking the project you sent a > link to and the email you sent and just setup something. You could > then probably check it out from git and offer advice on how you'd like > it modified. > > Thanks! > steven > > On Mon, Mar 24, 2014 at 10:47 AM, Tarmo Johannes > > |