Hi All,

I've been diagnosing various crashes in Blue reported to me and that
I've found in running Blue.  The one due to the ->self has greatly
improved things, but there are still other crashes.

I have found one today due to the signal flow graph opcodes:


Thread 10 Crashed:: Java: Finalizer
0   libsystem_kernel.dylib         0x00007fff90568866 __pthread_kill + 10
1   libsystem_pthread.dylib       0x00007fff8f7bc35c pthread_kill + 92
2   libsystem_c.dylib             0x00007fff96441bba abort + 125
3   libsystem_malloc.dylib         0x00007fff8ea80093 free + 411
4                                 0x0000000114f425d3
csoundDestroyMutex + 51 (threads.c:533)
5   libsignalflowgraph.dylib       0x000000011c038a86
csoundModuleDestroy + 1350 (signalflowgraph.cpp:1710)
6                                 0x0000000114f30fce
csoundDestroyModules + 110 (csmodule.c:646)
7                                 0x0000000114f32fdd reset + 189 (csound.c:2650)
8                                 0x0000000114f32e85 csoundDestroy +
229 (csound.c:1206)
9   lib_jcsound6.jnilib           0x0000000114ce44b2 Csound::~Csound()
+ 34 (csound.hpp:856)
10  lib_jcsound6.jnilib           0x0000000114ce693e Csound::~Csound()
+ 14 (csound.hpp:856)
11  lib_jcsound6.jnilib           0x0000000114ce6956 Csound::~Csound()
+ 22 (csound.hpp:849)
12  lib_jcsound6.jnilib           0x0000000114ccfd5c
Java_csnd6_csnd6JNI_delete_1Csound + 76
(java_interfaceJAVA_wrap.cxx:12858)
13  ???                           0x0000000105b79698 0 + 4390885016
14  ???                           0x0000000105b6d058 0 + 4390834264
15  ???                           0x0000000105b6d058 0 + 4390834264
16  ???                           0x0000000105b674e7 0 + 4390810855
17  libjvm.dylib                   0x00000001052d6d90
JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*,
Thread*) + 554
18  libjvm.dylib                   0x00000001052d6b60
JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*)
+ 40
19  libjvm.dylib                   0x000000010530a605
jni_invoke_nonstatic(JNIEnv_*, JavaValue*, _jobject*, JNICallType,
_jmethodID*, JNI_ArgumentPusher*, Thread*) + 698
20  libjvm.dylib                   0x00000001052fe176 jni_CallVoidMethod + 278
21  libjava.dylib                 0x00000001047cb99c
Java_java_lang_ref_Finalizer_invokeFinalizeMethod + 88
22  ???                           0x0000000105b79698 0 + 4390885016
23  ???                           0x0000000105b6d058 0 + 4390834264
24  ???                           0x0000000105b6d058 0 + 4390834264
25  ???                           0x0000000105b6d058 0 + 4390834264
26  ???                           0x0000000105b674e7 0 + 4390810855
27  libjvm.dylib                   0x00000001052d6d90
JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*,
Thread*) + 554
28  libjvm.dylib                   0x00000001052d72a7
JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*,
JavaCallArguments*, Thread*) + 283
29  libjvm.dylib                   0x00000001052d73e4
JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*,
Symbol*, Thread*) + 74
30  libjvm.dylib                   0x00000001053263ca
thread_entry(JavaThread*, Thread*) + 173
31  libjvm.dylib                   0x00000001054efb47
JavaThread::thread_main_inner() + 155
32  libjvm.dylib                   0x00000001054f124f JavaThread::run() + 419
33  libjvm.dylib                   0x000000010541b1c6 java_start(Thread*) + 294
34  libsystem_pthread.dylib       0x00007fff8f7bb899 _pthread_body + 138
35  libsystem_pthread.dylib       0x00007fff8f7bb72a _pthread_start + 137


At the time, I had concurrent Csound instances running (one rendering
to disk, repeated start/stopping of Csound in realtime).  The code in
signalflowgraph.cpp contains:

static void* cs_sfg_ftables;
static void* cs_sfg_ports;

which is apparently not thread-safe.

Michael: Could you take a look at this?  My assumption for a fix is
that those mutexes should be stored in/retrieved from Csound using
csoundCreateGlobalVariable/csoundQueryGlobalVariable.

Thanks,
steven

------------------------------------------------------------------------------
Sponsored by Intel(R) XDK 
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

These mutexes need to have the same scope exactly as the plugin opcode module itself. I have been assuming that that is per process, ie one process may have more than one instance of Csound, but all of the instances in one process would share the module and the static variables. Is this the case?

Best,
Mike

If I understand your email correctly, yes.  The problem is that the
create and destroy module callbacks are run per instance. So let's say
you have 2 csound instances, you can start both and they will have
both called csoundModuleCreate.  Then you stop and destroy both and
they will have both called csoundModuleDestroy.  Since the the lib
uses static vars, it's possible to get two csoundModuleDestroy calls
in a row, thus the mutexes on the second time would try to free a null
pointer. That's why I was thinking they should store the mutex with
the instance of Csound rather than be static.

On Wed, Dec 4, 2013 at 5:31 PM, Michael Gogins  wrote:
> These mutexes need to have the same scope exactly as the plugin opcode
> module itself. I have been assuming that that is per process, ie one process
> may have more than one instance of Csound, but all of the instances in one
> process would share the module and the static variables. Is this the case?
>
> Best,
> Mike
>
> On Dec 4, 2013 10:53 PM, "Steven Yi"  wrote:
>>
>> Hi All,
>>
>> I've been diagnosing various crashes in Blue reported to me and that
>> I've found in running Blue.  The one due to the ->self has greatly
>> improved things, but there are still other crashes.
>>
>> I have found one today due to the signal flow graph opcodes:
>>
>>
>> Thread 10 Crashed:: Java: Finalizer
>> 0   libsystem_kernel.dylib         0x00007fff90568866 __pthread_kill + 10
>> 1   libsystem_pthread.dylib       0x00007fff8f7bc35c pthread_kill + 92
>> 2   libsystem_c.dylib             0x00007fff96441bba abort + 125
>> 3   libsystem_malloc.dylib         0x00007fff8ea80093 free + 411
>> 4                                 0x0000000114f425d3
>> csoundDestroyMutex + 51 (threads.c:533)
>> 5   libsignalflowgraph.dylib       0x000000011c038a86
>> csoundModuleDestroy + 1350 (signalflowgraph.cpp:1710)
>> 6                                 0x0000000114f30fce
>> csoundDestroyModules + 110 (csmodule.c:646)
>> 7                                 0x0000000114f32fdd reset + 189
>> (csound.c:2650)
>> 8                                 0x0000000114f32e85 csoundDestroy +
>> 229 (csound.c:1206)
>> 9   lib_jcsound6.jnilib           0x0000000114ce44b2 Csound::~Csound()
>> + 34 (csound.hpp:856)
>> 10  lib_jcsound6.jnilib           0x0000000114ce693e Csound::~Csound()
>> + 14 (csound.hpp:856)
>> 11  lib_jcsound6.jnilib           0x0000000114ce6956 Csound::~Csound()
>> + 22 (csound.hpp:849)
>> 12  lib_jcsound6.jnilib           0x0000000114ccfd5c
>> Java_csnd6_csnd6JNI_delete_1Csound + 76
>> (java_interfaceJAVA_wrap.cxx:12858)
>> 13  ???                           0x0000000105b79698 0 + 4390885016
>> 14  ???                           0x0000000105b6d058 0 + 4390834264
>> 15  ???                           0x0000000105b6d058 0 + 4390834264
>> 16  ???                           0x0000000105b674e7 0 + 4390810855
>> 17  libjvm.dylib                   0x00000001052d6d90
>> JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*,
>> Thread*) + 554
>> 18  libjvm.dylib                   0x00000001052d6b60
>> JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*)
>> + 40
>> 19  libjvm.dylib                   0x000000010530a605
>> jni_invoke_nonstatic(JNIEnv_*, JavaValue*, _jobject*, JNICallType,
>> _jmethodID*, JNI_ArgumentPusher*, Thread*) + 698
>> 20  libjvm.dylib                   0x00000001052fe176 jni_CallVoidMethod +
>> 278
>> 21  libjava.dylib                 0x00000001047cb99c
>> Java_java_lang_ref_Finalizer_invokeFinalizeMethod + 88
>> 22  ???                           0x0000000105b79698 0 + 4390885016
>> 23  ???                           0x0000000105b6d058 0 + 4390834264
>> 24  ???                           0x0000000105b6d058 0 + 4390834264
>> 25  ???                           0x0000000105b6d058 0 + 4390834264
>> 26  ???                           0x0000000105b674e7 0 + 4390810855
>> 27  libjvm.dylib                   0x00000001052d6d90
>> JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*,
>> Thread*) + 554
>> 28  libjvm.dylib                   0x00000001052d72a7
>> JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*,
>> JavaCallArguments*, Thread*) + 283
>> 29  libjvm.dylib                   0x00000001052d73e4
>> JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*,
>> Symbol*, Thread*) + 74
>> 30  libjvm.dylib                   0x00000001053263ca
>> thread_entry(JavaThread*, Thread*) + 173
>> 31  libjvm.dylib                   0x00000001054efb47
>> JavaThread::thread_main_inner() + 155
>> 32  libjvm.dylib                   0x00000001054f124f JavaThread::run() +
>> 419
>> 33  libjvm.dylib                   0x000000010541b1c6 java_start(Thread*)
>> + 294
>> 34  libsystem_pthread.dylib       0x00007fff8f7bb899 _pthread_body + 138
>> 35  libsystem_pthread.dylib       0x00007fff8f7bb72a _pthread_start + 137
>>
>>
>> At the time, I had concurrent Csound instances running (one rendering
>> to disk, repeated start/stopping of Csound in realtime).  The code in
>> signalflowgraph.cpp contains:
>>
>> static void* cs_sfg_ftables;
>> static void* cs_sfg_ports;
>>
>> which is apparently not thread-safe.
>>
>> Michael: Could you take a look at this?  My assumption for a fix is
>> that those mutexes should be stored in/retrieved from Csound using
>> csoundCreateGlobalVariable/csoundQueryGlobalVariable.
>>
>> Thanks,
>> steven
>>
>>
>> ------------------------------------------------------------------------------
>> Sponsored by Intel(R) XDK
>> Develop, test and display web and hybrid apps with a single code base.
>> Download it for free now!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>
>
> ------------------------------------------------------------------------------
> Sponsored by Intel(R) XDK
> Develop, test and display web and hybrid apps with a single code base.
> Download it for free now!
> http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel
>

------------------------------------------------------------------------------
Sponsored by Intel(R) XDK 
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Thanks for the info, that clears things up. The mutexes have to synchronize the collections across instances, so I'll take them out of the module calls and put then in static initializers, which is where the OpenMP code had them.

Best,
Mike

Hi Michael,

I pulled, compiled, and ran through the same sequence of testing I was
doing earlier (disk render going concurrently with me stopping and
starting realtime renders over and over).  Looks good!  I got no
crashes at all.

Thanks for the quick turnaround!
steve

On Thu, Dec 5, 2013 at 2:17 AM, Michael Gogins  wrote:
> Thanks for the info, that clears things up. The mutexes have to synchronize
> the collections across instances, so I'll take them out of the module calls
> and put then in static initializers, which is where the OpenMP code had
> them.
>
> Best,
> Mike
>
> On Dec 5, 2013 12:50 AM, "Steven Yi"  wrote:
>>
>> If I understand your email correctly, yes.  The problem is that the
>> create and destroy module callbacks are run per instance. So let's say
>> you have 2 csound instances, you can start both and they will have
>> both called csoundModuleCreate.  Then you stop and destroy both and
>> they will have both called csoundModuleDestroy.  Since the the lib
>> uses static vars, it's possible to get two csoundModuleDestroy calls
>> in a row, thus the mutexes on the second time would try to free a null
>> pointer. That's why I was thinking they should store the mutex with
>> the instance of Csound rather than be static.
>>
>> On Wed, Dec 4, 2013 at 5:31 PM, Michael Gogins 
>> wrote:
>> > These mutexes need to have the same scope exactly as the plugin opcode
>> > module itself. I have been assuming that that is per process, ie one
>> > process
>> > may have more than one instance of Csound, but all of the instances in
>> > one
>> > process would share the module and the static variables. Is this the
>> > case?
>> >
>> > Best,
>> > Mike
>> >
>> > On Dec 4, 2013 10:53 PM, "Steven Yi"  wrote:
>> >>
>> >> Hi All,
>> >>
>> >> I've been diagnosing various crashes in Blue reported to me and that
>> >> I've found in running Blue.  The one due to the ->self has greatly
>> >> improved things, but there are still other crashes.
>> >>
>> >> I have found one today due to the signal flow graph opcodes:
>> >>
>> >>
>> >> Thread 10 Crashed:: Java: Finalizer
>> >> 0   libsystem_kernel.dylib         0x00007fff90568866 __pthread_kill +
>> >> 10
>> >> 1   libsystem_pthread.dylib       0x00007fff8f7bc35c pthread_kill + 92
>> >> 2   libsystem_c.dylib             0x00007fff96441bba abort + 125
>> >> 3   libsystem_malloc.dylib         0x00007fff8ea80093 free + 411
>> >> 4                                 0x0000000114f425d3
>> >> csoundDestroyMutex + 51 (threads.c:533)
>> >> 5   libsignalflowgraph.dylib       0x000000011c038a86
>> >> csoundModuleDestroy + 1350 (signalflowgraph.cpp:1710)
>> >> 6                                 0x0000000114f30fce
>> >> csoundDestroyModules + 110 (csmodule.c:646)
>> >> 7                                 0x0000000114f32fdd reset + 189
>> >> (csound.c:2650)
>> >> 8                                 0x0000000114f32e85 csoundDestroy +
>> >> 229 (csound.c:1206)
>> >> 9   lib_jcsound6.jnilib           0x0000000114ce44b2 Csound::~Csound()
>> >> + 34 (csound.hpp:856)
>> >> 10  lib_jcsound6.jnilib           0x0000000114ce693e Csound::~Csound()
>> >> + 14 (csound.hpp:856)
>> >> 11  lib_jcsound6.jnilib           0x0000000114ce6956 Csound::~Csound()
>> >> + 22 (csound.hpp:849)
>> >> 12  lib_jcsound6.jnilib           0x0000000114ccfd5c
>> >> Java_csnd6_csnd6JNI_delete_1Csound + 76
>> >> (java_interfaceJAVA_wrap.cxx:12858)
>> >> 13  ???                           0x0000000105b79698 0 + 4390885016
>> >> 14  ???                           0x0000000105b6d058 0 + 4390834264
>> >> 15  ???                           0x0000000105b6d058 0 + 4390834264
>> >> 16  ???                           0x0000000105b674e7 0 + 4390810855
>> >> 17  libjvm.dylib                   0x00000001052d6d90
>> >> JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*,
>> >> Thread*) + 554
>> >> 18  libjvm.dylib                   0x00000001052d6b60
>> >> JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*)
>> >> + 40
>> >> 19  libjvm.dylib                   0x000000010530a605
>> >> jni_invoke_nonstatic(JNIEnv_*, JavaValue*, _jobject*, JNICallType,
>> >> _jmethodID*, JNI_ArgumentPusher*, Thread*) + 698
>> >> 20  libjvm.dylib                   0x00000001052fe176
>> >> jni_CallVoidMethod +
>> >> 278
>> >> 21  libjava.dylib                 0x00000001047cb99c
>> >> Java_java_lang_ref_Finalizer_invokeFinalizeMethod + 88
>> >> 22  ???                           0x0000000105b79698 0 + 4390885016
>> >> 23  ???                           0x0000000105b6d058 0 + 4390834264
>> >> 24  ???                           0x0000000105b6d058 0 + 4390834264
>> >> 25  ???                           0x0000000105b6d058 0 + 4390834264
>> >> 26  ???                           0x0000000105b674e7 0 + 4390810855
>> >> 27  libjvm.dylib                   0x00000001052d6d90
>> >> JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*,
>> >> Thread*) + 554
>> >> 28  libjvm.dylib                   0x00000001052d72a7
>> >> JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*,
>> >> JavaCallArguments*, Thread*) + 283
>> >> 29  libjvm.dylib                   0x00000001052d73e4
>> >> JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*,
>> >> Symbol*, Thread*) + 74
>> >> 30  libjvm.dylib                   0x00000001053263ca
>> >> thread_entry(JavaThread*, Thread*) + 173
>> >> 31  libjvm.dylib                   0x00000001054efb47
>> >> JavaThread::thread_main_inner() + 155
>> >> 32  libjvm.dylib                   0x00000001054f124f JavaThread::run()
>> >> +
>> >> 419
>> >> 33  libjvm.dylib                   0x000000010541b1c6
>> >> java_start(Thread*)
>> >> + 294
>> >> 34  libsystem_pthread.dylib       0x00007fff8f7bb899 _pthread_body +
>> >> 138
>> >> 35  libsystem_pthread.dylib       0x00007fff8f7bb72a _pthread_start +
>> >> 137
>> >>
>> >>
>> >> At the time, I had concurrent Csound instances running (one rendering
>> >> to disk, repeated start/stopping of Csound in realtime).  The code in
>> >> signalflowgraph.cpp contains:
>> >>
>> >> static void* cs_sfg_ftables;
>> >> static void* cs_sfg_ports;
>> >>
>> >> which is apparently not thread-safe.
>> >>
>> >> Michael: Could you take a look at this?  My assumption for a fix is
>> >> that those mutexes should be stored in/retrieved from Csound using
>> >> csoundCreateGlobalVariable/csoundQueryGlobalVariable.
>> >>
>> >> Thanks,
>> >> steven
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Sponsored by Intel(R) XDK
>> >> Develop, test and display web and hybrid apps with a single code base.
>> >> Download it for free now!
>> >>
>> >>
>> >> http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
>> >> _______________________________________________
>> >> Csound-devel mailing list
>> >> Csound-devel@lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Sponsored by Intel(R) XDK
>> > Develop, test and display web and hybrid apps with a single code base.
>> > Download it for free now!
>> >
>> > http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
>> > _______________________________________________
>> > Csound-devel mailing list
>> > Csound-devel@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Sponsored by Intel(R) XDK
>> Develop, test and display web and hybrid apps with a single code base.
>> Download it for free now!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>
>
> ------------------------------------------------------------------------------
> Sponsored by Intel(R) XDK
> Develop, test and display web and hybrid apps with a single code base.
> Download it for free now!
> http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel
>

------------------------------------------------------------------------------
Sponsored by Intel(R) XDK 
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net