Csound Csound-dev Csound-tekno Search About

Re: [Cs-dev] Multi-threading spinlocks

Date2008-08-11 00:09
FromMichael Gogins
SubjectRe: [Cs-dev] Multi-threading spinlocks
Your test does not work for me. It must have to do with int32_t. Change your test to check for (int *) 0, 0 instead of 0,0 in the sync call. But this will be no good if we build on 64 bit systems.

Hope this helps,
Mike

-----Original Message-----
>From: Michael Gogins 
>Sent: Aug 10, 2008 7:05 PM
>To: Developer discussions 
>Subject: Re: [Cs-dev] Multi-threading spinlocks
>
>I am quite sure this is not because of anything I did. I only added two lines to SConstruct, they are identical in structure to existing lines, and they worked for me. Either you need to do the whole cleanup.sh thing, or your custom.py changed or something.
>
>Hope this helps,
>Mike
>
>-----Original Message-----
>>From: victor 
>>Sent: Aug 10, 2008 4:01 PM
>>To: Developer discussions 
>>Subject: Re: [Cs-dev] Multi-threading spinlocks
>>
>>Now configure does not pass the stdio.h test.
>>
>>Checking for C header file stdio.h... no
>> *** Failed to compile a simple test program. The compiler is
>> *** possibly not set up correctly, or is used with invalid flags.
>> *** Check config.log to find out more about the error.
>>
>>
>>config.log
>>file C:\msys\1.0\home\Victor\csound5\SConstruct,line 621:
>> Configure(confdir = .sconf_temp)
>>scons: Configure: Checking for C header file stdio.h...
>>scons: Configure: ".sconf_temp\conftest_0.c" is up to date.
>>scons: Configure: The original builder output was:
>>  |.sconf_temp\conftest_0.c <-
>>  |  |
>>  |  |#include 
>>  |  |
>>  |  |
>>  |
>>gcc -o 
>>.sconf_temp\conftest_0.o -c -Wall -DNOGETTEXT -g -fomit-frame-pointer -D_WIN32 
>> -DWIN32 -DPIPES -DOS_IS_WIN32 -mthreads -DSDFT -DBETA -I. -IH -IC:\msys\1.0\local\include 
>> -IC:\msys\1.0\include -IC:\Python25\include "-IC:\Program 
>>Files\java\jdk1.6.0_06\include" "-IC:\Program 
>>Files\java\jdk1.6.0_06\include\win32" -IC:\msys\1.0\local\java\include -IC:\msys\1.0\local\java\include\win32 
>> -IC:\usr\local\include -IC:\usr\include .sconf_temp\conftest_0.c
>>gcc: .sconf_temp\conftest_0.c: No such file or directory
>>gcc: no input files
>>scons: Configure: no
>>
>>
>>So??
>>
>>Victor
>>---- Original Message ----- 
>>From: "Michael Gogins" 
>>To: "victor" 
>>Sent: Sunday, August 10, 2008 8:27 PM
>>Subject: Re: [Cs-dev] Multi-threading spinlocks
>>
>>
>>> That's not a complete cleanup. You need to run cleanup.sh.
>>>
>>> You might be able to get by by deleting the configuration stuff 
>>> (.sconf_temp dir and config.log).
>>>
>>> If this works, it does the #ifdef that you want.
>>>
>>> Regards,
>>> Mike
>>>
>>> -----Original Message-----
>>>>From: victor 
>>>>Sent: Aug 10, 2008 3:02 PM
>>>>To: Michael Gogins 
>>>>Subject: Re: [Cs-dev] Multi-threading spinlocks
>>>>
>>>>scons -c & scons
>>>>
>>>>same thing. Can this code be protected by ifdefs? It would be nice
>>>>to be able to build csound regardless of  compiler version.
>>>>
>>>>Victor
>>>>----- Original Message ----- 
>>>>From: "Michael Gogins" 
>>>>To: "victor" 
>>>>Sent: Sunday, August 10, 2008 7:58 PM
>>>>Subject: Re: [Cs-dev] Multi-threading spinlocks
>>>>
>>>>
>>>>> This should have failed the configure test. Try a complete blow away and
>>>>> retest?...
>>>>>
>>>>> What version of GCC are you using?
>>>>>
>>>>> Regards,
>>>>> Mike
>>>>>
>>>>> -----Original Message-----
>>>>>>From: victor 
>>>>>>Sent: Aug 10, 2008 2:38 PM
>>>>>>To: Michael Gogins , Developer discussions
>>>>>>
>>>>>>Subject: Re: [Cs-dev] Multi-threading spinlocks
>>>>>>
>>>>>>Fails to build here, on windows/mingw
>>>>>>
>>>>>>OOps\aops.o(.text+0x3531): In function `in':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:952: undefined reference to
>>>>>>`__sync_
>>>>>>lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x3572):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:954:
>>>>>>undef
>>>>>>ined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x35a1): In function `ins':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:963: undefined reference to
>>>>>>`__sync_
>>>>>>lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x3647):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:971:
>>>>>>undef
>>>>>>ined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x36b1): In function `inq':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:986: undefined reference to
>>>>>>`__sync_
>>>>>>lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x377e):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:993:
>>>>>>undef
>>>>>>ined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x397c): In function `ino':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1035: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x3ad1):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1046:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x3b19): In function `inn':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1058: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x3b83):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1063:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x3c49): In function `inch_opcode':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1085: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x3cbf):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1092:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x3d23): In function `inall_opcode':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1102: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x3e0a):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1112:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x3e47): In function `out':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1121: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x3ef8):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1130:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x3f50): In function `outs':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1141: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x409e):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1157:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x40d2): In function `outq':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1166: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x42f9):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1191:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x4346): In function `outs1':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1202: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x443e):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1217:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x448b): In function `outs2':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1227: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x4583):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1242:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x45d0): In function `outs12':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1252: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x4701):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1268:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x474e): In function `outq1':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1279: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x486d):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1296:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x48ba): In function `outq2':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1306: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x49d9):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1323:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x4a26): In function `outq3':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1333: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x4b45):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1350:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x4b92): In function `outq4':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1360: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x4cb1):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1377:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x4d35): In function `outh':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1392: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x4fca):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1416:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x5064): In function `outo':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1433: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x53a1):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1461:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x53e5): In function `outn':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1470: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x557f):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1491:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>OOps\aops.o(.text+0x5673): In function `outch':
>>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1522: undefined reference to
>>>>>>`__sync
>>>>>>_lock_test_and_set'
>>>>>>OOps\aops.o(.text+0x580a):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1545:
>>>>>>unde
>>>>>>fined reference to `__sync_lock_release'
>>>>>>collect2: ld returned 1 exit status
>>>>>>scons: *** [csound32.dll.5.1] Error 1
>>>>>>scons: building terminated because of errors.
>>>>>>
>>>>>>----- Original Message ----- 
>>>>>>From: "Michael Gogins" 
>>>>>>To: "Michael Gogins" ; "Developer discussions"
>>>>>>
>>>>>>Sent: Sunday, August 10, 2008 7:31 PM
>>>>>>Subject: Re: [Cs-dev] Multi-threading spinlocks
>>>>>>
>>>>>>
>>>>>>>I should have added...
>>>>>>>
>>>>>>> There may be some bug fixes required for odd numbers of threads. But, 
>>>>>>> by
>>>>>>> the next release, multi-threading should be usable with many Csound
>>>>>>> orchestras, and will show a real performance gain even on machines 
>>>>>>> with
>>>>>>> only 2 cores.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Mike
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>>>From: Michael Gogins 
>>>>>>>>Sent: Aug 10, 2008 2:03 PM
>>>>>>>>To: Csound Developers 
>>>>>>>>Subject: [Cs-dev] Multi-threading spinlocks
>>>>>>>>
>>>>>>>>I have committed to Csound CVS implementations of spinlocks in 
>>>>>>>>csound.h,
>>>>>>>>based on compiler intrinsics, for MSVC and GCC. The spinlock macros
>>>>>>>>compile to no-ops if the compiler does not have the required 
>>>>>>>>intrinsics.
>>>>>>>>Note that GCC 4.1 and later has these intrinsics, but I think GCC 3.4
>>>>>>>>does
>>>>>>>>not have them.
>>>>>>>>
>>>>>>>>(Note: a compiler intrinsic is a function for which code is emitted
>>>>>>>>directly by the compiler, instead of being linked in from a library.
>>>>>>>>Current compilers use intrinsics for atomic operations and sometimes 
>>>>>>>>for
>>>>>>>>elementary math functions, bit twiddling, memory operations, and other
>>>>>>>>things.)
>>>>>>>>
>>>>>>>>I have used these spinlocks to protect the spin and spout buffers in 
>>>>>>>>all
>>>>>>>>in and out opcodes. I have not done anything for other shared global
>>>>>>>>data,
>>>>>>>>although the channel, mixer, and table opcodes would be obvious next
>>>>>>>>steps.
>>>>>>>>
>>>>>>>>Steven Yi raised the possibility of using private spout buffers for 
>>>>>>>>out
>>>>>>>>opcode instances, to be reduced after multiple opcodes have been 
>>>>>>>>mapped
>>>>>>>>to
>>>>>>>>different threads. However, I decided to go ahead and commit spinlocks
>>>>>>>>for
>>>>>>>>the following reasons:
>>>>>>>>
>>>>>>>>(1) Spinlocks have proved useful and scalable in scenarios where the
>>>>>>>>functions that are protected typically execute for only a short time.
>>>>>>>>That
>>>>>>>>is exactly the case here. Typically, the out opcodes add a few to few
>>>>>>>>hundred samples, and are done. This is usually a rather small part of
>>>>>>>>the
>>>>>>>>total overhead for an instrument instance.
>>>>>>>>
>>>>>>>>(2) Because of this small amount of time required for out opcodes
>>>>>>>>compared
>>>>>>>>to other computation in the thread, the likelihood of actually having 
>>>>>>>>to
>>>>>>>>wait in a spinlock is also rather small. This is verified by the 
>>>>>>>>almost
>>>>>>>>complete lack of clicks or pops in the multi-thread testing that we 
>>>>>>>>have
>>>>>>>>done to date, without any protection for spout.
>>>>>>>>
>>>>>>>>(3) By contrast, private spout buffers would involve adding spout once
>>>>>>>>for
>>>>>>>>each thread. I do not know the exact overhead of the atomic compare 
>>>>>>>>and
>>>>>>>>swap intrinsic used in the spinlock, but I think it is probably
>>>>>>>>comparable
>>>>>>>>to the cost of adding a few sample frames. More information on this
>>>>>>>>question would be appreciated! But if I am right, then as soon as 
>>>>>>>>ksmps
>>>>>>>>is
>>>>>>>>much greater than 1, spinlocks are more efficient than private spouts.
>>>>>>>>
>>>>>>>>(4) My tests show no appreciable performance impact from spinlocks 
>>>>>>>>with
>>>>>>>>1
>>>>>>>>or 2 threads.
>>>>>>>>
>>>>>>>>(5) As the number of cores increases, the value of spinlocks increases
>>>>>>>>even faster, because they involve no operating system locks at all. 
>>>>>>>>The
>>>>>>>>only OS overhead is memory scheduling, which would also occur with OS
>>>>>>>>locks.
>>>>>>>>
>>>>>>>>If anyone has compile-time, run-time, or engineering problems with the
>>>>>>>>spinlocks, please let us know.
>>>>>>>>
>>>>>>>>If anyone has an alternative implementation of thread-safety for the 
>>>>>>>>out
>>>>>>>>and in opcodes, please benchmark it against the spinlock 
>>>>>>>>implementation
>>>>>>>>before committing it.
>>>>>>>>
>>>>>>>>Regards,
>>>>>>>>Mike
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>-------------------------------------------------------------------------
>>>>>>>>This SF.Net email is sponsored by the Moblin Your Move Developer's
>>>>>>>>challenge
>>>>>>>>Build the coolest Linux based applications with Moblin SDK & win great
>>>>>>>>prizes
>>>>>>>>Grand prize is a trip for two to an Open Source event anywhere in the
>>>>>>>>world
>>>>>>>>http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>>>>>>_______________________________________________
>>>>>>>>Csound-devel mailing list
>>>>>>>>Csound-devel@lists.sourceforge.net
>>>>>>>>https://lists.sourceforge.net/lists/listinfo/csound-devel
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -------------------------------------------------------------------------
>>>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>>>>>>> challenge
>>>>>>> Build the coolest Linux based applications with Moblin SDK & win great
>>>>>>> prizes
>>>>>>> Grand prize is a trip for two to an Open Source event anywhere in the
>>>>>>> world
>>>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>>>>> _______________________________________________
>>>>>>> Csound-devel mailing list
>>>>>>> Csound-devel@lists.sourceforge.net
>>>>>>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> 
>>
>>
>>-------------------------------------------------------------------------
>>This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>Build the coolest Linux based applications with Moblin SDK & win great prizes
>>Grand prize is a trip for two to an Open Source event anywhere in the world
>>http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>_______________________________________________
>>Csound-devel mailing list
>>Csound-devel@lists.sourceforge.net
>>https://lists.sourceforge.net/lists/listinfo/csound-devel
>
>
>
>
>-------------------------------------------------------------------------
>This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>Build the coolest Linux based applications with Moblin SDK & win great prizes
>Grand prize is a trip for two to an Open Source event anywhere in the world
>http://moblin-contest.org/redirect.php?banner_id=100&url=/
>_______________________________________________
>Csound-devel mailing list
>Csound-devel@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/csound-devel




-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net