Csound Csound-dev Csound-tekno Search About

Re: [Cs-dev] Multi-threading spinlocks

Date2008-08-10 21:56
Fromvictor
SubjectRe: [Cs-dev] Multi-threading spinlocks
The SConstruct check was incorrect, the function it needed to check
wasn't seen. I committed a fix.  Can you see if it detects the
sync test function?


----- Original Message ----- 
From: "victor" 
To: "Developer discussions" 
Sent: Sunday, August 10, 2008 9:01 PM
Subject: Re: [Cs-dev] Multi-threading spinlocks


> Now configure does not pass the stdio.h test.
>
> Checking for C header file stdio.h... no
> *** Failed to compile a simple test program. The compiler is
> *** possibly not set up correctly, or is used with invalid flags.
> *** Check config.log to find out more about the error.
>
>
> config.log
> file C:\msys\1.0\home\Victor\csound5\SConstruct,line 621:
> Configure(confdir = .sconf_temp)
> scons: Configure: Checking for C header file stdio.h...
> scons: Configure: ".sconf_temp\conftest_0.c" is up to date.
> scons: Configure: The original builder output was:
>  |.sconf_temp\conftest_0.c <-
>  |  |
>  |  |#include 
>  |  |
>  |  |
>  |
> gcc -o
> .sconf_temp\conftest_0.o -c -Wall -DNOGETTEXT -g -fomit-frame-pointer -D_WIN32
> -DWIN32 -DPIPES -DOS_IS_WIN32 -mthreads -DSDFT -DBETA -I. -IH -IC:\msys\1.0\local\include
> -IC:\msys\1.0\include -IC:\Python25\include "-IC:\Program
> Files\java\jdk1.6.0_06\include" "-IC:\Program
> Files\java\jdk1.6.0_06\include\win32" -IC:\msys\1.0\local\java\include -IC:\msys\1.0\local\java\include\win32
> -IC:\usr\local\include -IC:\usr\include .sconf_temp\conftest_0.c
> gcc: .sconf_temp\conftest_0.c: No such file or directory
> gcc: no input files
> scons: Configure: no
>
>
> So??
>
> Victor
> ---- Original Message ----- 
> From: "Michael Gogins" 
> To: "victor" 
> Sent: Sunday, August 10, 2008 8:27 PM
> Subject: Re: [Cs-dev] Multi-threading spinlocks
>
>
>> That's not a complete cleanup. You need to run cleanup.sh.
>>
>> You might be able to get by by deleting the configuration stuff
>> (.sconf_temp dir and config.log).
>>
>> If this works, it does the #ifdef that you want.
>>
>> Regards,
>> Mike
>>
>> -----Original Message-----
>>>From: victor 
>>>Sent: Aug 10, 2008 3:02 PM
>>>To: Michael Gogins 
>>>Subject: Re: [Cs-dev] Multi-threading spinlocks
>>>
>>>scons -c & scons
>>>
>>>same thing. Can this code be protected by ifdefs? It would be nice
>>>to be able to build csound regardless of  compiler version.
>>>
>>>Victor
>>>----- Original Message ----- 
>>>From: "Michael Gogins" 
>>>To: "victor" 
>>>Sent: Sunday, August 10, 2008 7:58 PM
>>>Subject: Re: [Cs-dev] Multi-threading spinlocks
>>>
>>>
>>>> This should have failed the configure test. Try a complete blow away 
>>>> and
>>>> retest?...
>>>>
>>>> What version of GCC are you using?
>>>>
>>>> Regards,
>>>> Mike
>>>>
>>>> -----Original Message-----
>>>>>From: victor 
>>>>>Sent: Aug 10, 2008 2:38 PM
>>>>>To: Michael Gogins , Developer discussions
>>>>>
>>>>>Subject: Re: [Cs-dev] Multi-threading spinlocks
>>>>>
>>>>>Fails to build here, on windows/mingw
>>>>>
>>>>>OOps\aops.o(.text+0x3531): In function `in':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:952: undefined reference to
>>>>>`__sync_
>>>>>lock_test_and_set'
>>>>>OOps\aops.o(.text+0x3572):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:954:
>>>>>undef
>>>>>ined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x35a1): In function `ins':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:963: undefined reference to
>>>>>`__sync_
>>>>>lock_test_and_set'
>>>>>OOps\aops.o(.text+0x3647):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:971:
>>>>>undef
>>>>>ined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x36b1): In function `inq':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:986: undefined reference to
>>>>>`__sync_
>>>>>lock_test_and_set'
>>>>>OOps\aops.o(.text+0x377e):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:993:
>>>>>undef
>>>>>ined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x397c): In function `ino':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1035: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x3ad1):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1046:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x3b19): In function `inn':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1058: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x3b83):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1063:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x3c49): In function `inch_opcode':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1085: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x3cbf):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1092:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x3d23): In function `inall_opcode':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1102: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x3e0a):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1112:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x3e47): In function `out':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1121: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x3ef8):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1130:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x3f50): In function `outs':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1141: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x409e):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1157:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x40d2): In function `outq':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1166: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x42f9):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1191:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x4346): In function `outs1':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1202: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x443e):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1217:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x448b): In function `outs2':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1227: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x4583):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1242:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x45d0): In function `outs12':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1252: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x4701):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1268:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x474e): In function `outq1':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1279: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x486d):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1296:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x48ba): In function `outq2':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1306: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x49d9):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1323:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x4a26): In function `outq3':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1333: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x4b45):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1350:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x4b92): In function `outq4':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1360: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x4cb1):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1377:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x4d35): In function `outh':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1392: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x4fca):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1416:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x5064): In function `outo':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1433: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x53a1):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1461:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x53e5): In function `outn':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1470: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x557f):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1491:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>OOps\aops.o(.text+0x5673): In function `outch':
>>>>>C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1522: undefined reference 
>>>>>to
>>>>>`__sync
>>>>>_lock_test_and_set'
>>>>>OOps\aops.o(.text+0x580a):C:/msys/1.0/home/Victor/csound5/OOps/aops.c:1545:
>>>>>unde
>>>>>fined reference to `__sync_lock_release'
>>>>>collect2: ld returned 1 exit status
>>>>>scons: *** [csound32.dll.5.1] Error 1
>>>>>scons: building terminated because of errors.
>>>>>
>>>>>----- Original Message ----- 
>>>>>From: "Michael Gogins" 
>>>>>To: "Michael Gogins" ; "Developer discussions"
>>>>>
>>>>>Sent: Sunday, August 10, 2008 7:31 PM
>>>>>Subject: Re: [Cs-dev] Multi-threading spinlocks
>>>>>
>>>>>
>>>>>>I should have added...
>>>>>>
>>>>>> There may be some bug fixes required for odd numbers of threads. But,
>>>>>> by
>>>>>> the next release, multi-threading should be usable with many Csound
>>>>>> orchestras, and will show a real performance gain even on machines
>>>>>> with
>>>>>> only 2 cores.
>>>>>>
>>>>>> Regards,
>>>>>> Mike
>>>>>>
>>>>>> -----Original Message-----
>>>>>>>From: Michael Gogins 
>>>>>>>Sent: Aug 10, 2008 2:03 PM
>>>>>>>To: Csound Developers 
>>>>>>>Subject: [Cs-dev] Multi-threading spinlocks
>>>>>>>
>>>>>>>I have committed to Csound CVS implementations of spinlocks in
>>>>>>>csound.h,
>>>>>>>based on compiler intrinsics, for MSVC and GCC. The spinlock macros
>>>>>>>compile to no-ops if the compiler does not have the required
>>>>>>>intrinsics.
>>>>>>>Note that GCC 4.1 and later has these intrinsics, but I think GCC 3.4
>>>>>>>does
>>>>>>>not have them.
>>>>>>>
>>>>>>>(Note: a compiler intrinsic is a function for which code is emitted
>>>>>>>directly by the compiler, instead of being linked in from a library.
>>>>>>>Current compilers use intrinsics for atomic operations and sometimes
>>>>>>>for
>>>>>>>elementary math functions, bit twiddling, memory operations, and 
>>>>>>>other
>>>>>>>things.)
>>>>>>>
>>>>>>>I have used these spinlocks to protect the spin and spout buffers in
>>>>>>>all
>>>>>>>in and out opcodes. I have not done anything for other shared global
>>>>>>>data,
>>>>>>>although the channel, mixer, and table opcodes would be obvious next
>>>>>>>steps.
>>>>>>>
>>>>>>>Steven Yi raised the possibility of using private spout buffers for
>>>>>>>out
>>>>>>>opcode instances, to be reduced after multiple opcodes have been
>>>>>>>mapped
>>>>>>>to
>>>>>>>different threads. However, I decided to go ahead and commit 
>>>>>>>spinlocks
>>>>>>>for
>>>>>>>the following reasons:
>>>>>>>
>>>>>>>(1) Spinlocks have proved useful and scalable in scenarios where the
>>>>>>>functions that are protected typically execute for only a short time.
>>>>>>>That
>>>>>>>is exactly the case here. Typically, the out opcodes add a few to few
>>>>>>>hundred samples, and are done. This is usually a rather small part of
>>>>>>>the
>>>>>>>total overhead for an instrument instance.
>>>>>>>
>>>>>>>(2) Because of this small amount of time required for out opcodes
>>>>>>>compared
>>>>>>>to other computation in the thread, the likelihood of actually having
>>>>>>>to
>>>>>>>wait in a spinlock is also rather small. This is verified by the
>>>>>>>almost
>>>>>>>complete lack of clicks or pops in the multi-thread testing that we
>>>>>>>have
>>>>>>>done to date, without any protection for spout.
>>>>>>>
>>>>>>>(3) By contrast, private spout buffers would involve adding spout 
>>>>>>>once
>>>>>>>for
>>>>>>>each thread. I do not know the exact overhead of the atomic compare
>>>>>>>and
>>>>>>>swap intrinsic used in the spinlock, but I think it is probably
>>>>>>>comparable
>>>>>>>to the cost of adding a few sample frames. More information on this
>>>>>>>question would be appreciated! But if I am right, then as soon as
>>>>>>>ksmps
>>>>>>>is
>>>>>>>much greater than 1, spinlocks are more efficient than private 
>>>>>>>spouts.
>>>>>>>
>>>>>>>(4) My tests show no appreciable performance impact from spinlocks
>>>>>>>with
>>>>>>>1
>>>>>>>or 2 threads.
>>>>>>>
>>>>>>>(5) As the number of cores increases, the value of spinlocks 
>>>>>>>increases
>>>>>>>even faster, because they involve no operating system locks at all.
>>>>>>>The
>>>>>>>only OS overhead is memory scheduling, which would also occur with OS
>>>>>>>locks.
>>>>>>>
>>>>>>>If anyone has compile-time, run-time, or engineering problems with 
>>>>>>>the
>>>>>>>spinlocks, please let us know.
>>>>>>>
>>>>>>>If anyone has an alternative implementation of thread-safety for the
>>>>>>>out
>>>>>>>and in opcodes, please benchmark it against the spinlock
>>>>>>>implementation
>>>>>>>before committing it.
>>>>>>>
>>>>>>>Regards,
>>>>>>>Mike
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>-------------------------------------------------------------------------
>>>>>>>This SF.Net email is sponsored by the Moblin Your Move Developer's
>>>>>>>challenge
>>>>>>>Build the coolest Linux based applications with Moblin SDK & win 
>>>>>>>great
>>>>>>>prizes
>>>>>>>Grand prize is a trip for two to an Open Source event anywhere in the
>>>>>>>world
>>>>>>>http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>>>>>_______________________________________________
>>>>>>>Csound-devel mailing list
>>>>>>>Csound-devel@lists.sourceforge.net
>>>>>>>https://lists.sourceforge.net/lists/listinfo/csound-devel
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -------------------------------------------------------------------------
>>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>>>>>> challenge
>>>>>> Build the coolest Linux based applications with Moblin SDK & win 
>>>>>> great
>>>>>> prizes
>>>>>> Grand prize is a trip for two to an Open Source event anywhere in the
>>>>>> world
>>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>>>> _______________________________________________
>>>>>> Csound-devel mailing list
>>>>>> Csound-devel@lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's 
> challenge
> Build the coolest Linux based applications with Moblin SDK & win great 
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the 
> world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel 


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net