Csound Csound-dev Csound-tekno Search About

[Csnd] More on running parallel csound processes

Date2024-01-26 03:29
FromTobiah
Subject[Csnd] More on running parallel csound processes
Linux on a Thinkpad with an AMD Ryzen 7 PRO 6850U.
Appears as 16 CPU's.

bench.csd:  http://tinyurl.com/n6fw5j7c

I run bench.csd, and it takes about 9.6 seconds.
Then, I do this:

         for each in `seq 1 16`; do
                 csound bench.csd &
         done

I see 16 cpu's in 'htop' all cranking at 100% usage.
It takes about 113 seconds to complete, so faster then
running sequentially (about 153 seconds), but not stellar.

Then I try a C program:

         #include 

         int count = 120000;

         int main(){
                 for(int x = 0; x < count; ++x){
                         for(int y = 0; y < count; ++y){
                         }
                 }
         }

cc wait.c -o wait

for each in `seq 1 16`; do
         ./wait&
done


Running 'wait' takes about 7 seconds to complete, and
running 16 in parallel takes about 8 seconds.  It's
almost like running the processes on 16 different computers.

So why does csound benefit so much less from using multiple cores?


Toby

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2024-01-26 07:02
FromVictor Lazzarini
SubjectRe: [Csnd] [EXTERNAL] [Csnd] More on running parallel csound processes
I'd say there are other things happening like memory allocation and access that can cause a bottleneck, as opposed to a counter, which is just counting and nothing else.

Btw, on my Mac I got a much bigger speed up. I reported it here.

One thing you can measure and check is Csound running in multiple threads versus multiple processes, by writing a Csound host program that creates several Csound instances and runs them in separate threads.


Prof. Victor Lazzarini
Maynooth University
Ireland

> On 26 Jan 2024, at 03:30, Tobiah  wrote:
>
> *Warning*
>
> This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
>
> Linux on a Thinkpad with an AMD Ryzen 7 PRO 6850U.
> Appears as 16 CPU's.
>
> bench.csd:  http://tinyurl.com/n6fw5j7c
>
> I run bench.csd, and it takes about 9.6 seconds.
> Then, I do this:
>
>        for each in `seq 1 16`; do
>                csound bench.csd &
>        done
>
> I see 16 cpu's in 'htop' all cranking at 100% usage.
> It takes about 113 seconds to complete, so faster then
> running sequentially (about 153 seconds), but not stellar.
>
> Then I try a C program:
>
>        #include 
>
>        int count = 120000;
>
>        int main(){
>                for(int x = 0; x < count; ++x){
>                        for(int y = 0; y < count; ++y){
>                        }
>                }
>        }
>
> cc wait.c -o wait
>
> for each in `seq 1 16`; do
>        ./wait&
> done
>
>
> Running 'wait' takes about 7 seconds to complete, and
> running 16 in parallel takes about 8 seconds.  It's
> almost like running the processes on 16 different computers.
>
> So why does csound benefit so much less from using multiple cores?
>
>
> Toby
>
> Csound mailing list
> Csound@listserv.heanet.ie
> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
> Send bugs reports to
>       https://github.com/csound/csound/issues
> Discussions of bugs and features can be posted here

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2024-01-27 03:00
FromTobiah
SubjectRe: [Csnd] [EXTERNAL] [Csnd] More on running parallel csound processes
On 1/25/24 23:02, Victor Lazzarini wrote:
> I'd say there are other things happening like memory allocation and
> access that can cause a bottleneck, as opposed to a counter, which is
> just counting and nothing else.

The allocation seems to happen quite quickly at the beginning, but
I can imagine the bottle neck being memory I/O.

I made another .csd with just one instrument running one oscilator
for 3000 score-seconds.  It took about 7 seconds to render, while
running 16 of those processes in parallel took about 20 seconds.  Much greater
benefit from the multiple cores then previously with the thousands
of instrument instances.  Memory size was never an issue, hardly making
a dent in the 32 gig installed.

By the way, I've been running these tests with -n (no write to disk)
but writing 16 separate 3000 second sound files (~8 Gig) to disk made little
difference in render time.  Yay for nvme disks!


Toby

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2024-01-27 09:13
FromVictor Lazzarini
SubjectRe: [Csnd] [EXTERNAL] [Csnd] More on running parallel csound processes
I think that difference shows a bottleneck  at memory access in the mixing of the signals at the output.

It would be interesting to see what happens if you remove the "out" opcode line, and run the instrument without sending the sound to the output.

Prof. Victor Lazzarini
Maynooth University
Ireland

> On 27 Jan 2024, at 03:01, Tobiah  wrote:
>
> On 1/25/24 23:02, Victor Lazzarini wrote:
>> I'd say there are other things happening like memory allocation and
>> access that can cause a bottleneck, as opposed to a counter, which is
>> just counting and nothing else.
>
> The allocation seems to happen quite quickly at the beginning, but
> I can imagine the bottle neck being memory I/O.
>
> I made another .csd with just one instrument running one oscilator
> for 3000 score-seconds.  It took about 7 seconds to render, while
> running 16 of those processes in parallel took about 20 seconds.  Much greater
> benefit from the multiple cores then previously with the thousands
> of instrument instances.  Memory size was never an issue, hardly making
> a dent in the 32 gig installed.
>
> By the way, I've been running these tests with -n (no write to disk)
> but writing 16 separate 3000 second sound files (~8 Gig) to disk made little
> difference in render time.  Yay for nvme disks!
>
>
> Toby
>
> Csound mailing list
> Csound@listserv.heanet.ie
> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
> Send bugs reports to
>       https://github.com/csound/csound/issues
> Discussions of bugs and features can be posted here

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here