Csound Csound-dev Csound-tekno Search About

[Cs-dev] Memory problem in Csound 6

Date2013-09-13 12:28
FromMichael Gogins
Subject[Cs-dev] Memory problem in Csound 6
AttachmentsBlue_Leaves_6.log  trapped.log  Blue_Leaves_6.cpp  None  None  
I have discovered a serious memory problem in Csound 6 that is preventing me from composing as I used to with Csound 5.

I have a piece written in C++ (Blue_Leaves_6.cpp, attached) which uses the Csound C++ API and CsoundAC and no other dependencies. If you compile and link with this csound, csnd6, and CsoundAC it should run with flags --midi --csound and produce a MIDI file version of a generated score and a Csound-generated WAV soundfile. In fact I have submitted an earlier version of this piece to the Csound 2013 conference. 

The build command for my piece on Windows is:

cd D:\Dropbox\Csound2013-mkg\Gogins && g++ Blue_Leaves_6.cpp -o Blue_Leaves_6 -O2 -g -fopenmp -mtune=native -mstackrealign -std=c++0x -ID:/msys/local/include -IC:/Users/mkg.sorabji/csound-csound6-git/include -IC:/Users/mkg.sorabji/csound-csound6-git/interfaces -IC:/Users/mkg.sorabji/csound-csound6-git/frontends/CsoundAC -LC:/Users/mkg.sorabji/csound-csound6-git -lCsoundAC -lcsnd6 -lcsound64 -lpthread -lm

When I run this piece compiled with Csound 6 on Windows it runs out of memory and dies. In the Windows performance monitor, you can see the memory load climbing rapidly until the piece crashes. To diagnose what is happening, I inserted a print statement in insert.c at line 1965:

    pextent = sizeof(INSDS) + tp->pextrab + pextra*sizeof(MYFLT *);
    printf("mcalloc(csound [%p], (size_t) pextent [%d] + tp->varPool->poolSize [%d] + tp->opdstot [%d]);\n", csound, pextent, tp->varPool->poolSize, tp->opdstot);
    ip = (INSDS*) mcalloc(csound, (size_t) pextent + tp->varPool->poolSize + tp->opdstot);

I have attached log files showing runs of trapped.csd and my piece. Either the memory management in Csound 6 is no longer suitable for complex orchestras as in my piece, or there is a bug in the memory management or type system.

Any help is most appreciated.

Regards,
Mike

===========================
Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com

Date2013-09-13 12:40
FromVictor Lazzarini
SubjectRe: [Cs-dev] Memory problem in Csound 6
Hi Michael, 

one thing that I saw in your Csound code and I don't understand

sr                              =                       96000
ksmps                           =             1000000

that would mean a kr > sr, would it not ? How come Csound did not complain?

Victor
On 13 Sep 2013, at 12:28, Michael Gogins  wrote:

> I have discovered a serious memory problem in Csound 6 that is preventing me from composing as I used to with Csound 5.
> 
> I have a piece written in C++ (Blue_Leaves_6.cpp, attached) which uses the Csound C++ API and CsoundAC and no other dependencies. If you compile and link with this csound, csnd6, and CsoundAC it should run with flags --midi --csound and produce a MIDI file version of a generated score and a Csound-generated WAV soundfile. In fact I have submitted an earlier version of this piece to the Csound 2013 conference. 
> 
> The build command for my piece on Windows is:
> 
> cd D:\Dropbox\Csound2013-mkg\Gogins && g++ Blue_Leaves_6.cpp -o Blue_Leaves_6 -O2 -g -fopenmp -mtune=native -mstackrealign -std=c++0x -ID:/msys/local/include -IC:/Users/mkg.sorabji/csound-csound6-git/include -IC:/Users/mkg.sorabji/csound-csound6-git/interfaces -IC:/Users/mkg.sorabji/csound-csound6-git/frontends/CsoundAC -LC:/Users/mkg.sorabji/csound-csound6-git -lCsoundAC -lcsnd6 -lcsound64 -lpthread -lm
> 
> When I run this piece compiled with Csound 6 on Windows it runs out of memory and dies. In the Windows performance monitor, you can see the memory load climbing rapidly until the piece crashes. To diagnose what is happening, I inserted a print statement in insert.c at line 1965:
> 
>     pextent = sizeof(INSDS) + tp->pextrab + pextra*sizeof(MYFLT *);
>     printf("mcalloc(csound [%p], (size_t) pextent [%d] + tp->varPool->poolSize [%d] + tp->opdstot [%d]);\n", csound, pextent, tp->varPool->poolSize, tp->opdstot);
>     ip = (INSDS*) mcalloc(csound, (size_t) pextent + tp->varPool->poolSize + tp->opdstot);
> 
> I have attached log files showing runs of trapped.csd and my piece. Either the memory management in Csound 6 is no longer suitable for complex orchestras as in my piece, or there is a bug in the memory management or type system.
> 
> Any help is most appreciated.
> 
> Regards,
> Mike
> 
> ===========================
> Michael Gogins
> Irreducible Productions
> http://michaelgogins.tumblr.com
> Michael dot Gogins at gmail dot com
> ------------------------------------------------------------------------------
> How ServiceNow helps IT people transform IT departments:
> 1. Consolidate legacy IT systems to a single system of record for IT
> 2. Standardize and globalize service processes across IT
> 3. Implement zero-touch automation to replace manual, redundant tasks
> http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk_______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel


------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Date2013-09-13 12:54
FromVictor Lazzarini
SubjectRe: [Cs-dev] Memory problem in Csound 6
actually it doesn't mean that, I am sorry, but I think to have ksmps > sr might be problematic
anyway.

Victor

On 13 Sep 2013, at 12:40, Victor Lazzarini wrote:

> Hi Michael, 
> 
> one thing that I saw in your Csound code and I don't understand
> 
> sr                              =                       96000
> ksmps                           =             1000000
> 
> that would mean a kr > sr, would it not ? How come Csound did not complain?
> 
> Victor
> On 13 Sep 2013, at 12:28, Michael Gogins  wrote:
> 
>> I have discovered a serious memory problem in Csound 6 that is preventing me from composing as I used to with Csound 5.
>> 
>> I have a piece written in C++ (Blue_Leaves_6.cpp, attached) which uses the Csound C++ API and CsoundAC and no other dependencies. If you compile and link with this csound, csnd6, and CsoundAC it should run with flags --midi --csound and produce a MIDI file version of a generated score and a Csound-generated WAV soundfile. In fact I have submitted an earlier version of this piece to the Csound 2013 conference. 
>> 
>> The build command for my piece on Windows is:
>> 
>> cd D:\Dropbox\Csound2013-mkg\Gogins && g++ Blue_Leaves_6.cpp -o Blue_Leaves_6 -O2 -g -fopenmp -mtune=native -mstackrealign -std=c++0x -ID:/msys/local/include -IC:/Users/mkg.sorabji/csound-csound6-git/include -IC:/Users/mkg.sorabji/csound-csound6-git/interfaces -IC:/Users/mkg.sorabji/csound-csound6-git/frontends/CsoundAC -LC:/Users/mkg.sorabji/csound-csound6-git -lCsoundAC -lcsnd6 -lcsound64 -lpthread -lm
>> 
>> When I run this piece compiled with Csound 6 on Windows it runs out of memory and dies. In the Windows performance monitor, you can see the memory load climbing rapidly until the piece crashes. To diagnose what is happening, I inserted a print statement in insert.c at line 1965:
>> 
>>    pextent = sizeof(INSDS) + tp->pextrab + pextra*sizeof(MYFLT *);
>>    printf("mcalloc(csound [%p], (size_t) pextent [%d] + tp->varPool->poolSize [%d] + tp->opdstot [%d]);\n", csound, pextent, tp->varPool->poolSize, tp->opdstot);
>>    ip = (INSDS*) mcalloc(csound, (size_t) pextent + tp->varPool->poolSize + tp->opdstot);
>> 
>> I have attached log files showing runs of trapped.csd and my piece. Either the memory management in Csound 6 is no longer suitable for complex orchestras as in my piece, or there is a bug in the memory management or type system.
>> 
>> Any help is most appreciated.
>> 
>> Regards,
>> Mike
>> 
>> ===========================
>> Michael Gogins
>> Irreducible Productions
>> http://michaelgogins.tumblr.com
>> Michael dot Gogins at gmail dot com
>> ------------------------------------------------------------------------------
>> How ServiceNow helps IT people transform IT departments:
>> 1. Consolidate legacy IT systems to a single system of record for IT
>> 2. Standardize and globalize service processes across IT
>> 3. Implement zero-touch automation to replace manual, redundant tasks
>> http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk_______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
> 
> 
> ------------------------------------------------------------------------------
> How ServiceNow helps IT people transform IT departments:
> 1. Consolidate legacy IT systems to a single system of record for IT
> 2. Standardize and globalize service processes across IT
> 3. Implement zero-touch automation to replace manual, redundant tasks
> http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel

Dr Victor Lazzarini
Senior Lecturer
Dept. of Music
NUI Maynooth Ireland
tel.: +353 1 708 3545
Victor dot Lazzarini AT nuim dot ie




------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Date2013-09-13 12:58
FromVictor Lazzarini
SubjectRe: [Cs-dev] Memory problem in Csound 6
It does cause problems (both in Csound 6 and Csound 5), so I think, for now, it should not be allowed. 
I've modified the consistency check to give an error for this.

Victor
On 13 Sep 2013, at 12:54, Victor Lazzarini wrote:

> actually it doesn't mean that, I am sorry, but I think to have ksmps > sr might be problematic
> anyway.
> 
> Victor
> 
> On 13 Sep 2013, at 12:40, Victor Lazzarini wrote:
> 
>> Hi Michael, 
>> 
>> one thing that I saw in your Csound code and I don't understand
>> 
>> sr                              =                       96000
>> ksmps                           =             1000000
>> 
>> that would mean a kr > sr, would it not ? How come Csound did not complain?
>> 
>> Victor
>> On 13 Sep 2013, at 12:28, Michael Gogins  wrote:
>> 
>>> I have discovered a serious memory problem in Csound 6 that is preventing me from composing as I used to with Csound 5.
>>> 
>>> I have a piece written in C++ (Blue_Leaves_6.cpp, attached) which uses the Csound C++ API and CsoundAC and no other dependencies. If you compile and link with this csound, csnd6, and CsoundAC it should run with flags --midi --csound and produce a MIDI file version of a generated score and a Csound-generated WAV soundfile. In fact I have submitted an earlier version of this piece to the Csound 2013 conference. 
>>> 
>>> The build command for my piece on Windows is:
>>> 
>>> cd D:\Dropbox\Csound2013-mkg\Gogins && g++ Blue_Leaves_6.cpp -o Blue_Leaves_6 -O2 -g -fopenmp -mtune=native -mstackrealign -std=c++0x -ID:/msys/local/include -IC:/Users/mkg.sorabji/csound-csound6-git/include -IC:/Users/mkg.sorabji/csound-csound6-git/interfaces -IC:/Users/mkg.sorabji/csound-csound6-git/frontends/CsoundAC -LC:/Users/mkg.sorabji/csound-csound6-git -lCsoundAC -lcsnd6 -lcsound64 -lpthread -lm
>>> 
>>> When I run this piece compiled with Csound 6 on Windows it runs out of memory and dies. In the Windows performance monitor, you can see the memory load climbing rapidly until the piece crashes. To diagnose what is happening, I inserted a print statement in insert.c at line 1965:
>>> 
>>>   pextent = sizeof(INSDS) + tp->pextrab + pextra*sizeof(MYFLT *);
>>>   printf("mcalloc(csound [%p], (size_t) pextent [%d] + tp->varPool->poolSize [%d] + tp->opdstot [%d]);\n", csound, pextent, tp->varPool->poolSize, tp->opdstot);
>>>   ip = (INSDS*) mcalloc(csound, (size_t) pextent + tp->varPool->poolSize + tp->opdstot);
>>> 
>>> I have attached log files showing runs of trapped.csd and my piece. Either the memory management in Csound 6 is no longer suitable for complex orchestras as in my piece, or there is a bug in the memory management or type system.
>>> 
>>> Any help is most appreciated.
>>> 
>>> Regards,
>>> Mike
>>> 
>>> ===========================
>>> Michael Gogins
>>> Irreducible Productions
>>> http://michaelgogins.tumblr.com
>>> Michael dot Gogins at gmail dot com
>>> ------------------------------------------------------------------------------
>>> How ServiceNow helps IT people transform IT departments:
>>> 1. Consolidate legacy IT systems to a single system of record for IT
>>> 2. Standardize and globalize service processes across IT
>>> 3. Implement zero-touch automation to replace manual, redundant tasks
>>> http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk_______________________________________________
>>> Csound-devel mailing list
>>> Csound-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>> 
>> 
>> ------------------------------------------------------------------------------
>> How ServiceNow helps IT people transform IT departments:
>> 1. Consolidate legacy IT systems to a single system of record for IT
>> 2. Standardize and globalize service processes across IT
>> 3. Implement zero-touch automation to replace manual, redundant tasks
>> http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
> 
> Dr Victor Lazzarini
> Senior Lecturer
> Dept. of Music
> NUI Maynooth Ireland
> tel.: +353 1 708 3545
> Victor dot Lazzarini AT nuim dot ie
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> How ServiceNow helps IT people transform IT departments:
> 1. Consolidate legacy IT systems to a single system of record for IT
> 2. Standardize and globalize service processes across IT
> 3. Implement zero-touch automation to replace manual, redundant tasks
> http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel

Dr Victor Lazzarini
Senior Lecturer
Dept. of Music
NUI Maynooth Ireland
tel.: +353 1 708 3545
Victor dot Lazzarini AT nuim dot ie




------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Date2013-09-13 13:48
FromMichael Gogins
SubjectRe: [Cs-dev] Memory problem in Csound 6
AttachmentsNone  None  
Thanks. I will try my test again with ksmps reasonable. That would perhaps explain things... probably a typo.

Regards,
Mike


===========================
Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com


On Fri, Sep 13, 2013 at 7:58 AM, Victor Lazzarini <Victor.Lazzarini@nuim.ie> wrote:
It does cause problems (both in Csound 6 and Csound 5), so I think, for now, it should not be allowed.
I've modified the consistency check to give an error for this.

Victor
On 13 Sep 2013, at 12:54, Victor Lazzarini wrote:

> actually it doesn't mean that, I am sorry, but I think to have ksmps > sr might be problematic
> anyway.
>
> Victor
>
> On 13 Sep 2013, at 12:40, Victor Lazzarini wrote:
>
>> Hi Michael,
>>
>> one thing that I saw in your Csound code and I don't understand
>>
>> sr                              =                       96000
>> ksmps                           =             1000000
>>
>> that would mean a kr > sr, would it not ? How come Csound did not complain?
>>
>> Victor
>> On 13 Sep 2013, at 12:28, Michael Gogins <michael.gogins@gmail.com> wrote:
>>
>>> I have discovered a serious memory problem in Csound 6 that is preventing me from composing as I used to with Csound 5.
>>>
>>> I have a piece written in C++ (Blue_Leaves_6.cpp, attached) which uses the Csound C++ API and CsoundAC and no other dependencies. If you compile and link with this csound, csnd6, and CsoundAC it should run with flags --midi --csound and produce a MIDI file version of a generated score and a Csound-generated WAV soundfile. In fact I have submitted an earlier version of this piece to the Csound 2013 conference.
>>>
>>> The build command for my piece on Windows is:
>>>
>>> cd D:\Dropbox\Csound2013-mkg\Gogins && g++ Blue_Leaves_6.cpp -o Blue_Leaves_6 -O2 -g -fopenmp -mtune=native -mstackrealign -std=c++0x -ID:/msys/local/include -IC:/Users/mkg.sorabji/csound-csound6-git/include -IC:/Users/mkg.sorabji/csound-csound6-git/interfaces -IC:/Users/mkg.sorabji/csound-csound6-git/frontends/CsoundAC -LC:/Users/mkg.sorabji/csound-csound6-git -lCsoundAC -lcsnd6 -lcsound64 -lpthread -lm
>>>
>>> When I run this piece compiled with Csound 6 on Windows it runs out of memory and dies. In the Windows performance monitor, you can see the memory load climbing rapidly until the piece crashes. To diagnose what is happening, I inserted a print statement in insert.c at line 1965:
>>>
>>>   pextent = sizeof(INSDS) + tp->pextrab + pextra*sizeof(MYFLT *);
>>>   printf("mcalloc(csound [%p], (size_t) pextent [%d] + tp->varPool->poolSize [%d] + tp->opdstot [%d]);\n", csound, pextent, tp->varPool->poolSize, tp->opdstot);
>>>   ip = (INSDS*) mcalloc(csound, (size_t) pextent + tp->varPool->poolSize + tp->opdstot);
>>>
>>> I have attached log files showing runs of trapped.csd and my piece. Either the memory management in Csound 6 is no longer suitable for complex orchestras as in my piece, or there is a bug in the memory management or type system.
>>>
>>> Any help is most appreciated.
>>>
>>> Regards,
>>> Mike
>>>
>>> ===========================
>>> Michael Gogins
>>> Irreducible Productions
>>> http://michaelgogins.tumblr.com
>>> Michael dot Gogins at gmail dot com
>>> <Blue_Leaves_6.log><trapped.log><Blue_Leaves_6.cpp>------------------------------------------------------------------------------
>>> How ServiceNow helps IT people transform IT departments:
>>> 1. Consolidate legacy IT systems to a single system of record for IT
>>> 2. Standardize and globalize service processes across IT
>>> 3. Implement zero-touch automation to replace manual, redundant tasks
>>> http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk_______________________________________________
>>> Csound-devel mailing list
>>> Csound-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>>
>>
>> ------------------------------------------------------------------------------
>> How ServiceNow helps IT people transform IT departments:
>> 1. Consolidate legacy IT systems to a single system of record for IT
>> 2. Standardize and globalize service processes across IT
>> 3. Implement zero-touch automation to replace manual, redundant tasks
>> http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>
> Dr Victor Lazzarini
> Senior Lecturer
> Dept. of Music
> NUI Maynooth Ireland
> tel.: +353 1 708 3545
> Victor dot Lazzarini AT nuim dot ie
>
>
>
>
> ------------------------------------------------------------------------------
> How ServiceNow helps IT people transform IT departments:
> 1. Consolidate legacy IT systems to a single system of record for IT
> 2. Standardize and globalize service processes across IT
> 3. Implement zero-touch automation to replace manual, redundant tasks
> http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel

Dr Victor Lazzarini
Senior Lecturer
Dept. of Music
NUI Maynooth Ireland
tel.: +353 1 708 3545
Victor dot Lazzarini AT nuim dot ie




------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/csound-devel