Hi Ben,

I ran this here and got a long pause after parsing as well.  I do not
get a segfault however; it's possible that the segfault may have been
fixed with a kcounter fix that Victor applied.  I'll run Instruments
with CS5 and CS6 now to see if I can find out where the performance
issue is.

Just a note, all of the new compiler and semantic checker changes are
not really optimized; there's a number of places where we can replace
data structures used with much more efficient ones.  I know personally
that I added a number of things that used linked lists because it made
it easier to code, but at least they're known places to optimize.

Thanks!
steven

On Fri, Apr 19, 2013 at 10:55 AM, Ben Hackbarth  wrote:
> hello,
>
> i am continuing the process of trying to debug a large orchestra in csound6.
> i am getting segfaults which i am having trouble chasing down, but i have a
> large number of strsets (~12000) and i am suspicious that their quantity is
> playing a role is bringing my csd down.  in my large project, around 14000
> lines, there is a delay of ~15 seconds between the "Parsing successful!"
> message and the segfault.
>
> anyhow, to test i've made a csd with 10000 strset commands.  with csound 5.2
> i can compile the csd in 0m1.250s.  with csound6, it takes 0m5.548s.  so i
> suspect something is a foot here.
>
> http://pastebin.com/Q1Qx2pzT
>
> if i manage to find a more concrete reason for my segfaults, i'll post
> again.
>
> thanks.
> --  ben
>
> ------------------------------------------------------------------------------
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel
>

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Hi Ben,

It looks like the source of the problem is as I mentioned.  In this
case, CS6 has a different mechanism for it's string constants pool
than CS5, and it's currently using a linked list.  Moving to using a
hash set structure should do the job.

To make life simpler, we should probably add a
csound_data_structures.c/.h for reusable data structures.  I'll see
about starting a stub for that now.

steven

On Fri, Apr 19, 2013 at 5:52 PM, Steven Yi  wrote:
> Hi Ben,
>
> I ran this here and got a long pause after parsing as well.  I do not
> get a segfault however; it's possible that the segfault may have been
> fixed with a kcounter fix that Victor applied.  I'll run Instruments
> with CS5 and CS6 now to see if I can find out where the performance
> issue is.
>
> Just a note, all of the new compiler and semantic checker changes are
> not really optimized; there's a number of places where we can replace
> data structures used with much more efficient ones.  I know personally
> that I added a number of things that used linked lists because it made
> it easier to code, but at least they're known places to optimize.
>
> Thanks!
> steven
>
> On Fri, Apr 19, 2013 at 10:55 AM, Ben Hackbarth  wrote:
>> hello,
>>
>> i am continuing the process of trying to debug a large orchestra in csound6.
>> i am getting segfaults which i am having trouble chasing down, but i have a
>> large number of strsets (~12000) and i am suspicious that their quantity is
>> playing a role is bringing my csd down.  in my large project, around 14000
>> lines, there is a delay of ~15 seconds between the "Parsing successful!"
>> message and the segfault.
>>
>> anyhow, to test i've made a csd with 10000 strset commands.  with csound 5.2
>> i can compile the csd in 0m1.250s.  with csound6, it takes 0m5.548s.  so i
>> suspect something is a foot here.
>>
>> http://pastebin.com/Q1Qx2pzT
>>
>> if i manage to find a more concrete reason for my segfaults, i'll post
>> again.
>>
>> thanks.
>> --  ben
>>
>> ------------------------------------------------------------------------------
>> Precog is a next-generation analytics platform capable of advanced
>> analytics on semi-structured data. The platform includes APIs for building
>> apps and a phenomenal toolset for data science. Developers can use
>> our toolset for easy data analysis & visualization. Get a free account!
>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>>

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Hi Ben,

I've put in a generic hash table/hash set into Csound6 (in
Engine/csound_data_structures.c).  It reuses the hash function that
currently existed in Csound's codebase.  I've replaced STRING_POOL
with CS_HASH_TABLE and it looks like CS6 is parsing/compiling about as
fast as it did with CS5, at least for your example.

Please pull from GIT and let me know if you run into any issues.

Thanks!
steven


On Sun, Apr 21, 2013 at 5:22 PM, Ben Hackbarth  wrote:
> ok, great.  let me know when you've pushed something.  i'll try it out of my
> end and report back.
>
> --  ben
>
>
> On Fri, Apr 19, 2013 at 7:14 PM, Steven Yi  wrote:
>>
>> Hi Ben,
>>
>> It looks like the source of the problem is as I mentioned.  In this
>> case, CS6 has a different mechanism for it's string constants pool
>> than CS5, and it's currently using a linked list.  Moving to using a
>> hash set structure should do the job.
>>
>> To make life simpler, we should probably add a
>> csound_data_structures.c/.h for reusable data structures.  I'll see
>> about starting a stub for that now.
>>
>> steven
>>
>> On Fri, Apr 19, 2013 at 5:52 PM, Steven Yi  wrote:
>> > Hi Ben,
>> >
>> > I ran this here and got a long pause after parsing as well.  I do not
>> > get a segfault however; it's possible that the segfault may have been
>> > fixed with a kcounter fix that Victor applied.  I'll run Instruments
>> > with CS5 and CS6 now to see if I can find out where the performance
>> > issue is.
>> >
>> > Just a note, all of the new compiler and semantic checker changes are
>> > not really optimized; there's a number of places where we can replace
>> > data structures used with much more efficient ones.  I know personally
>> > that I added a number of things that used linked lists because it made
>> > it easier to code, but at least they're known places to optimize.
>> >
>> > Thanks!
>> > steven
>> >
>> > On Fri, Apr 19, 2013 at 10:55 AM, Ben Hackbarth 
>> > wrote:
>> >> hello,
>> >>
>> >> i am continuing the process of trying to debug a large orchestra in
>> >> csound6.
>> >> i am getting segfaults which i am having trouble chasing down, but i
>> >> have a
>> >> large number of strsets (~12000) and i am suspicious that their
>> >> quantity is
>> >> playing a role is bringing my csd down.  in my large project, around
>> >> 14000
>> >> lines, there is a delay of ~15 seconds between the "Parsing
>> >> successful!"
>> >> message and the segfault.
>> >>
>> >> anyhow, to test i've made a csd with 10000 strset commands.  with
>> >> csound 5.2
>> >> i can compile the csd in 0m1.250s.  with csound6, it takes 0m5.548s.
>> >> so i
>> >> suspect something is a foot here.
>> >>
>> >> http://pastebin.com/Q1Qx2pzT
>> >>
>> >> if i manage to find a more concrete reason for my segfaults, i'll post
>> >> again.
>> >>
>> >> thanks.
>> >> --  ben
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Precog is a next-generation analytics platform capable of advanced
>> >> analytics on semi-structured data. The platform includes APIs for
>> >> building
>> >> apps and a phenomenal toolset for data science. Developers can use
>> >> our toolset for easy data analysis & visualization. Get a free account!
>> >> http://www2.precog.com/precogplatform/slashdotnewsletter
>> >> _______________________________________________
>> >> Csound-devel mailing list
>> >> Csound-devel@lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >>
>>
>>
>> ------------------------------------------------------------------------------
>> Precog is a next-generation analytics platform capable of advanced
>> analytics on semi-structured data. The platform includes APIs for building
>> apps and a phenomenal toolset for data science. Developers can use
>> our toolset for easy data analysis & visualization. Get a free account!
>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>
>
>
> ------------------------------------------------------------------------------
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel
>

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Thanks for the confirmation!

On Mon, Apr 29, 2013 at 6:10 PM, Ben Hackbarth  wrote:
> hi steven,
>
> just gave this a spin.  it works.  and the delay incurred from filling the
> hash table now seems on par with that of csound5.
>
> thanks!
>
> --  ben
>
>
> On Mon, Apr 22, 2013 at 7:21 PM, Steven Yi  wrote:
>>
>> Hi Ben,
>>
>> I've put in a generic hash table/hash set into Csound6 (in
>> Engine/csound_data_structures.c).  It reuses the hash function that
>> currently existed in Csound's codebase.  I've replaced STRING_POOL
>> with CS_HASH_TABLE and it looks like CS6 is parsing/compiling about as
>> fast as it did with CS5, at least for your example.
>>
>> Please pull from GIT and let me know if you run into any issues.
>>
>> Thanks!
>> steven
>>
>>
>> On Sun, Apr 21, 2013 at 5:22 PM, Ben Hackbarth 
>> wrote:
>> > ok, great.  let me know when you've pushed something.  i'll try it out
>> > of my
>> > end and report back.
>> >
>> > --  ben
>> >
>> >
>> > On Fri, Apr 19, 2013 at 7:14 PM, Steven Yi  wrote:
>> >>
>> >> Hi Ben,
>> >>
>> >> It looks like the source of the problem is as I mentioned.  In this
>> >> case, CS6 has a different mechanism for it's string constants pool
>> >> than CS5, and it's currently using a linked list.  Moving to using a
>> >> hash set structure should do the job.
>> >>
>> >> To make life simpler, we should probably add a
>> >> csound_data_structures.c/.h for reusable data structures.  I'll see
>> >> about starting a stub for that now.
>> >>
>> >> steven
>> >>
>> >> On Fri, Apr 19, 2013 at 5:52 PM, Steven Yi  wrote:
>> >> > Hi Ben,
>> >> >
>> >> > I ran this here and got a long pause after parsing as well.  I do not
>> >> > get a segfault however; it's possible that the segfault may have been
>> >> > fixed with a kcounter fix that Victor applied.  I'll run Instruments
>> >> > with CS5 and CS6 now to see if I can find out where the performance
>> >> > issue is.
>> >> >
>> >> > Just a note, all of the new compiler and semantic checker changes are
>> >> > not really optimized; there's a number of places where we can replace
>> >> > data structures used with much more efficient ones.  I know
>> >> > personally
>> >> > that I added a number of things that used linked lists because it
>> >> > made
>> >> > it easier to code, but at least they're known places to optimize.
>> >> >
>> >> > Thanks!
>> >> > steven
>> >> >
>> >> > On Fri, Apr 19, 2013 at 10:55 AM, Ben Hackbarth 
>> >> > wrote:
>> >> >> hello,
>> >> >>
>> >> >> i am continuing the process of trying to debug a large orchestra in
>> >> >> csound6.
>> >> >> i am getting segfaults which i am having trouble chasing down, but i
>> >> >> have a
>> >> >> large number of strsets (~12000) and i am suspicious that their
>> >> >> quantity is
>> >> >> playing a role is bringing my csd down.  in my large project, around
>> >> >> 14000
>> >> >> lines, there is a delay of ~15 seconds between the "Parsing
>> >> >> successful!"
>> >> >> message and the segfault.
>> >> >>
>> >> >> anyhow, to test i've made a csd with 10000 strset commands.  with
>> >> >> csound 5.2
>> >> >> i can compile the csd in 0m1.250s.  with csound6, it takes 0m5.548s.
>> >> >> so i
>> >> >> suspect something is a foot here.
>> >> >>
>> >> >> http://pastebin.com/Q1Qx2pzT
>> >> >>
>> >> >> if i manage to find a more concrete reason for my segfaults, i'll
>> >> >> post
>> >> >> again.
>> >> >>
>> >> >> thanks.
>> >> >> --  ben
>> >> >>
>> >> >>
>> >> >>
>> >> >> ------------------------------------------------------------------------------
>> >> >> Precog is a next-generation analytics platform capable of advanced
>> >> >> analytics on semi-structured data. The platform includes APIs for
>> >> >> building
>> >> >> apps and a phenomenal toolset for data science. Developers can use
>> >> >> our toolset for easy data analysis & visualization. Get a free
>> >> >> account!
>> >> >> http://www2.precog.com/precogplatform/slashdotnewsletter
>> >> >> _______________________________________________
>> >> >> Csound-devel mailing list
>> >> >> Csound-devel@lists.sourceforge.net
>> >> >> https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >> >>
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Precog is a next-generation analytics platform capable of advanced
>> >> analytics on semi-structured data. The platform includes APIs for
>> >> building
>> >> apps and a phenomenal toolset for data science. Developers can use
>> >> our toolset for easy data analysis & visualization. Get a free account!
>> >> http://www2.precog.com/precogplatform/slashdotnewsletter
>> >> _______________________________________________
>> >> Csound-devel mailing list
>> >> Csound-devel@lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Precog is a next-generation analytics platform capable of advanced
>> > analytics on semi-structured data. The platform includes APIs for
>> > building
>> > apps and a phenomenal toolset for data science. Developers can use
>> > our toolset for easy data analysis & visualization. Get a free account!
>> > http://www2.precog.com/precogplatform/slashdotnewsletter
>> > _______________________________________________
>> > Csound-devel mailing list
>> > Csound-devel@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Precog is a next-generation analytics platform capable of advanced
>> analytics on semi-structured data. The platform includes APIs for building
>> apps and a phenomenal toolset for data science. Developers can use
>> our toolset for easy data analysis & visualization. Get a free account!
>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel
>

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net