Hi All,

Question: Why is there a max channels variable?  I'm looking at the
code and it's using a hash table that should have no limits.  It looks
like the max channels was being used for the max bucket size of the
hash table, but I don't see any reason why there should be a limit. I
think this should just be replaced with a CS_HASH_TABLE and all
limitations removed, but this is my first time through this code.
Does anyone have any perspective on this?

Thanks!
steven

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Is this to do with bus channel names?
On 24 Apr 2013, at 12:33, Steven Yi wrote:

> Hi All,
> 
> Question: Why is there a max channels variable?  I'm looking at the
> code and it's using a hash table that should have no limits.  It looks
> like the max channels was being used for the max bucket size of the
> hash table, but I don't see any reason why there should be a limit. I
> think this should just be replaced with a CS_HASH_TABLE and all
> limitations removed, but this is my first time through this code.
> Does anyone have any perspective on this?
> 
> Thanks!
> steven
> 
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service 
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel

Dr Victor Lazzarini
Senior Lecturer
Dept. of Music
NUI Maynooth Ireland
tel.: +353 1 708 3545
Victor dot Lazzarini AT nuim dot ie




------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

I think so.  I think this may be related to Andres' merging of the
different channel systems into one system, and that the old
chani/chano may have used a channel[256] array.  I think it's now
merged to all use the named channel interface, and so perhaps that
channel num limit may just no longer apply. That's what I'm
understanding from the code so far at least.

On Wed, Apr 24, 2013 at 12:52 PM, Victor Lazzarini
 wrote:
> Is this to do with bus channel names?
> On 24 Apr 2013, at 12:33, Steven Yi wrote:
>
>> Hi All,
>>
>> Question: Why is there a max channels variable?  I'm looking at the
>> code and it's using a hash table that should have no limits.  It looks
>> like the max channels was being used for the max bucket size of the
>> hash table, but I don't see any reason why there should be a limit. I
>> think this should just be replaced with a CS_HASH_TABLE and all
>> limitations removed, but this is my first time through this code.
>> Does anyone have any perspective on this?
>>
>> Thanks!
>> steven
>>
>> ------------------------------------------------------------------------------
>> Try New Relic Now & We'll Send You this Cool Shirt
>> New Relic is the only SaaS-based application performance monitoring service
>> that delivers powerful full stack analytics. Optimize and monitor your
>> browser, app, & servers with just a few lines of code. Try New Relic
>> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>
> Dr Victor Lazzarini
> Senior Lecturer
> Dept. of Music
> NUI Maynooth Ireland
> tel.: +353 1 708 3545
> Victor dot Lazzarini AT nuim dot ie
>
>
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Yes, it would be great to remove that limitation (it was inherited from the existing chn mechanism). I think it would be important to minimize allocation during performance. How does the hash table mechanism work?

Cheers,
Andres

Hi Andres,

The CS_HASH_TABLE used follows typical hash table implementations in
that it uses an array of buckets.  A name is hashed to create an index
into the array of buckets, and each bucket contains a linked list of
items.  The linked list handles hash collisions where different names
give the same hash value.  Insertion then does hash => lookup bucket
=> search for item already defined for key and if found replace value,
or append new item to linked list.  Hash lookup follows the same hash
=> lookup bucket => search list.

Csound had a lot of different hash implementations and differently
sized buckets.  I've just made a generic CS_HASH_TABLE out of existing
code such that the key is always char* and the value is a void*.  NULL
values are allowed, but NULL keys are not.  These are all using a 4099
bucket size, which should provide less collisions than the 256 used in
a number of other places.

The code for channels is also using its own hash table.  I'll look at
replacing that now and removing the limitation on CS_MAX_CHANNELS.

Thanks!
steven

On Wed, Apr 24, 2013 at 3:41 PM, Andres Cabrera  wrote:
> Yes, it would be great to remove that limitation (it was inherited from the
> existing chn mechanism). I think it would be important to minimize
> allocation during performance. How does the hash table mechanism work?
>
> Cheers,
> Andres
>
> On Apr 24, 2013 5:01 AM, "Steven Yi"  wrote:
>>
>> I think so.  I think this may be related to Andres' merging of the
>> different channel systems into one system, and that the old
>> chani/chano may have used a channel[256] array.  I think it's now
>> merged to all use the named channel interface, and so perhaps that
>> channel num limit may just no longer apply. That's what I'm
>> understanding from the code so far at least.
>>
>> On Wed, Apr 24, 2013 at 12:52 PM, Victor Lazzarini
>>  wrote:
>> > Is this to do with bus channel names?
>> > On 24 Apr 2013, at 12:33, Steven Yi wrote:
>> >
>> >> Hi All,
>> >>
>> >> Question: Why is there a max channels variable?  I'm looking at the
>> >> code and it's using a hash table that should have no limits.  It looks
>> >> like the max channels was being used for the max bucket size of the
>> >> hash table, but I don't see any reason why there should be a limit. I
>> >> think this should just be replaced with a CS_HASH_TABLE and all
>> >> limitations removed, but this is my first time through this code.
>> >> Does anyone have any perspective on this?
>> >>
>> >> Thanks!
>> >> steven
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Try New Relic Now & We'll Send You this Cool Shirt
>> >> New Relic is the only SaaS-based application performance monitoring
>> >> service
>> >> that delivers powerful full stack analytics. Optimize and monitor your
>> >> browser, app, & servers with just a few lines of code. Try New Relic
>> >> and get this awesome Nerd Life shirt!
>> >> http://p.sf.net/sfu/newrelic_d2d_apr
>> >> _______________________________________________
>> >> Csound-devel mailing list
>> >> Csound-devel@lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >
>> > Dr Victor Lazzarini
>> > Senior Lecturer
>> > Dept. of Music
>> > NUI Maynooth Ireland
>> > tel.: +353 1 708 3545
>> > Victor dot Lazzarini AT nuim dot ie
>> >
>> >
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Try New Relic Now & We'll Send You this Cool Shirt
>> > New Relic is the only SaaS-based application performance monitoring
>> > service
>> > that delivers powerful full stack analytics. Optimize and monitor your
>> > browser, app, & servers with just a few lines of code. Try New Relic
>> > and get this awesome Nerd Life shirt!
>> > http://p.sf.net/sfu/newrelic_d2d_apr
>> > _______________________________________________
>> > Csound-devel mailing list
>> > Csound-devel@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/csound-devel
>>
>>
>> ------------------------------------------------------------------------------
>> Try New Relic Now & We'll Send You this Cool Shirt
>> New Relic is the only SaaS-based application performance monitoring
>> service
>> that delivers powerful full stack analytics. Optimize and monitor your
>> browser, app, & servers with just a few lines of code. Try New Relic
>> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel
>

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Hi Justin,

A few notes regarding allocation here:

1) In general, allocation happens only once per channel name.  Most of
the cost in the channel mechanism then is in searching for the channel
rather than allocating it.
2) mmalloc does not do anything regarding threads.  It's a wrapper to
malloc but also holds a reference to the memory, so that it can be
released when the mem pool is freed.
3) The channel system, as it was before today, was doing allocation.
I think before Andres' merge of the different channel systems, that
there may have been a pre-allocation of the channel block of 256
numbered channels.
4) With csound 6's realtime mode, init passes of opcodes will be run
on their own thread, so that any long-running calculation will not
block the audio thread.
5) However... the numbered channel opcodes allowed k-rate changes,
such that they require getting the channel pointer each k-pass.  This
could be updated to use a caching strategy, but this won't be helped
by the cs6 realtime mode.

The change I did today only refactored what was already there,
replacing the manual hash code lookup and custom struct for holding
channels with using the generic CS_HASH_TABLE.  Performance wise,
there should be no difference between what was there and what I put
in.  Now regarding malloc on the realtime thread, that could be a
possible concern, but I'd be surprised if anyone runs into a dropout.
They'd have to be allocating a lot of new channels in a single k-pass
I think.

I don't remember what the API and internals looked like for channels
before Andres' changes, but if this doesn't work out, I'm sure some
changes could be made. For example, with CS_HASH_TABLE, because it
holds a void*, a 256 sized channel block could be allocated like it
was before, then added with a special NUM_CHAN_BLOCK key to the hash
table.  Then the old system could be like it used to (those opcodes
could get the channel block from the table, then index into it).  So
there's some options if it's necessary.

steven

On Wed, Apr 24, 2013 at 5:50 PM, Justin Smith  wrote:
> Isn't the concern here that one shouldn't call malloc in realtime threads?
> Is this obviated by csmalloc?
>
>
> On Wed, Apr 24, 2013 at 9:42 AM, Steven Yi  wrote:
>>
>> Hi Andres,
>>
>> The CS_HASH_TABLE used follows typical hash table implementations in
>> that it uses an array of buckets.  A name is hashed to create an index
>> into the array of buckets, and each bucket contains a linked list of
>> items.  The linked list handles hash collisions where different names
>> give the same hash value.  Insertion then does hash => lookup bucket
>> => search for item already defined for key and if found replace value,
>> or append new item to linked list.  Hash lookup follows the same hash
>> => lookup bucket => search list.
>>
>> Csound had a lot of different hash implementations and differently
>> sized buckets.  I've just made a generic CS_HASH_TABLE out of existing
>> code such that the key is always char* and the value is a void*.  NULL
>> values are allowed, but NULL keys are not.  These are all using a 4099
>> bucket size, which should provide less collisions than the 256 used in
>> a number of other places.
>>
>> The code for channels is also using its own hash table.  I'll look at
>> replacing that now and removing the limitation on CS_MAX_CHANNELS.
>>
>> Thanks!
>> steven
>>
>> On Wed, Apr 24, 2013 at 3:41 PM, Andres Cabrera 
>> wrote:
>> > Yes, it would be great to remove that limitation (it was inherited from
>> > the
>> > existing chn mechanism). I think it would be important to minimize
>> > allocation during performance. How does the hash table mechanism work?
>> >
>> > Cheers,
>> > Andres
>> >
>> > On Apr 24, 2013 5:01 AM, "Steven Yi"  wrote:
>> >>
>> >> I think so.  I think this may be related to Andres' merging of the
>> >> different channel systems into one system, and that the old
>> >> chani/chano may have used a channel[256] array.  I think it's now
>> >> merged to all use the named channel interface, and so perhaps that
>> >> channel num limit may just no longer apply. That's what I'm
>> >> understanding from the code so far at least.
>> >>
>> >> On Wed, Apr 24, 2013 at 12:52 PM, Victor Lazzarini
>> >>  wrote:
>> >> > Is this to do with bus channel names?
>> >> > On 24 Apr 2013, at 12:33, Steven Yi wrote:
>> >> >
>> >> >> Hi All,
>> >> >>
>> >> >> Question: Why is there a max channels variable?  I'm looking at the
>> >> >> code and it's using a hash table that should have no limits.  It
>> >> >> looks
>> >> >> like the max channels was being used for the max bucket size of the
>> >> >> hash table, but I don't see any reason why there should be a limit.
>> >> >> I
>> >> >> think this should just be replaced with a CS_HASH_TABLE and all
>> >> >> limitations removed, but this is my first time through this code.
>> >> >> Does anyone have any perspective on this?
>> >> >>
>> >> >> Thanks!
>> >> >> steven
>> >> >>
>> >> >>
>> >> >>
>> >> >> ------------------------------------------------------------------------------
>> >> >> Try New Relic Now & We'll Send You this Cool Shirt
>> >> >> New Relic is the only SaaS-based application performance monitoring
>> >> >> service
>> >> >> that delivers powerful full stack analytics. Optimize and monitor
>> >> >> your
>> >> >> browser, app, & servers with just a few lines of code. Try New Relic
>> >> >> and get this awesome Nerd Life shirt!
>> >> >> http://p.sf.net/sfu/newrelic_d2d_apr
>> >> >> _______________________________________________
>> >> >> Csound-devel mailing list
>> >> >> Csound-devel@lists.sourceforge.net
>> >> >> https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >> >
>> >> > Dr Victor Lazzarini
>> >> > Senior Lecturer
>> >> > Dept. of Music
>> >> > NUI Maynooth Ireland
>> >> > tel.: +353 1 708 3545
>> >> > Victor dot Lazzarini AT nuim dot ie
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > ------------------------------------------------------------------------------
>> >> > Try New Relic Now & We'll Send You this Cool Shirt
>> >> > New Relic is the only SaaS-based application performance monitoring
>> >> > service
>> >> > that delivers powerful full stack analytics. Optimize and monitor
>> >> > your
>> >> > browser, app, & servers with just a few lines of code. Try New Relic
>> >> > and get this awesome Nerd Life shirt!
>> >> > http://p.sf.net/sfu/newrelic_d2d_apr
>> >> > _______________________________________________
>> >> > Csound-devel mailing list
>> >> > Csound-devel@lists.sourceforge.net
>> >> > https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Try New Relic Now & We'll Send You this Cool Shirt
>> >> New Relic is the only SaaS-based application performance monitoring
>> >> service
>> >> that delivers powerful full stack analytics. Optimize and monitor your
>> >> browser, app, & servers with just a few lines of code. Try New Relic
>> >> and get this awesome Nerd Life shirt!
>> >> http://p.sf.net/sfu/newrelic_d2d_apr
>> >> _______________________________________________
>> >> Csound-devel mailing list
>> >> Csound-devel@lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Try New Relic Now & We'll Send You this Cool Shirt
>> > New Relic is the only SaaS-based application performance monitoring
>> > service
>> > that delivers powerful full stack analytics. Optimize and monitor your
>> > browser, app, & servers with just a few lines of code. Try New Relic
>> > and get this awesome Nerd Life shirt!
>> > http://p.sf.net/sfu/newrelic_d2d_apr
>> > _______________________________________________
>> > Csound-devel mailing list
>> > Csound-devel@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/csound-devel
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Try New Relic Now & We'll Send You this Cool Shirt
>> New Relic is the only SaaS-based application performance monitoring
>> service
>> that delivers powerful full stack analytics. Optimize and monitor your
>> browser, app, & servers with just a few lines of code. Try New Relic
>> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel
>

------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net