[Csnd] use tables efficiently

[Csnd] use tables efficiently

Date	2011-12-28 16:44
From	zappfinger
Subject	[Csnd] use tables efficiently
	In my midirecording CSD I use tables to write midi events at k-rate. The length of the table is initialised for the duration of the recording. But I would like to only write to the table when there is an event. No need to write so many 0's . So I guess i need to create another table for the index. If there are only 3 midi events during a 60 second recording, the index table would contain the indexes for the 3 events and the data tables would only contain these midi events. Is this the way to do it or are there other suggestions? Richard -- View this message in context: http://csound.1045644.n5.nabble.com/use-tables-efficiently-tp5105629p5105629.html Sent from the Csound - General mailing list archive at Nabble.com. Send bugs reports to the Sourceforge bug tracker https://sourceforge.net/tracker/?group_id=81968&atid=564599 Discussions of bugs and features can be posted here To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Date	2011-12-28 18:36
From	Iain Duncan
Subject	Re: [Csnd] use tables efficiently
	On Wed, Dec 28, 2011 at 8:44 AM, zappfinger <zappfinger@gmail.com> wrote: In my midirecording CSD I use tables to write midi events at k-rate. The length of the table is initialised for the duration of the recording. But I would like to only write to the table when there is an event. No need to write so many 0's . So I guess i need to create another table for the index. If there are only 3 midi events during a 60 second recording, the index table would contain the indexes for the 3 events and the data tables would only contain these midi events. Is this the way to do it or are there other suggestions? Hi Richard, this is a tricker issue than it seems, unless you know that you will always play back in order. If you need your player to have the ability to jump around in time, in real time, (ie any kind of seamless looping ) then any kind of sparse data structure introduces trade offs of ram size against processing time needed to find event times. If you know you'll always play back from the top, you could use a table of events as opposed to table representing time, but then ask yourself before committing, how will you play starting from say bar 4? How does the player know where to seek to for bar 4? Does it iterate through all events in that table seeking the first event scheduled for bar 4 and then begin playing, and is that iteration process going to be fast enough for you? I may be misunderstanding your question, but I would start with checking whether tables with lots of zeros wlll do the job for now within acceptable ram rates as it's definitely the easiest to code and the fastest way for seeking instantly to a given time ( I'm assuming a table index represents a bit of time here). I'm dealing with the same question right now in my csound hosting jack app, so feel free to explain further if I don't get what you're asking. You'll also need to think about what is acceptable latency for recording and playback. In my case, I can live with higher latency for recording, as long as the time stamp is accurate, so I intend incoming events to get timestamped by my audio thread but do the actual data structure manipulation in my non real time thread. It makes no difference to me if they get inserted a few ms later than played as long as we heard them ok, they only need to be in there properly by the next loop pass. So given that, I'm choosing data structures that do the house work on recording and are optimized for fast lookup on playback. FWIW, the solution I am leaning to right now, but haven't benchmarked yet, is to have a lower resolution regular array/table, with slots indicating some division corresponding to the finest I expect people to want to look at ( ie 16 triplet for me, may be 64th or something for classical music ). If events are in that bin ( the dur of the 16th ), they will go in a linked data structure with the first node in that time line slot. I think I can reasonably say to my users that there will be a maximum of say 64 notes across all tracks in that one bin. ( Like saying 64 note polyphony if notes were quantized to the 16th). The engine will be able hop to that bin instantly using simple array lookup, which is dead fast. At that point, it has to traverse a linked list, but we know that traversal can take a max of 64 event-traversal time. The events can store their actual non quantized time, and the engine can deal with that. This way the engine: a) can hop anywhere instantly, zero penalty for non-linear access b) on our smallest time division ( whatever the tick rate is, say 32 samples, giving us a time res of 0.7 ms ) the engine has to iterate through only events within that sixteenths notes, and has 32 samples to do so. c) we have a known worst case scenario, which is very important for real time dependability I'm not sure how one would translate that to pure csound, but thought I'd share my explorations in case it's useful. Iain

Date	2011-12-28 20:25
From	Justin Smith
Subject	Re: [Csnd] use tables efficiently
Attachments	auto.csd
	Since I have a use for a sparse automation recorder like this, I coded one up just now as a UDO. I am thinking of another UDO that creates a full recording (which you could insert new values into easily etc.) from an existing sparse recording. These UDOs use two tables, one for signal values and another for time offsets. ----- Original message ----- > In my midirecording CSD I use tables to write midi events at k-rate. > The length of the table is initialised for the duration of the recording. > But I would like to only write to the table when there is an event. No > need to write so many 0's . > So I guess i need to create another table for the index. > If there are only 3 midi events during a 60 second recording, the index > table would contain the indexes for the 3 events and the data tables > would only contain these midi events. > > Is this the way to do it or are there other suggestions? > > Richard > > -- > View this message in context: > http://csound.1045644.n5.nabble.com/use-tables-efficiently-tp5105629p5105629.html > Sent from the Csound - General mailing list archive at Nabble.com. > > > Send bugs reports to the Sourceforge bug tracker > https://sourceforge.net/tracker/?group_id=81968&atid=564599 > Discussions of bugs and features can be posted here > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe > csound" >

Date	2011-12-28 22:27
From	zappfinger
Subject	[Csnd] Re: use tables efficiently
	Ian, I am also considering using flat files to store the events in normal score style format. My simple 'poor-mans' midirecorder works with small phrases, that can be combined or 'multiplied' to form longer sequences. I'd be interested to know what you are working on. Is it for midi only or also audio? Richard -- View this message in context: http://csound.1045644.n5.nabble.com/use-tables-efficiently-tp5105629p5106226.html Sent from the Csound - General mailing list archive at Nabble.com. Send bugs reports to the Sourceforge bug tracker https://sourceforge.net/tracker/?group_id=81968&atid=564599 Discussions of bugs and features can be posted here To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Date	2011-12-28 22:29
From	zappfinger
Subject	[Csnd] Re: use tables efficiently
	Justin, Interesting UDO's, they contain some new opcodes for me... Richard -- View this message in context: http://csound.1045644.n5.nabble.com/use-tables-efficiently-tp5105629p5106235.html Sent from the Csound - General mailing list archive at Nabble.com. Send bugs reports to the Sourceforge bug tracker https://sourceforge.net/tracker/?group_id=81968&atid=564599 Discussions of bugs and features can be posted here To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Date	2011-12-29 10:10
From	Alex Weiss
Subject	Re: [Csnd] use tables efficiently
	Out of curiosity: since you only record new values, how do you know how big a table to allocate? On Wed, Dec 28, 2011 at 9:25 PM, Justin Smith wrote: > Since I have a use for a sparse automation recorder like this, I coded one > up just now as a UDO. I am thinking of another UDO that creates a full > recording (which you could insert new values into easily etc.) from an > existing sparse recording. These UDOs use two tables, one for signal values > and another for time offsets. > > > ----- Original message ----- >> In my midirecording CSD I use tables to write midi events at k-rate. >> The length of the table is initialised for the duration of the recording. >> But I would like to only write to the table when there is an event. No >> need to write so many 0's . >> So I guess i need to create another table for the index. >> If there are only 3 midi events during a 60 second recording, the index >> table would contain the indexes for the 3 events and the data tables >> would only contain these midi events. >> >> Is this the way to do it or are there other suggestions? >> >> Richard >> >> -- >> View this message in context: >> >> http://csound.1045644.n5.nabble.com/use-tables-efficiently-tp5105629p5105629.html >> Sent from the Csound - General mailing list archive at Nabble.com. >> >> >> Send bugs reports to the Sourceforge bug tracker >> >> https://sourceforge.net/tracker/?group_id=81968&atid=564599 >> Discussions of bugs and features can be posted here >> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe >> csound" >> > Send bugs reports to the Sourceforge bug tracker https://sourceforge.net/tracker/?group_id=81968&atid=564599 Discussions of bugs and features can be posted here To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Date	2011-12-29 17:25
From	Justin Smith
Subject	Re: [Csnd] use tables efficiently
	One probably can't know for sure, but depending on how the values are being generated you could make an estimation. ----- Original message ----- > Out of curiosity: since you only record new values, how do you know > how big a table to allocate? > > On Wed, Dec 28, 2011 at 9:25 PM, Justin Smith <noisesmith@gmail.com> > wrote: > > Since I have a use for a sparse automation recorder like this, I coded > > one up just now as a UDO. I am thinking of another UDO that creates a > > full recording (which you could insert new values into easily etc.) > > from an existing sparse recording. These UDOs use two tables, one for > > signal values and another for time offsets. > > > > > > ----- Original message ----- > > > In my midirecording CSD I use tables to write midi events at k-rate. > > > The length of the table is initialised for the duration of the > > > recording. But I would like to only write to the table when there is > > > an event. No need to write so many 0's . > > > So I guess i need to create another table for the index. > > > If there are only 3 midi events during a 60 second recording, the > > > index table would contain the indexes for the 3 events and the data > > > tables would only contain these midi events. > > > > > > Is this the way to do it or are there other suggestions? > > > > > > Richard > > > > > > -- > > > View this message in context: > > > > > > http://csound.1045644.n5.nabble.com/use-tables-efficiently-tp5105629p5105629.html > > > Sent from the Csound - General mailing list archive at Nabble.com. > > > > > > > > > Send bugs reports to the Sourceforge bug tracker > > > > > > https://sourceforge.net/tracker/?group_id=81968&atid=564599 > > > Discussions of bugs and features can be posted here > > > To unsubscribe, send email sympa@lists.bath.ac.uk with body > > > "unsubscribe csound" > > > > > > > > Send bugs reports to the Sourceforge bug tracker > https://sourceforge.net/tracker/?group_id=81968&atid=564599 > Discussions of bugs and features can be posted here > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe > csound" >

Date	2011-12-29 17:48
From	Alex Weiss
Subject	Re: [Csnd] use tables efficiently
	In any case, thanks for the .csd. It's great food for thought, as I'm trying to implement something very similar. The only possible problem I see is erratic behavior when the kr used for playback is much lower than the kr used for recording. But then again, that's seldom the case -- I'd assume usually it's the other way around. I'm also curious to hear what you mean by a full recorder and how it differs from your sparse recorder. Alex On Thu, Dec 29, 2011 at 6:25 PM, Justin Smith wrote: > One probably can't know for sure, but depending on how the values are being > generated you could make an estimation. > > > ----- Original message ----- >> Out of curiosity: since you only record new values, how do you know >> how big a table to allocate? >> >> On Wed, Dec 28, 2011 at 9:25 PM, Justin Smith >> wrote: >> > Since I have a use for a sparse automation recorder like this, I coded >> > one up just now as a UDO. I am thinking of another UDO that creates a >> > full recording (which you could insert new values into easily etc.) >> > from an existing sparse recording. These UDOs use two tables, one for >> > signal values and another for time offsets. >> > >> > >> > ----- Original message ----- >> > > In my midirecording CSD I use tables to write midi events at k-rate. >> > > The length of the table is initialised for the duration of the >> > > recording. But I would like to only write to the table when there is >> > > an event. No need to write so many 0's . >> > > So I guess i need to create another table for the index. >> > > If there are only 3 midi events during a 60 second recording, the >> > > index table would contain the indexes for the 3 events and the data >> > > tables would only contain these midi events. >> > > >> > > Is this the way to do it or are there other suggestions? >> > > >> > > Richard >> > > >> > > -- >> > > View this message in context: >> > > >> > > >> > > http://csound.1045644.n5.nabble.com/use-tables-efficiently-tp5105629p5105629.html >> > > Sent from the Csound - General mailing list archive at Nabble.com. >> > > >> > > >> > > Send bugs reports to the Sourceforge bug tracker >> > > >> > > https://sourceforge.net/tracker/?group_id=81968&atid=564599 >> > > Discussions of bugs and features can be posted here >> > > To unsubscribe, send email sympa@lists.bath.ac.uk with body >> > > "unsubscribe csound" >> > > >> > >> >> >> Send bugs reports to the Sourceforge bug tracker >> >> https://sourceforge.net/tracker/?group_id=81968&atid=564599 >> Discussions of bugs and features can be posted here >> To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe >> csound" >> > Send bugs reports to the Sourceforge bug tracker https://sourceforge.net/tracker/?group_id=81968&atid=564599 Discussions of bugs and features can be posted here To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound"

Date	2011-12-29 20:50
From	Justin Smith
Subject	Re: [Csnd] use tables efficiently
	The radically changed kr issue would be a problem, yeah. It could be mitigated by looping through read/increment until the time of the next event is greater than the current time. By a "full recorder" I meant one that records a new value for every k cycle, (needing a table of at least kr*duration size). ----- Original message ----- > In any case, thanks for the .csd. It's great food for thought, as I'm > trying to implement something very similar. The only possible problem > I see is erratic behavior when the kr used for playback is much lower > than the kr used for recording. But then again, that's seldom the case > -- I'd assume usually it's the other way around. > I'm also curious to hear what you mean by a full recorder and how it > differs from your sparse recorder. > > Alex > > On Thu, Dec 29, 2011 at 6:25 PM, Justin Smith <noisesmith@gmail.com> > wrote: > > One probably can't know for sure, but depending on how the values are > > being generated you could make an estimation. > > > > > > ----- Original message ----- > > > Out of curiosity: since you only record new values, how do you know > > > how big a table to allocate? > > > > > > On Wed, Dec 28, 2011 at 9:25 PM, Justin Smith <noisesmith@gmail.com> > > > wrote: > > > > Since I have a use for a sparse automation recorder like this, I > > > > coded one up just now as a UDO. I am thinking of another UDO that > > > > creates a full recording (which you could insert new values into > > > > easily etc.) from an existing sparse recording. These UDOs use two > > > > tables, one for signal values and another for time offsets. > > > > > > > > > > > > ----- Original message ----- > > > > > In my midirecording CSD I use tables to write midi events at > > > > > k-rate. The length of the table is initialised for the duration > > > > > of the recording. But I would like to only write to the table > > > > > when there is an event. No need to write so many 0's . > > > > > So I guess i need to create another table for the index. > > > > > If there are only 3 midi events during a 60 second recording, the > > > > > index table would contain the indexes for the 3 events and the > > > > > data tables would only contain these midi events. > > > > > > > > > > Is this the way to do it or are there other suggestions? > > > > > > > > > > Richard > > > > > > > > > > -- > > > > > View this message in context: > > > > > > > > > > > > > > > http://csound.1045644.n5.nabble.com/use-tables-efficiently-tp5105629p5105629.html > > > > > Sent from the Csound - General mailing list archive at > > > > > Nabble.com. > > > > > > > > > > > > > > > Send bugs reports to the Sourceforge bug tracker > > > > > > > > > > https://sourceforge.net/tracker/?group_id=81968&atid=564599 > > > > > Discussions of bugs and features can be posted here > > > > > To unsubscribe, send email sympa@lists.bath.ac.uk with body > > > > > "unsubscribe csound" > > > > > > > > > > > > > > > > > > Send bugs reports to the Sourceforge bug tracker > > > > > > https://sourceforge.net/tracker/?group_id=81968&atid=564599 > > > Discussions of bugs and features can be posted here > > > To unsubscribe, send email sympa@lists.bath.ac.uk with body > > > "unsubscribe csound" > > > > > > > > Send bugs reports to the Sourceforge bug tracker > https://sourceforge.net/tracker/?group_id=81968&atid=564599 > Discussions of bugs and features can be posted here > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe > csound" >

Date	2012-01-04 17:06
From	Iain Duncan
Subject	Re: [Csnd] Re: use tables efficiently
	On Wed, Dec 28, 2011 at 2:27 PM, zappfinger <zappfinger@gmail.com> wrote: Ian, I am also considering using flat files to store the events in normal score style format. My simple 'poor-mans' midirecorder works with small phrases, that can be combined or 'multiplied' to form longer sequences. I'd be interested to know what you are working on. Is it for midi only or also audio? Hi Richard, I'm redoing a live sequencing and sythesis rig I used years ago, that was done entirely in csound, but is now a jack app that handles all the sequencing and user input in C++, and only calls csound over the Csound API for rendering audio ( audio output is also handled by the host app ). This way csound becomes mostly a dsp engine, though I still use the score system for triggering events. The app is a real time live step sequencer that uses floats as the resolution, and stores notes as things that look like csound note events ( 16 params of system floats ) My ultimate intention is that the output and input layers will become plugin based so that note output can be handled by Csound, SuperCollider, Max/PD, the STK, and midi, and users can write new input modes as Python or QT plugins. That will probably take a long while. At the moment all i/o is either handled by jack or QT, and midi input passes through a python layer allowing one to easily hack up new input modes. It's all very pre-alpha at the moment, but I do intend to publish parts of it as tutorials and a 'roll-your-own' live sequencer kit down the road. I guess the elevator speech would be that it's supposed to be a the Vim/Emacs of live sequencers. Ugly, not easy to use, and super duper fast and powerful for users who have practiced and put some time into customizing it, the intention being that it should be used as a performance tool for improvising electronic music. When there's something to look at, I will certainly post to the list. Iain