Hello,
I am writing opcodes that work basically like pvread and pvadd but use ATS 
data files:
http://ccrma-www.stanford.edu/~juan/ATS.html
http://sourceforge.net/projects/atsa/
Anyways,
I have my opcodes working but I've come to realize that I open/read in the 
data file for each note.
I'm wondering, what's a good way to deal with (large) data files in 
Csound?
I know that pvoc and pvread etc use Memfiles and ldmemfil.  Does that 
actually keep the data in memory from note to note?
Another person in my research group suggested that I might try a Global 
opcode that actually loads the data and then just have my other opcodes 
read that data, but even that seems like it could cause 
problems/confusion.

It would be nice if there was some pre-initialization for an opcode that 
was only done 1 time (maybe for the first note) where I could load the 
data, then a destructor that is called when csound exits that could be 
used to deallocate.  Is this sort of functionality going to be in csound5?

Any suggestions?
Thanks,
Alex Norman

Yes, you need to use the ldmemfiles system. Look at other examples to 
see the procedure. I use it in the pvs opcodes, for example, to load 
analysis files (look at pstream.c: I have a custom load function using 
the ldmemfile mechanism). It deals with the first-time init issue easily 
- if a pointer is null, load the file, else don't load it - it's already 
there.
  Global data is definitely a bad idea.

If you use mmalloc to allocate memory, deallocation is taken care of by 
Csound (using the mfree stuff - see memfiles.c). The has been discussion 
of a different system for Csound5 (I have speculated on a preload system 
requiring statements at global level in the orch header, but not got as 
far as plotting any code), but I am not sure how far this has gone yet.

There is a dealloc mechanism for instruments, but it is far from being 
in universal use yet - a fairly recent addition IIRC. Others on this 
list will be able to describe it better.

You may in any case want to create any new opcodes as loadable opcode 
libraries, as that is the direction everthing is going - I expect to 
move all the pvs opcodes into plugin libraries ere long, for example.


With respect to the ATS libraries, we (well I anyway) could do with 
utilities to create the data files, without having to go the whole lisp 
route. What options are there for that? Has anyone ported the whole ATS 
package into good ol' C?


Richard Dobson



Alex Norman wrote:
> Hello,
> I am writing opcodes that work basically like pvread and pvadd but use ATS 
> data files:
> http://ccrma-www.stanford.edu/~juan/ATS.html
> http://sourceforge.net/projects/atsa/
> Anyways,
> I have my opcodes working but I've come to realize that I open/read in the 
> data file for each note.
> I'm wondering, what's a good way to deal with (large) data files in 
> Csound?
> I know that pvoc and pvread etc use Memfiles and ldmemfil.  Does that 
> actually keep the data in memory from note to note?
> Another person in my research group suggested that I might try a Global 
> opcode that actually loads the data and then just have my other opcodes 
> read that data, but even that seems like it could cause 
> problems/confusion.
> 
> It would be nice if there was some pre-initialization for an opcode that 
> was only done 1 time (maybe for the first note) where I could load the 
> data, then a destructor that is called when csound exits that could be 
> used to deallocate.  Is this sort of functionality going to be in csound5?
> 
> Any suggestions?
> Thanks,
> Alex Norman
> 
> 

>>>>> "Alex" == Alex Norman  writes:

 Alex> I'm wondering, what's a good way to deal with (large) data files in 
 Alex> Csound?
 Alex> I know that pvoc and pvread etc use Memfiles and ldmemfil.  Does that 
 Alex> actually keep the data in memory from note to note?

yes.  ldmemfil is supposed to read and remember, and later instances
can use the same data.  There has at some time been a bug on Mac so
this did not work, but it was a bug.

 Alex> It would be nice if there was some pre-initialization for an opcode that 
 Alex> was only done 1 time (maybe for the first note) where I could load the 
 Alex> data, then a destructor that is called when csound exits that could be 
 Alex> used to deallocate.  Is this sort of functionality going to be in csound5?

The prealloc form could be extended I guess.  Not sure how useful this
is, again a zero length call.

==John ffitch

Yes, soon (within in the next month or so) we will be "officially" 
releasing 
a C version of ats, there is a commandline version for creating files 
(atsa nogui) and a gtk based gui version (atsh) that allows for analysis 
and visualization as well as synthesis.  Maybe the command line version 
would even be something to include in future versions of csound, like the 
pvanal utility.
This is the webpage:
sourceforge.net/projects/atsa

I will have these ats-csound opcodes ready for the official ats release.

-Alex Norman

> 
> With respect to the ATS libraries, we (well I anyway) could do with 
> utilities to create the data files, without having to go the whole lisp 
> route. What options are there for that? Has anyone ported the whole ATS 
> package into good ol' C?
> 
> 
> Richard Dobson

OK, one last question before I re-write the opcodes to use ldmemfile.
It looks like the aux channel may persist across notes.  Is this true?  So 
if I don't want to use ldmemfile could I put all my data in the auxchannel 
and just test to make sure that 1: the aux channel is allocated and 2: 
that the aux channel contains the correct data.
My issue with using ldmemfile is that I read from various data 
file formats, if I use some memory allocation that persists across notes I 
could copy the data into an array of structures that would be the same no matter what the 
data file type, and also access the data as an array of structures instead 
of having to do more complex pointer arithmetic.

Thanks,
Alex Norman

On Sat, 10 Jan 2004 jpff@codemist.co.uk wrote:

> >>>>> "Alex" == Alex Norman  writes:
> 
>  Alex> I'm wondering, what's a good way to deal with (large) data files in 
>  Alex> Csound?
>  Alex> I know that pvoc and pvread etc use Memfiles and ldmemfil.  Does that 
>  Alex> actually keep the data in memory from note to note?
> 
> yes.  ldmemfil is supposed to read and remember, and later instances
> can use the same data.  There has at some time been a bug on Mac so
> this did not work, but it was a bug.
> 
>  Alex> It would be nice if there was some pre-initialization for an opcode that 
>  Alex> was only done 1 time (maybe for the first note) where I could load the 
>  Alex> data, then a destructor that is called when csound exits that could be 
>  Alex> used to deallocate.  Is this sort of functionality going to be in csound5?
> 
> The prealloc form could be extended I guess.  Not sure how useful this
> is, again a zero length call.
> 
> ==John ffitch
>