It's probably due to historical baggage that we have both tables and 
arrays. In the case of one dimensional arrays, they seem to share the same 
underlying structure, a c-array of doubles. As there seems to be a great 
number of operations available for arrays which are not for tables, and in 
general arrays being much more ergonomic, I would like to be able to have 
an array as a "view" or an "alias" over a table, so that they share the 
underlying memory. Such arrays would not own their own memory and would not 
be resizable but would otherwise behave just like any other array.
As a prove of concept I implemented it as an i-rate opcode:

typedef struct {
    OPDS h;
    ARRAYDAT *out;
    MYFLT *ifn, *iend;
    FUNC *ftp;
} TABALIAS;

static int tabalias_init(CSOUND *csound, TABALIAS *p) {
    FUNC *ftp;
    ftp = csound->FTFind(csound, p->ifn);
    if (UNLIKELY(ftp == NULL)) {
        return NOTOK;
    }
    p->ftp = ftp;
    int end = *p->iend;
    if(end == 0)
        end = ftp->flen;
    if(p->out->data != NULL) {
        return INITERR("left hand side array already initialized");
    }
    size_t ss;
    CS_VARIABLE* var = p->out->arrayType->createVariable(csound, NULL);
    p->out->arrayMemberSize = var->memBlockSize;
    p->out->data = ftp->ftable;
    p->out->dimensions = 1;
    p->out->sizes = (int*)csound->Malloc(csound, sizeof(int));
    p->out->sizes[0] = end;
    return OK;
}

It works fine and saves me significant cpu when dealing with big arrays 
which need to be copyied back and forth on every cycle. But since the array 
is borrowing the memory from the table, it should not free it when it is 
deallocated. 

Best regards,

Hi Eduardo,

This is pretty interesting. I've been doing some synthesis where I'm
generating data into an array, then creating an ftable and copying
over values using copya2ftab.  It's only at init-time for each note so
the penalty hasn't been horrible, but it's inefficient for sure to
have two copies of the data.

Doing views raises the issue you mentioned about freeing memory, but I
wonder too about some array opcodes that might resize data. (Probably
need to review the code to see about that...)

Another possibility is to wrap the other way, and do:

inum ftwrap iarray[]
inum ftwrap karray[]

so code could look like:

iarr[] init 1025
... gen a table manually ...

asig = oscili(0.5, 440, ftwrap(iarr))
outc(asig, asig)

steven



On Tue, Aug 29, 2017 at 9:16 AM, Eduardo Moguillansky
 wrote:
> It's probably due to historical baggage that we have both tables and arrays.
> In the case of one dimensional arrays, they seem to share the same
> underlying structure, a c-array of doubles. As there seems to be a great
> number of operations available for arrays which are not for tables, and in
> general arrays being much more ergonomic, I would like to be able to have an
> array as a "view" or an "alias" over a table, so that they share the
> underlying memory. Such arrays would not own their own memory and would not
> be resizable but would otherwise behave just like any other array.
> As a prove of concept I implemented it as an i-rate opcode:
>
> typedef struct {
>    OPDS h;
>    ARRAYDAT *out;
>    MYFLT *ifn, *iend;
>    FUNC *ftp;
> } TABALIAS;
>
> static int tabalias_init(CSOUND *csound, TABALIAS *p) {
>    FUNC *ftp;
>    ftp = csound->FTFind(csound, p->ifn);
>    if (UNLIKELY(ftp == NULL)) {
>        return NOTOK;
>    }
>    p->ftp = ftp;
>    int end = *p->iend;
>    if(end == 0)
>        end = ftp->flen;
>    if(p->out->data != NULL) {
>        return INITERR("left hand side array already initialized");
>    }
>    size_t ss;
>    CS_VARIABLE* var = p->out->arrayType->createVariable(csound, NULL);
>    p->out->arrayMemberSize = var->memBlockSize;
>    p->out->data = ftp->ftable;
>    p->out->dimensions = 1;
>    p->out->sizes = (int*)csound->Malloc(csound, sizeof(int));
>    p->out->sizes[0] = end;
>    return OK;
> }
>
> It works fine and saves me significant cpu when dealing with big arrays
> which need to be copyied back and forth on every cycle. But since the array
> is borrowing the memory from the table, it should not free it when it is
> deallocated.
> Best regards,

I found a way which seems at least not to pose bigger problems without the 
need to add a field to ARRAYDAT to indicate if the array owns its memory 
(which would be the right way to do views). This solution implements a 
krate opcode, arrayview, which sets the pointer of the array to memory held 
by another object (a table or another array), and at the end of the 
instrument a corresponding arrayviewend opcode, which sets the memory to 
NULL. Within the instrument then the opcode is an alias of the table and as 
long as it is used as a right hand side argument or in operations such as 
'+=', which preserve the size, I have not encountered any problem. Code 
looks like:

gitab1 ftgen ...
gitab2 ftgen ...

instr 1
  istart = 2
  iend = 20
  kA[] arrayview gitab1, istart, iend
  kB[] arrayview gitab2 
  kC[] arrayview kB, 0, iend-istart
  kC += kA * 2
 
  arrayviewend kA, kB, kC
endin

Here kA is a view to a slice of a table. kB is a view to another table. kC 
is a view to a slice of kB, used to modify only a specific slice of it. 
Without views you would need to copy data around all the time just to be 
able to use the array operations. Both arrayview and arrayviewend are 
k-time opcodes, so that at the end of the instrument all aliases have been 
cleared and the performance can end. 


On Freitag, 1. September 2017 17:01:14 CEST, Steven Yi wrote:
> Hi Eduardo,
>
> This is pretty interesting. I've been doing some synthesis where I'm
> generating data into an array, then creating an ftable and copying
> over values using copya2ftab.  It's only at init-time for each note so
> the penalty hasn't been horrible, but it's inefficient for sure to
> have two copies of the data.
>
> Doing views raises the issue you mentioned about freeing memory, but I
> wonder too about some array opcodes that might resize data. (Probably
> need to review the code to see about that...)
>
> Another possibility is to wrap the other way, and do:
>
> inum ftwrap iarray[]
> inum ftwrap karray[]
>
> so code could look like:
>
> iarr[] init 1025
> ... gen a table manually ...
>
> asig = oscili(0.5, 440, ftwrap(iarr))
> outc(asig, asig)
>
> steven
>
>
>
> On Tue, Aug 29, 2017 at 9:16 AM, Eduardo Moguillansky
>  wrote:
>> It's probably due to historical baggage that we have both 
>> tables and arrays.
>> In the case of one dimensional arrays, they seem to share the same
>> underlying structure, a c-array of doubles. As there seems to be a great
>> number of operations available for arrays which are not for tables, and in
>> general arrays being much more ergonomic, I would like to be 
>> able to have an
>> array as a "view" or an "alias" over a table, so that they share the
>> underlying memory. Such arrays would not own their own memory 
>> and would not
>> be resizable but would otherwise behave just like any other array.
>> As a prove of concept I implemented it as an i-rate opcode:
>> 
>> typedef struct {
>>    OPDS h;
>>    ARRAYDAT *out;
>>    MYFLT *ifn, *iend;
>>    FUNC *ftp;
>> } TABALIAS;
>> 
>> static int tabalias_init(CSOUND *csound, TABALIAS *p) {
>>    FUNC *ftp;
>>    ftp = csound->FTFind(csound, p->ifn);
>>    if (UNLIKELY(ftp == NULL)) {
>>        return NOTOK;
>>    }
>>    p->ftp = ftp;
>>    int end = *p->iend;
>>    if(end == 0)
>>        end = ftp->flen;
>>    if(p->out->data != NULL) {
>>        return INITERR("left hand side array already initialized");
>>    }
>>    size_t ss;
>>    CS_VARIABLE* var = p->out->arrayType->createVariable(csound, NULL);
>>    p->out->arrayMemberSize = var->memBlockSize;
>>    p->out->data = ftp->ftable;
>>    p->out->dimensions = 1;
>>    p->out->sizes = (int*)csound->Malloc(csound, sizeof(int));
>>    p->out->sizes[0] = end;
>>    return OK;
>> }
>> 
>> It works fine and saves me significant cpu when dealing with big arrays
>> which need to be copyied back and forth on every cycle. But 
>> since the array
>> is borrowing the memory from the table, it should not free it when it is
>> deallocated.
>> Best regards,
>> Eduardo Moguillansky
>

We had done some work to allow oscillators to access arrays directly as if they were tables. It was a while ago and I can't  recall
the details, but the memory was shared.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

> On 1 Sep 2017, at 16:24, Eduardo Moguillansky  wrote:
> 
> I found a way which seems at least not to pose bigger problems without the need to add a field to ARRAYDAT to indicate if the array owns its memory (which would be the right way to do views). This solution implements a krate opcode, arrayview, which sets the pointer of the array to memory held by another object (a table or another array), and at the end of the instrument a corresponding arrayviewend opcode, which sets the memory to NULL. Within the instrument then the opcode is an alias of the table and as long as it is used as a right hand side argument or in operations such as '+=', which preserve the size, I have not encountered any problem. Code looks like:
> 
> gitab1 ftgen ...
> gitab2 ftgen ...
> 
> instr 1
> istart = 2
> iend = 20
> kA[] arrayview gitab1, istart, iend
> kB[] arrayview gitab2  kC[] arrayview kB, 0, iend-istart
> kC += kA * 2
> arrayviewend kA, kB, kC
> endin
> 
> Here kA is a view to a slice of a table. kB is a view to another table. kC is a view to a slice of kB, used to modify only a specific slice of it. Without views you would need to copy data around all the time just to be able to use the array operations. Both arrayview and arrayviewend are k-time opcodes, so that at the end of the instrument all aliases have been cleared and the performance can end. 
> 
>> On Freitag, 1. September 2017 17:01:14 CEST, Steven Yi wrote:
>> Hi Eduardo,
>> 
>> This is pretty interesting. I've been doing some synthesis where I'm
>> generating data into an array, then creating an ftable and copying
>> over values using copya2ftab.  It's only at init-time for each note so
>> the penalty hasn't been horrible, but it's inefficient for sure to
>> have two copies of the data.
>> 
>> Doing views raises the issue you mentioned about freeing memory, but I
>> wonder too about some array opcodes that might resize data. (Probably
>> need to review the code to see about that...)
>> 
>> Another possibility is to wrap the other way, and do:
>> 
>> inum ftwrap iarray[]
>> inum ftwrap karray[]
>> 
>> so code could look like:
>> 
>> iarr[] init 1025
>> ... gen a table manually ...
>> 
>> asig = oscili(0.5, 440, ftwrap(iarr))
>> outc(asig, asig)
>> 
>> steven
>> 
>> 
>> 
>> On Tue, Aug 29, 2017 at 9:16 AM, Eduardo Moguillansky
>>  wrote:
>>> It's probably due to historical baggage that we have both tables and arrays.
>>> In the case of one dimensional arrays, they seem to share the same
>>> underlying structure, a c-array of doubles. As there seems to be a great
>>> number of operations available for arrays which are not for tables, and in
>>> general arrays being much more ergonomic, I would like to be able to have an
>>> array as a "view" or an "alias" over a table, so that they share the
>>> underlying memory. Such arrays would not own their own memory and would not
>>> be resizable but would otherwise behave just like any other array.
>>> As a prove of concept I implemented it as an i-rate opcode:
>>> typedef struct {
>>>   OPDS h;
>>>   ARRAYDAT *out;
>>>   MYFLT *ifn, *iend;
>>>   FUNC *ftp;
>>> } TABALIAS;
>>> static int tabalias_init(CSOUND *csound, TABALIAS *p) {
>>>   FUNC *ftp;
>>>   ftp = csound->FTFind(csound, p->ifn);
>>>   if (UNLIKELY(ftp == NULL)) {
>>>       return NOTOK;
>>>   }
>>>   p->ftp = ftp;
>>>   int end = *p->iend;
>>>   if(end == 0)
>>>       end = ftp->flen;
>>>   if(p->out->data != NULL) {
>>>       return INITERR("left hand side array already initialized");
>>>   }
>>>   size_t ss;
>>>   CS_VARIABLE* var = p->out->arrayType->createVariable(csound, NULL);
>>>   p->out->arrayMemberSize = var->memBlockSize;
>>>   p->out->data = ftp->ftable;
>>>   p->out->dimensions = 1;
>>>   p->out->sizes = (int*)csound->Malloc(csound, sizeof(int));
>>>   p->out->sizes[0] = end;
>>>   return OK;
>>> }
>>> It works fine and saves me significant cpu when dealing with big arrays
>>> which need to be copyied back and forth on every cycle. But since the array
>>> is borrowing the memory from the table, it should not free it when it is
>>> deallocated.
>>> Best regards,
>>> Eduardo Moguillansky
>> 

I remember doing a change to the data structure to allow the array to be used as an ftable but I cannot remember the details.

Sent from Blue

On 1 Sep 2017, at 17:22, Victor Lazzarini <Victor.Lazzarini@MU.IE> wrote:

We had done some work to allow oscillators to access arrays directly as if they were tables. It was a while ago and I can't  recall
the details, but the memory was shared.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

 On 1 Sep 2017, at 16:24, Eduardo Moguillansky <eduardo.moguillansky@GMAIL.COM> wrote:
 
 I found a way which seems at least not to pose bigger problems without the need to add a field to ARRAYDAT to indicate if the array owns its memory (which would be the right way to do views). This solution implements a krate opcode, arrayview, which sets the pointer of the array to memory held by another object (a table or another array), and at the end of the instrument a corresponding arrayviewend opcode, which sets the memory to NULL. Within the instrument then the opcode is an alias of the table and as long as it is used as a right hand side argument or in operations such as '+=', which preserve the size, I have not encountered any problem. Code looks like:
 
 gitab1 ftgen ...
 gitab2 ftgen ...
 
 instr 1
 istart = 2
 iend = 20
 kA[] arrayview gitab1, istart, iend
 kB[] arrayview gitab2  kC[] arrayview kB, 0, iend-istart
 kC += kA * 2
 arrayviewend kA, kB, kC
 endin
 
 Here kA is a view to a slice of a table. kB is a view to another table. kC is a view to a slice of kB, used to modify only a specific slice of it. Without views you would need to copy data around all the time just to be able to use the array operations. Both arrayview and arrayviewend are k-time opcodes, so that at the end of the instrument all aliases have been cleared and the performance can end. 
 
 On Freitag, 1. September 2017 17:01:14 CEST, Steven Yi wrote:
 Hi Eduardo,
 
 This is pretty interesting. I've been doing some synthesis where I'm
 generating data into an array, then creating an ftable and copying
 over values using copya2ftab.  It's only at init-time for each note so
 the penalty hasn't been horrible, but it's inefficient for sure to
 have two copies of the data.
 
 Doing views raises the issue you mentioned about freeing memory, but I
 wonder too about some array opcodes that might resize data. (Probably
 need to review the code to see about that...)
 
 Another possibility is to wrap the other way, and do:
 
 inum ftwrap iarray[]
 inum ftwrap karray[]
 
 so code could look like:
 
 iarr[] init 1025
 ... gen a table manually ...
 
 asig = oscili(0.5, 440, ftwrap(iarr))
 outc(asig, asig)
 
 steven
 
 
 
 On Tue, Aug 29, 2017 at 9:16 AM, Eduardo Moguillansky
 <eduardo.moguillansky@gmail.com> wrote:
 It's probably due to historical baggage that we have both tables and arrays.
 In the case of one dimensional arrays, they seem to share the same
 underlying structure, a c-array of doubles. As there seems to be a great
 number of operations available for arrays which are not for tables, and in
 general arrays being much more ergonomic, I would like to be able to have an
 array as a "view" or an "alias" over a table, so that they share the
 underlying memory. Such arrays would not own their own memory and would not
 be resizable but would otherwise behave just like any other array.
 As a prove of concept I implemented it as an i-rate opcode:
 typedef struct {
   OPDS h;
   ARRAYDAT *out;
   MYFLT *ifn, *iend;
   FUNC *ftp;
 } TABALIAS;
 static int tabalias_init(CSOUND *csound, TABALIAS *p) {
   FUNC *ftp;
   ftp = csound->FTFind(csound, p->ifn);
   if (UNLIKELY(ftp == NULL)) {
       return NOTOK;
   }
   p->ftp = ftp;
   int end = *p->iend;
   if(end == 0)
       end = ftp->flen;
   if(p->out->data != NULL) {
       return INITERR("left hand side array already initialized");
   }
   size_t ss;
   CS_VARIABLE* var = p->out->arrayType->createVariable(csound, NULL);
   p->out->arrayMemberSize = var->memBlockSize;
   p->out->data = ftp->ftable;
   p->out->dimensions = 1;
   p->out->sizes = (int*)csound->Malloc(csound, sizeof(int));
   p->out->sizes[0] = end;
   return OK;
 }
 It works fine and saves me significant cpu when dealing with big arrays
 which need to be copyied back and forth on every cycle. But since the array
 is borrowing the memory from the table, it should not free it when it is
 deallocated.
 Best regards,
 Eduardo Moguillansky
 
 

I just used csound -z1 and saw the overloaded oscillators. I adjusted
my code to use the array tables directly and it seems to work fine.
One oddity is that it requires the arrays to be pow-of-two, while I
had been generating pow-of-two-plus-one.  However, seems to work out
fine with pow-of two.  Solves my issue here at least.



On Fri, Sep 1, 2017 at 12:21 PM, Victor Lazzarini
 wrote:
> We had done some work to allow oscillators to access arrays directly as if they were tables. It was a while ago and I can't  recall
> the details, but the memory was shared.
>
> Victor Lazzarini
> Dean of Arts, Celtic Studies, and Philosophy
> Maynooth University
> Ireland
>
>> On 1 Sep 2017, at 16:24, Eduardo Moguillansky  wrote:
>>
>> I found a way which seems at least not to pose bigger problems without the need to add a field to ARRAYDAT to indicate if the array owns its memory (which would be the right way to do views). This solution implements a krate opcode, arrayview, which sets the pointer of the array to memory held by another object (a table or another array), and at the end of the instrument a corresponding arrayviewend opcode, which sets the memory to NULL. Within the instrument then the opcode is an alias of the table and as long as it is used as a right hand side argument or in operations such as '+=', which preserve the size, I have not encountered any problem. Code looks like:
>>
>> gitab1 ftgen ...
>> gitab2 ftgen ...
>>
>> instr 1
>> istart = 2
>> iend = 20
>> kA[] arrayview gitab1, istart, iend
>> kB[] arrayview gitab2  kC[] arrayview kB, 0, iend-istart
>> kC += kA * 2
>> arrayviewend kA, kB, kC
>> endin
>>
>> Here kA is a view to a slice of a table. kB is a view to another table. kC is a view to a slice of kB, used to modify only a specific slice of it. Without views you would need to copy data around all the time just to be able to use the array operations. Both arrayview and arrayviewend are k-time opcodes, so that at the end of the instrument all aliases have been cleared and the performance can end.
>>
>>> On Freitag, 1. September 2017 17:01:14 CEST, Steven Yi wrote:
>>> Hi Eduardo,
>>>
>>> This is pretty interesting. I've been doing some synthesis where I'm
>>> generating data into an array, then creating an ftable and copying
>>> over values using copya2ftab.  It's only at init-time for each note so
>>> the penalty hasn't been horrible, but it's inefficient for sure to
>>> have two copies of the data.
>>>
>>> Doing views raises the issue you mentioned about freeing memory, but I
>>> wonder too about some array opcodes that might resize data. (Probably
>>> need to review the code to see about that...)
>>>
>>> Another possibility is to wrap the other way, and do:
>>>
>>> inum ftwrap iarray[]
>>> inum ftwrap karray[]
>>>
>>> so code could look like:
>>>
>>> iarr[] init 1025
>>> ... gen a table manually ...
>>>
>>> asig = oscili(0.5, 440, ftwrap(iarr))
>>> outc(asig, asig)
>>>
>>> steven
>>>
>>>
>>>
>>> On Tue, Aug 29, 2017 at 9:16 AM, Eduardo Moguillansky
>>>  wrote:
>>>> It's probably due to historical baggage that we have both tables and arrays.
>>>> In the case of one dimensional arrays, they seem to share the same
>>>> underlying structure, a c-array of doubles. As there seems to be a great
>>>> number of operations available for arrays which are not for tables, and in
>>>> general arrays being much more ergonomic, I would like to be able to have an
>>>> array as a "view" or an "alias" over a table, so that they share the
>>>> underlying memory. Such arrays would not own their own memory and would not
>>>> be resizable but would otherwise behave just like any other array.
>>>> As a prove of concept I implemented it as an i-rate opcode:
>>>> typedef struct {
>>>>   OPDS h;
>>>>   ARRAYDAT *out;
>>>>   MYFLT *ifn, *iend;
>>>>   FUNC *ftp;
>>>> } TABALIAS;
>>>> static int tabalias_init(CSOUND *csound, TABALIAS *p) {
>>>>   FUNC *ftp;
>>>>   ftp = csound->FTFind(csound, p->ifn);
>>>>   if (UNLIKELY(ftp == NULL)) {
>>>>       return NOTOK;
>>>>   }
>>>>   p->ftp = ftp;
>>>>   int end = *p->iend;
>>>>   if(end == 0)
>>>>       end = ftp->flen;
>>>>   if(p->out->data != NULL) {
>>>>       return INITERR("left hand side array already initialized");
>>>>   }
>>>>   size_t ss;
>>>>   CS_VARIABLE* var = p->out->arrayType->createVariable(csound, NULL);
>>>>   p->out->arrayMemberSize = var->memBlockSize;
>>>>   p->out->data = ftp->ftable;
>>>>   p->out->dimensions = 1;
>>>>   p->out->sizes = (int*)csound->Malloc(csound, sizeof(int));
>>>>   p->out->sizes[0] = end;
>>>>   return OK;
>>>> }
>>>> It works fine and saves me significant cpu when dealing with big arrays
>>>> which need to be copyied back and forth on every cycle. But since the array
>>>> is borrowing the memory from the table, it should not free it when it is
>>>> deallocated.
>>>> Best regards,
>>>> Eduardo Moguillansky
>>>