This short note considers what needs to be done to incorporate a new
opcode safely into the parallel execution environment.

Background:
==========

The basic question is why can one not run two instruments in
parallel.  It only takes a few moments' thought to realise that the
issue is if one of the instruments changes something on which the
other relies.  In the simple cases this is global variables that
communicate between instruments.  With that in mind, and following
the PhD of Jed Marti (Utah, 1980), we created an analysis program that
tracked globals in an instruments, and arranged multiple dispatch
based on the directed graph of dependencies -- taking account of
reading and writing to globals separately.  This has been described
in ICMC 2009 and LAC 2011.  There remain performance issues that need
the code to be revised, but it is basically semantically correct,
taking notice of instrument numeric order.

Developments:
============

An objection to what is written above is that there are more ways that
an instrument may influence another, at least in Csound5.  These need
to be identified and a scheme developed to handle them.

Three such influences have been found, writing to tables, using
channels, zap buss, and using the stack opcodes.

The stack codes are easiest; push and pop both require unique access
for the instrument to the stack.  We can thus treat this as a hidden
global variable, as long as the parser knows which opcodes require
this.

If we consider tables we are immediately into a commoner and wider
area or use.  Many opcodes read table values, but only a few create or
modify tables.  Ideally we should track each table separately, but use
of variables or expressions makes this impractical.  What is easy is
to lump all tables together, and track reading and writing to ANY
table.  Not the most efficient but table modification is not that
common.  We can thus treat table access as reading/writing to another
global variable. 

Channels and zak are then treatable like tables, with a different
global each.

This is what Csound5 does now.  

Implementation:
==============

This requires that the parser knows which opcodes use tables, and this
information is not clear in the system, short of reading the manual.
What we need is a mechanism to signal to the parser that a particular
opcode uses a table.

So, the under-utilised field "thread" in OENTRY is pressed into
service.  In earlier Csound this indicated whether the opcode was used
at i-, k-, or a- rate, and also is used in some MIDI activity, but
only a few bits are necessary for this in a 16-bit field.  So we can
use the other bits.

The file H/interlocks.h contains definitions for these bits

ZR  ZAK reading
ZW  ZAK writing
ZB  ZAK read and write

TR  tables reading
TW  tables writing
TB  tables read and write

CR  channel reading
CW  channel writing
CB  channel read and write

SB  stack used

so in Engine/entries.c we see (for example)

{ "zkr",    S(ZKR),     ZR|3,   "k",  "k",    (SUBR)zkset,   (SUBR)zkr,   NULL},
{ "ziw",    S(ZKW),     ZW|1,   "",   "ii",   (SUBR)ziw,     NULL,  NULL},
{ "tablera", S(TABLERA),TR|5,   "a",  "kkk",
                                      (SUBR)tableraset, NULL, (SUBR)tablera},
{ "tablewa", S(TABLEWA),TW|5,   "k",  "kak",
                                      (SUBR)tablewaset, NULL, (SUBR)tablewa},
{ "chnget",      0xFFFF,    CR   },


Implications:
============

The implications of this are that anyone writing a new opcode needs to
add these flags to the OENTRY to ensure parallel semantics.  If you
have sone other secret persistent data you can use one of theses
classes.

Footnote:
========

The final bit in the 16 (_QQ) is currently used to flag deprecated
opcodes, but that is not currently inspected; it will be



==John ffitch

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Thanks Johh, for this. Could we find somewhere to put this so it is seen more widely?
On 29 Jul 2012, at 20:07, john ffitch wrote:

> This short note considers what needs to be done to incorporate a new
> opcode safely into the parallel execution environment.
> 
> Background:
> ==========
> 
> The basic question is why can one not run two instruments in
> parallel.  It only takes a few moments' thought to realise that the
> issue is if one of the instruments changes something on which the
> other relies.  In the simple cases this is global variables that
> communicate between instruments.  With that in mind, and following
> the PhD of Jed Marti (Utah, 1980), we created an analysis program that
> tracked globals in an instruments, and arranged multiple dispatch
> based on the directed graph of dependencies -- taking account of
> reading and writing to globals separately.  This has been described
> in ICMC 2009 and LAC 2011.  There remain performance issues that need
> the code to be revised, but it is basically semantically correct,
> taking notice of instrument numeric order.
> 
> Developments:
> ============
> 
> An objection to what is written above is that there are more ways that
> an instrument may influence another, at least in Csound5.  These need
> to be identified and a scheme developed to handle them.
> 
> Three such influences have been found, writing to tables, using
> channels, zap buss, and using the stack opcodes.
> 
> The stack codes are easiest; push and pop both require unique access
> for the instrument to the stack.  We can thus treat this as a hidden
> global variable, as long as the parser knows which opcodes require
> this.
> 
> If we consider tables we are immediately into a commoner and wider
> area or use.  Many opcodes read table values, but only a few create or
> modify tables.  Ideally we should track each table separately, but use
> of variables or expressions makes this impractical.  What is easy is
> to lump all tables together, and track reading and writing to ANY
> table.  Not the most efficient but table modification is not that
> common.  We can thus treat table access as reading/writing to another
> global variable. 
> 
> Channels and zak are then treatable like tables, with a different
> global each.
> 
> This is what Csound5 does now.  
> 
> Implementation:
> ==============
> 
> This requires that the parser knows which opcodes use tables, and this
> information is not clear in the system, short of reading the manual.
> What we need is a mechanism to signal to the parser that a particular
> opcode uses a table.
> 
> So, the under-utilised field "thread" in OENTRY is pressed into
> service.  In earlier Csound this indicated whether the opcode was used
> at i-, k-, or a- rate, and also is used in some MIDI activity, but
> only a few bits are necessary for this in a 16-bit field.  So we can
> use the other bits.
> 
> The file H/interlocks.h contains definitions for these bits
> 
> ZR  ZAK reading
> ZW  ZAK writing
> ZB  ZAK read and write
> 
> TR  tables reading
> TW  tables writing
> TB  tables read and write
> 
> CR  channel reading
> CW  channel writing
> CB  channel read and write
> 
> SB  stack used
> 
> so in Engine/entries.c we see (for example)
> 
> { "zkr",    S(ZKR),     ZR|3,   "k",  "k",    (SUBR)zkset,   (SUBR)zkr,   NULL},
> { "ziw",    S(ZKW),     ZW|1,   "",   "ii",   (SUBR)ziw,     NULL,  NULL},
> { "tablera", S(TABLERA),TR|5,   "a",  "kkk",
>                                      (SUBR)tableraset, NULL, (SUBR)tablera},
> { "tablewa", S(TABLEWA),TW|5,   "k",  "kak",
>                                      (SUBR)tablewaset, NULL, (SUBR)tablewa},
> { "chnget",      0xFFFF,    CR   },
> 
> 
> Implications:
> ============
> 
> The implications of this are that anyone writing a new opcode needs to
> add these flags to the OENTRY to ensure parallel semantics.  If you
> have sone other secret persistent data you can use one of theses
> classes.
> 
> Footnote:
> ========
> 
> The final bit in the 16 (_QQ) is currently used to flag deprecated
> opcodes, but that is not currently inspected; it will be
> 
> 
> 
> ==John ffitch
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel

Dr Victor Lazzarini
Senior Lecturer
Dept. of Music
NUI Maynooth Ireland
tel.: +353 1 708 3545
Victor dot Lazzarini AT nuim dot ie




------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Thanks John, it is much more clear now.
There is one thing I still don't understand fully, I think, why do we
use different bits for zak and chh access? Are these two using global
signals differently?
And for example the MixerSend/Receive opcodes, do they naturally fall
in with one or the other?
I can imagine that using different "paralellism locks" for different
kinds of global signals may give more options for parallell execution
in some cases (?),
or perhaps I misunderstood the reason for using separate bits for zak and chn.
(No criticism implied, just trying to understand more fully.)

best
Oeyvind


2012/7/29 john ffitch :
> This short note considers what needs to be done to incorporate a new
> opcode safely into the parallel execution environment.
>
> Background:
> ==========
>
> The basic question is why can one not run two instruments in
> parallel.  It only takes a few moments' thought to realise that the
> issue is if one of the instruments changes something on which the
> other relies.  In the simple cases this is global variables that
> communicate between instruments.  With that in mind, and following
> the PhD of Jed Marti (Utah, 1980), we created an analysis program that
> tracked globals in an instruments, and arranged multiple dispatch
> based on the directed graph of dependencies -- taking account of
> reading and writing to globals separately.  This has been described
> in ICMC 2009 and LAC 2011.  There remain performance issues that need
> the code to be revised, but it is basically semantically correct,
> taking notice of instrument numeric order.
>
> Developments:
> ============
>
> An objection to what is written above is that there are more ways that
> an instrument may influence another, at least in Csound5.  These need
> to be identified and a scheme developed to handle them.
>
> Three such influences have been found, writing to tables, using
> channels, zap buss, and using the stack opcodes.
>
> The stack codes are easiest; push and pop both require unique access
> for the instrument to the stack.  We can thus treat this as a hidden
> global variable, as long as the parser knows which opcodes require
> this.
>
> If we consider tables we are immediately into a commoner and wider
> area or use.  Many opcodes read table values, but only a few create or
> modify tables.  Ideally we should track each table separately, but use
> of variables or expressions makes this impractical.  What is easy is
> to lump all tables together, and track reading and writing to ANY
> table.  Not the most efficient but table modification is not that
> common.  We can thus treat table access as reading/writing to another
> global variable.
>
> Channels and zak are then treatable like tables, with a different
> global each.
>
> This is what Csound5 does now.
>
> Implementation:
> ==============
>
> This requires that the parser knows which opcodes use tables, and this
> information is not clear in the system, short of reading the manual.
> What we need is a mechanism to signal to the parser that a particular
> opcode uses a table.
>
> So, the under-utilised field "thread" in OENTRY is pressed into
> service.  In earlier Csound this indicated whether the opcode was used
> at i-, k-, or a- rate, and also is used in some MIDI activity, but
> only a few bits are necessary for this in a 16-bit field.  So we can
> use the other bits.
>
> The file H/interlocks.h contains definitions for these bits
>
> ZR  ZAK reading
> ZW  ZAK writing
> ZB  ZAK read and write
>
> TR  tables reading
> TW  tables writing
> TB  tables read and write
>
> CR  channel reading
> CW  channel writing
> CB  channel read and write
>
> SB  stack used
>
> so in Engine/entries.c we see (for example)
>
> { "zkr",    S(ZKR),     ZR|3,   "k",  "k",    (SUBR)zkset,   (SUBR)zkr,   NULL},
> { "ziw",    S(ZKW),     ZW|1,   "",   "ii",   (SUBR)ziw,     NULL,  NULL},
> { "tablera", S(TABLERA),TR|5,   "a",  "kkk",
>                                       (SUBR)tableraset, NULL, (SUBR)tablera},
> { "tablewa", S(TABLEWA),TW|5,   "k",  "kak",
>                                       (SUBR)tablewaset, NULL, (SUBR)tablewa},
> { "chnget",      0xFFFF,    CR   },
>
>
> Implications:
> ============
>
> The implications of this are that anyone writing a new opcode needs to
> add these flags to the OENTRY to ensure parallel semantics.  If you
> have sone other secret persistent data you can use one of theses
> classes.
>
> Footnote:
> ========
>
> The final bit in the 16 (_QQ) is currently used to flag deprecated
> opcodes, but that is not currently inspected; it will be
>
>
>
> ==John ffitch
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel



-- 

Oeyvind Brandtsegg
Professor of Music Technology
NTNU
7491 Trondheim
Norway
Cell: +47 92 203 205

http://flyndresang.no/
http://www.partikkelaudio.com/
http://soundcloud.com/brandtsegg
http://soundcloud.com/t-emp

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

"Thanks Johh, for this. Could we find somewhere to put this so it is
seen more widely?"

I will add more in response to Oeyvind's questions and repost.
Could be a short note in the Journal or added to the documentation
somewhere?
==John ffitch

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net