Data Flow Graph versus Programming Language

Michael Gogins

2 June 2012

Recently, Steven Yi proposed adding a type system to the Csound
orchestra language. I responded that it might be wiser to embed the
Csound data flow graph and opcodes into an existing high-level
programming language.

Steven then asked me to clarify my idea. So, my aim here is not to
make a concrete proposal for the future development of Csound, but
rather to talk about tradeoffs in software sound synthesis systems
design, and to clarify terms and concepts.

First, let’s be clear what we’re talking about here. A software sound
synthesis system (SWSS) is the term of art for systems such as Csound,
Max/MSP, or SuperCollider. These are user-programmable software sound
synthesizers that are used as musical instruments. Some have
additional capabilities for score generation, video processing, and so
on. Most are dynamic languages (e.g. Csound, Max/MSP, SuperCollider)
but some are statically compiled (JMusic, CLAM).

I have studied a number of SWSS. If I recall correctly, all the ones I
looked at are based on a data flow graph. Again, let’s be clear. A
data flow graph is a computer science term with a definite meaning.
Mathematically, it is a directed graph. The nodes generate or process
signals, and the nodes are connected by edges that conceptually are
first-in, first-out queues (FIFOs) of tokens, which represent signal
values (or other values). The topology of the graph determines the
order in which the nodes fire. The entire graph fires repeatedly until
all the required tokens have been produced.

A synchronous data flow graph is one in which the number of tokens
produced and consumed by each node is known in advance of firing.

Data flow graphs that contain special nodes called enable-invoke nodes
are capable of performing logical comparisons and jumps. Such graphs
are Turing complete and so are themselves a type of programming
language. They are called data flow control graphs.

Csound and most other SWSS are based on synchronous data flow control
graphs (SDFCGs). In Csound, the nodes are the instruments and opcodes,
and the edges are the variables that take a signal from the output of
one opcode and feed it into the parameters of other opcodes.
Instrument definitions also are nodes; they are the root nodes of the
subgraphs consisting of their opcodes. So the complete graph consists
of one list of instrument instances, each of which has its own
independent list of opcode instances. For every firing of the graph,
the input drivers are polled to feed the instrument pfields and input
buffers; this may cause new instrument instances to be created from a
template and patched into the graph. After that, each node in the
graph is fired by calling its subroutine, which consumes tokens from
the in edges (input variables), processes data, and produces tokens
into the out edges (output variables).

Similar data flow graphs are used not only in SWSS but also in a vast
variety of other signal processing applications in our culture. They
are used because they are well understood, and because there are
refined tools for translating block designs into data flow graphs and
from data flow graphs into code, even machine code. In particular,
modern software tools for data flow graphs are adept at scheduling
multi-rate graphs, determining an optimal static schedule, sharing
buffers, and so on.
SWSS in particular use data flow graphs because historically they have
been about the only way known to get decent runtime performance out of
a dynamic system (i.e. one that does not require static compilation).

The data flow graph design is, of course, not only only way to make a
SWSS. For example, it is possible to use an appropriately nested set
of closures in a functional programming language, then repeatedly call
the outermost closure. In that case, the function call graph performs
the same work on the same schedule as a data flow graph. It is also
possible to do the same work as a graph by explicitly coding all the
function calls in predetermined order in an ordinary procedural
programming language.
It is useful to note at this point that there now exist high-level
programming languages with just-in-time compilers whose programs run
as fast as statically compiled C programs. This obsoletes one of the
major reasons for using data flow graphs in SWSS.

Now, about the question of embedding.

In most data flow graphs, the nodes are written as blocks in a general
purpose programming language, and statically compiled as a library of
routines that the actual data flow graph will invoke upon firing. The
graph itself is often dynamically compiled. We can call this the
dynamic graph, static blocks design. Csound works this way (in Csound,
we call the blocks “opcodes”).
In other data flow graphs, the data flow graph itself is written as a
set of functions or as a class library in the same language as the
blocks for the nodes. We can call this the static graph, static blocks
design. CLAM and the Synthesis Toolkit in C++ (STK) work this way.

Logically one additional permutation is possible: the dynamic graph,
dynamic blocks design. It could be implemented in an efficient dynamic
language such as LuaJIT or JavaScript, and both the data flow graph
code and the block code would be written in the same language.

This last permutation is what I was trying to describe to Steven. The
existing Csound opcodes would be retained as C and C++ blocks. But it
would be possible to write efficient new blocks in the same dynamic
language as used for the data flow graph itself, such as LuaJIT. The
Lua opcodes in Csound have proved this concept. Such a language would
come with all the desirable features of a modern general-purpose, high
level programming language. There would be no need to create a new
type system as the dynamic language would already have an
industrial-strength type system. It should even be possible to change
the semantic actions in the new Csound compiler to wire up the data
flow graph in the dynamic language. It would also be possible to
directly code graphs in the dynamic language.

Now, let me list what I see as the pros and cons of each design permutation.

Dynamic graph, static blocks

Pro: Reasonably fast run time for the graph.
Pro: optimal run time for the blocks.
Pro: No external compiler required for graphs.
Con: External compiler required for blocks.
Con: The graph is one language, the blocks are another language. This
creates a perhaps unnecessary degree of complexity for both users and
programmers.

Static graph, static blocks

Pro: Optimal run time for graphs.
Pro: Optimal run time for blocks.
Con: External compiler required for graph.
Con: External compiler required for blocks.
Pro: Graph and blocks may easily be written in the same language
without a speed penalty. The design and use of the system are
therefore simpler.

Dynamic graph, dynamic blocks

Pro: Reasonably fast run time for graphs.
Pro: Reasonably fast run time for blocks.
Pro: No external compiler required for graphs.
Pro: No external compiler required for blocks.
Pro: Graph and blocks may easily be written in the same language
without a speed penalty. The design and use of the system are
therefore simpler.

I must add one more factor to be considered. As compiler technology
has matured and standardized, it has become easier and easier to
obtain free, open source compilers for most programming languages. I
recently have created dozens of compositions written directly in C++
using CsoundAC as an algorithmic composition and Csound as a SWSS
library. Simple batch commands in my text editor replace make or other
build systems. Compile times are quite reasonable, from a second or so
to a few dozen seconds. So it may be that not only are dynamic
languages now fast enough for SWSS, but external compilation also is
now fast enough and easy enough for SWSS.




-- 
Michael Gogins
Irreducible Productions
http://www.michael-gogins.com
Michael dot Gogins at gmail dot com

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Would there be some way to disconnect the graphs like you can do in ChucK?

On 6/3/12, Michael Gogins  wrote:
> Data Flow Graph versus Programming Language
>
> Michael Gogins
>
> 2 June 2012
>
> Recently, Steven Yi proposed adding a type system to the Csound
> orchestra language. I responded that it might be wiser to embed the
> Csound data flow graph and opcodes into an existing high-level
> programming language.
>
> Steven then asked me to clarify my idea. So, my aim here is not to
> make a concrete proposal for the future development of Csound, but
> rather to talk about tradeoffs in software sound synthesis systems
> design, and to clarify terms and concepts.
>
> First, let’s be clear what we’re talking about here. A software sound
> synthesis system (SWSS) is the term of art for systems such as Csound,
> Max/MSP, or SuperCollider. These are user-programmable software sound
> synthesizers that are used as musical instruments. Some have
> additional capabilities for score generation, video processing, and so
> on. Most are dynamic languages (e.g. Csound, Max/MSP, SuperCollider)
> but some are statically compiled (JMusic, CLAM).
>
> I have studied a number of SWSS. If I recall correctly, all the ones I
> looked at are based on a data flow graph. Again, let’s be clear. A
> data flow graph is a computer science term with a definite meaning.
> Mathematically, it is a directed graph. The nodes generate or process
> signals, and the nodes are connected by edges that conceptually are
> first-in, first-out queues (FIFOs) of tokens, which represent signal
> values (or other values). The topology of the graph determines the
> order in which the nodes fire. The entire graph fires repeatedly until
> all the required tokens have been produced.
>
> A synchronous data flow graph is one in which the number of tokens
> produced and consumed by each node is known in advance of firing.
>
> Data flow graphs that contain special nodes called enable-invoke nodes
> are capable of performing logical comparisons and jumps. Such graphs
> are Turing complete and so are themselves a type of programming
> language. They are called data flow control graphs.
>
> Csound and most other SWSS are based on synchronous data flow control
> graphs (SDFCGs). In Csound, the nodes are the instruments and opcodes,
> and the edges are the variables that take a signal from the output of
> one opcode and feed it into the parameters of other opcodes.
> Instrument definitions also are nodes; they are the root nodes of the
> subgraphs consisting of their opcodes. So the complete graph consists
> of one list of instrument instances, each of which has its own
> independent list of opcode instances. For every firing of the graph,
> the input drivers are polled to feed the instrument pfields and input
> buffers; this may cause new instrument instances to be created from a
> template and patched into the graph. After that, each node in the
> graph is fired by calling its subroutine, which consumes tokens from
> the in edges (input variables), processes data, and produces tokens
> into the out edges (output variables).
>
> Similar data flow graphs are used not only in SWSS but also in a vast
> variety of other signal processing applications in our culture. They
> are used because they are well understood, and because there are
> refined tools for translating block designs into data flow graphs and
> from data flow graphs into code, even machine code. In particular,
> modern software tools for data flow graphs are adept at scheduling
> multi-rate graphs, determining an optimal static schedule, sharing
> buffers, and so on.
> SWSS in particular use data flow graphs because historically they have
> been about the only way known to get decent runtime performance out of
> a dynamic system (i.e. one that does not require static compilation).
>
> The data flow graph design is, of course, not only only way to make a
> SWSS. For example, it is possible to use an appropriately nested set
> of closures in a functional programming language, then repeatedly call
> the outermost closure. In that case, the function call graph performs
> the same work on the same schedule as a data flow graph. It is also
> possible to do the same work as a graph by explicitly coding all the
> function calls in predetermined order in an ordinary procedural
> programming language.
> It is useful to note at this point that there now exist high-level
> programming languages with just-in-time compilers whose programs run
> as fast as statically compiled C programs. This obsoletes one of the
> major reasons for using data flow graphs in SWSS.
>
> Now, about the question of embedding.
>
> In most data flow graphs, the nodes are written as blocks in a general
> purpose programming language, and statically compiled as a library of
> routines that the actual data flow graph will invoke upon firing. The
> graph itself is often dynamically compiled. We can call this the
> dynamic graph, static blocks design. Csound works this way (in Csound,
> we call the blocks “opcodes”).
> In other data flow graphs, the data flow graph itself is written as a
> set of functions or as a class library in the same language as the
> blocks for the nodes. We can call this the static graph, static blocks
> design. CLAM and the Synthesis Toolkit in C++ (STK) work this way.
>
> Logically one additional permutation is possible: the dynamic graph,
> dynamic blocks design. It could be implemented in an efficient dynamic
> language such as LuaJIT or JavaScript, and both the data flow graph
> code and the block code would be written in the same language.
>
> This last permutation is what I was trying to describe to Steven. The
> existing Csound opcodes would be retained as C and C++ blocks. But it
> would be possible to write efficient new blocks in the same dynamic
> language as used for the data flow graph itself, such as LuaJIT. The
> Lua opcodes in Csound have proved this concept. Such a language would
> come with all the desirable features of a modern general-purpose, high
> level programming language. There would be no need to create a new
> type system as the dynamic language would already have an
> industrial-strength type system. It should even be possible to change
> the semantic actions in the new Csound compiler to wire up the data
> flow graph in the dynamic language. It would also be possible to
> directly code graphs in the dynamic language.
>
> Now, let me list what I see as the pros and cons of each design
> permutation.
>
> Dynamic graph, static blocks
>
> Pro: Reasonably fast run time for the graph.
> Pro: optimal run time for the blocks.
> Pro: No external compiler required for graphs.
> Con: External compiler required for blocks.
> Con: The graph is one language, the blocks are another language. This
> creates a perhaps unnecessary degree of complexity for both users and
> programmers.
>
> Static graph, static blocks
>
> Pro: Optimal run time for graphs.
> Pro: Optimal run time for blocks.
> Con: External compiler required for graph.
> Con: External compiler required for blocks.
> Pro: Graph and blocks may easily be written in the same language
> without a speed penalty. The design and use of the system are
> therefore simpler.
>
> Dynamic graph, dynamic blocks
>
> Pro: Reasonably fast run time for graphs.
> Pro: Reasonably fast run time for blocks.
> Pro: No external compiler required for graphs.
> Pro: No external compiler required for blocks.
> Pro: Graph and blocks may easily be written in the same language
> without a speed penalty. The design and use of the system are
> therefore simpler.
>
> I must add one more factor to be considered. As compiler technology
> has matured and standardized, it has become easier and easier to
> obtain free, open source compilers for most programming languages. I
> recently have created dozens of compositions written directly in C++
> using CsoundAC as an algorithmic composition and Csound as a SWSS
> library. Simple batch commands in my text editor replace make or other
> build systems. Compile times are quite reasonable, from a second or so
> to a few dozen seconds. So it may be that not only are dynamic
> languages now fast enough for SWSS, but external compilation also is
> now fast enough and easy enough for SWSS.
>
>
>
>
> --
> Michael Gogins
> Irreducible Productions
> http://www.michael-gogins.com
> Michael dot Gogins at gmail dot com
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel
>

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Possibly. The question is whether the graph is compiled and then runs
without change, or whether it is completely dynamic, or partly
dynamic. The static graphs run faster, the pluggable graphs have to be
partly dynamic. Csound is partly dynamic (incoming score events can
trigger insertion of new instrument instances into the graph). Even a
completely static graph could be pluggable, if the plugging resulted
in a new graph that was switched in for the old one just before the
next firing.

Regards,
Mike

On Mon, Jun 4, 2012 at 9:56 AM, Adam Puckett  wrote:
> Would there be some way to disconnect the graphs like you can do in ChucK?
>
> On 6/3/12, Michael Gogins  wrote:
>> Data Flow Graph versus Programming Language
>>
>> Michael Gogins
>>
>> 2 June 2012
>>
>> Recently, Steven Yi proposed adding a type system to the Csound
>> orchestra language. I responded that it might be wiser to embed the
>> Csound data flow graph and opcodes into an existing high-level
>> programming language.
>>
>> Steven then asked me to clarify my idea. So, my aim here is not to
>> make a concrete proposal for the future development of Csound, but
>> rather to talk about tradeoffs in software sound synthesis systems
>> design, and to clarify terms and concepts.
>>
>> First, let’s be clear what we’re talking about here. A software sound
>> synthesis system (SWSS) is the term of art for systems such as Csound,
>> Max/MSP, or SuperCollider. These are user-programmable software sound
>> synthesizers that are used as musical instruments. Some have
>> additional capabilities for score generation, video processing, and so
>> on. Most are dynamic languages (e.g. Csound, Max/MSP, SuperCollider)
>> but some are statically compiled (JMusic, CLAM).
>>
>> I have studied a number of SWSS. If I recall correctly, all the ones I
>> looked at are based on a data flow graph. Again, let’s be clear. A
>> data flow graph is a computer science term with a definite meaning.
>> Mathematically, it is a directed graph. The nodes generate or process
>> signals, and the nodes are connected by edges that conceptually are
>> first-in, first-out queues (FIFOs) of tokens, which represent signal
>> values (or other values). The topology of the graph determines the
>> order in which the nodes fire. The entire graph fires repeatedly until
>> all the required tokens have been produced.
>>
>> A synchronous data flow graph is one in which the number of tokens
>> produced and consumed by each node is known in advance of firing.
>>
>> Data flow graphs that contain special nodes called enable-invoke nodes
>> are capable of performing logical comparisons and jumps. Such graphs
>> are Turing complete and so are themselves a type of programming
>> language. They are called data flow control graphs.
>>
>> Csound and most other SWSS are based on synchronous data flow control
>> graphs (SDFCGs). In Csound, the nodes are the instruments and opcodes,
>> and the edges are the variables that take a signal from the output of
>> one opcode and feed it into the parameters of other opcodes.
>> Instrument definitions also are nodes; they are the root nodes of the
>> subgraphs consisting of their opcodes. So the complete graph consists
>> of one list of instrument instances, each of which has its own
>> independent list of opcode instances. For every firing of the graph,
>> the input drivers are polled to feed the instrument pfields and input
>> buffers; this may cause new instrument instances to be created from a
>> template and patched into the graph. After that, each node in the
>> graph is fired by calling its subroutine, which consumes tokens from
>> the in edges (input variables), processes data, and produces tokens
>> into the out edges (output variables).
>>
>> Similar data flow graphs are used not only in SWSS but also in a vast
>> variety of other signal processing applications in our culture. They
>> are used because they are well understood, and because there are
>> refined tools for translating block designs into data flow graphs and
>> from data flow graphs into code, even machine code. In particular,
>> modern software tools for data flow graphs are adept at scheduling
>> multi-rate graphs, determining an optimal static schedule, sharing
>> buffers, and so on.
>> SWSS in particular use data flow graphs because historically they have
>> been about the only way known to get decent runtime performance out of
>> a dynamic system (i.e. one that does not require static compilation).
>>
>> The data flow graph design is, of course, not only only way to make a
>> SWSS. For example, it is possible to use an appropriately nested set
>> of closures in a functional programming language, then repeatedly call
>> the outermost closure. In that case, the function call graph performs
>> the same work on the same schedule as a data flow graph. It is also
>> possible to do the same work as a graph by explicitly coding all the
>> function calls in predetermined order in an ordinary procedural
>> programming language.
>> It is useful to note at this point that there now exist high-level
>> programming languages with just-in-time compilers whose programs run
>> as fast as statically compiled C programs. This obsoletes one of the
>> major reasons for using data flow graphs in SWSS.
>>
>> Now, about the question of embedding.
>>
>> In most data flow graphs, the nodes are written as blocks in a general
>> purpose programming language, and statically compiled as a library of
>> routines that the actual data flow graph will invoke upon firing. The
>> graph itself is often dynamically compiled. We can call this the
>> dynamic graph, static blocks design. Csound works this way (in Csound,
>> we call the blocks “opcodes”).
>> In other data flow graphs, the data flow graph itself is written as a
>> set of functions or as a class library in the same language as the
>> blocks for the nodes. We can call this the static graph, static blocks
>> design. CLAM and the Synthesis Toolkit in C++ (STK) work this way.
>>
>> Logically one additional permutation is possible: the dynamic graph,
>> dynamic blocks design. It could be implemented in an efficient dynamic
>> language such as LuaJIT or JavaScript, and both the data flow graph
>> code and the block code would be written in the same language.
>>
>> This last permutation is what I was trying to describe to Steven. The
>> existing Csound opcodes would be retained as C and C++ blocks. But it
>> would be possible to write efficient new blocks in the same dynamic
>> language as used for the data flow graph itself, such as LuaJIT. The
>> Lua opcodes in Csound have proved this concept. Such a language would
>> come with all the desirable features of a modern general-purpose, high
>> level programming language. There would be no need to create a new
>> type system as the dynamic language would already have an
>> industrial-strength type system. It should even be possible to change
>> the semantic actions in the new Csound compiler to wire up the data
>> flow graph in the dynamic language. It would also be possible to
>> directly code graphs in the dynamic language.
>>
>> Now, let me list what I see as the pros and cons of each design
>> permutation.
>>
>> Dynamic graph, static blocks
>>
>> Pro: Reasonably fast run time for the graph.
>> Pro: optimal run time for the blocks.
>> Pro: No external compiler required for graphs.
>> Con: External compiler required for blocks.
>> Con: The graph is one language, the blocks are another language. This
>> creates a perhaps unnecessary degree of complexity for both users and
>> programmers.
>>
>> Static graph, static blocks
>>
>> Pro: Optimal run time for graphs.
>> Pro: Optimal run time for blocks.
>> Con: External compiler required for graph.
>> Con: External compiler required for blocks.
>> Pro: Graph and blocks may easily be written in the same language
>> without a speed penalty. The design and use of the system are
>> therefore simpler.
>>
>> Dynamic graph, dynamic blocks
>>
>> Pro: Reasonably fast run time for graphs.
>> Pro: Reasonably fast run time for blocks.
>> Pro: No external compiler required for graphs.
>> Pro: No external compiler required for blocks.
>> Pro: Graph and blocks may easily be written in the same language
>> without a speed penalty. The design and use of the system are
>> therefore simpler.
>>
>> I must add one more factor to be considered. As compiler technology
>> has matured and standardized, it has become easier and easier to
>> obtain free, open source compilers for most programming languages. I
>> recently have created dozens of compositions written directly in C++
>> using CsoundAC as an algorithmic composition and Csound as a SWSS
>> library. Simple batch commands in my text editor replace make or other
>> build systems. Compile times are quite reasonable, from a second or so
>> to a few dozen seconds. So it may be that not only are dynamic
>> languages now fast enough for SWSS, but external compilation also is
>> now fast enough and easy enough for SWSS.
>>
>>
>>
>>
>> --
>> Michael Gogins
>> Irreducible Productions
>> http://www.michael-gogins.com
>> Michael dot Gogins at gmail dot com
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Csound-devel mailing list
>> Csound-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/csound-devel
>>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel



-- 
Michael Gogins
Irreducible Productions
http://www.michael-gogins.com
Michael dot Gogins at gmail dot com

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Hi Michael,

Thanks for writing this.  However, I'm still confused, as what you
wrote did not change my understanding of your proposals.  I'll ask
again for some kind of pseudo-code to illustrate what you are
envisioning.  I have written some below to reiterate some points made
earlier; I'd welcome comments/critiques through code.

As I mentioned before, I do not think the type system proposal hinders
embedding, but rather helps enable it more easily.  You mention not
needing a type system, but I would argue as before that there already
is an intrinsic, hard-coded type system in csound.  A host language,
if creating a graph, would still need a means to connect the opcodes
(nodes) to each other through variables (edges).  Each edge has a type
in Csound, whether that is a-, k-, i-, f-, S-, w-, etc.  With a
generic type system, one can generate wrappers for each variable type
by querying.  With the current type system, one has to create wrappers
manually for each language. This is all for assembling a graph from a
host language.

For writing blocks or graphs from a host language, I would imagine
this to be clearer:

1. Instrument class, which host language can subclass.  In classic
Csound, we would instantiate a sub-class CsoundInstrument (well, it'd
be a struct with function pointers).  Pointers would have init,
perform, destroy.  Host languages could then just override methods.
This would allow creating instruments that runs in Csound's engine
that are completely written in a host-language.

2. Opcode class, which host language can subclass.  Host language
could then subclass to introduce opcodes written in host-language.

The coding styles I see from this (pseudo-python):

# Instrument with all methods in python
class MyInstr(Instrument):
  def init(self):
     super.init(self)
     ... insert standard python code ...

  def perform(self):
     ... do audio code in python, write output for csound to pickup ...

  def destroy(self):
    .. free up large resources ...

#instrument with Csound graph built in Python
class MyInstr2(Instrument):
  def init(self):
    super.init(self)
      ... create graph of csound opcodes...
      self.rootOp = someOpcode(1,2,3)

  def perform(self):
      writeOutput(CsoundGraphPerform(self.rootOp))

#Opcode
class MyOp(Opcode):
  def getInArgs(self):
    return "ak"
  def getOutArgs(self):
    return "a"
  def init(self):
    super.init(self)
  def process(self, csound):
    ...etc...

NOTE: MyInstr2 in the above does not perform the graph within the host
language, it only sets up the graph, and uses a utility method to
perform the graph within C code.  Otherwise, to perform manually, you
could do something like:

def perform(self):
  self.generator.perform()
  self.filterPerform.perform()
  pyVal = self.filterOutput.getValue() // self.filterOutput would be a
saved variable of a-type
  pyVal = doSomethingInPython(pyVal)
  writeOutput(pyVal)

One other thing to note: I see having a separate orc language as a
pro, not a con.  It has to do with point of view; I think
historically, it is important that the existing orc language has
survived some 20+ years as a means to describe and keep alive a body
of musical code.  I think practically, having a domain specific
language for audio is a good thing and allows sharing audio code
across different host languages.

So, I'll say again, I understand the desire to make more of Csound
available to a host language to have ways to implement in the host.
However, I do not see how one gets around the existing type-system in
Csound, nor how a newer type-system would not be an advantage for the
goals of more embedding possibilities.

Thanks,
steven

On Sun, Jun 3, 2012 at 6:37 PM, Michael Gogins  wrote:
> Data Flow Graph versus Programming Language
>
> Michael Gogins
>
> 2 June 2012
>
> Recently, Steven Yi proposed adding a type system to the Csound
> orchestra language. I responded that it might be wiser to embed the
> Csound data flow graph and opcodes into an existing high-level
> programming language.
>
> Steven then asked me to clarify my idea. So, my aim here is not to
> make a concrete proposal for the future development of Csound, but
> rather to talk about tradeoffs in software sound synthesis systems
> design, and to clarify terms and concepts.
>
> First, let’s be clear what we’re talking about here. A software sound
> synthesis system (SWSS) is the term of art for systems such as Csound,
> Max/MSP, or SuperCollider. These are user-programmable software sound
> synthesizers that are used as musical instruments. Some have
> additional capabilities for score generation, video processing, and so
> on. Most are dynamic languages (e.g. Csound, Max/MSP, SuperCollider)
> but some are statically compiled (JMusic, CLAM).
>
> I have studied a number of SWSS. If I recall correctly, all the ones I
> looked at are based on a data flow graph. Again, let’s be clear. A
> data flow graph is a computer science term with a definite meaning.
> Mathematically, it is a directed graph. The nodes generate or process
> signals, and the nodes are connected by edges that conceptually are
> first-in, first-out queues (FIFOs) of tokens, which represent signal
> values (or other values). The topology of the graph determines the
> order in which the nodes fire. The entire graph fires repeatedly until
> all the required tokens have been produced.
>
> A synchronous data flow graph is one in which the number of tokens
> produced and consumed by each node is known in advance of firing.
>
> Data flow graphs that contain special nodes called enable-invoke nodes
> are capable of performing logical comparisons and jumps. Such graphs
> are Turing complete and so are themselves a type of programming
> language. They are called data flow control graphs.
>
> Csound and most other SWSS are based on synchronous data flow control
> graphs (SDFCGs). In Csound, the nodes are the instruments and opcodes,
> and the edges are the variables that take a signal from the output of
> one opcode and feed it into the parameters of other opcodes.
> Instrument definitions also are nodes; they are the root nodes of the
> subgraphs consisting of their opcodes. So the complete graph consists
> of one list of instrument instances, each of which has its own
> independent list of opcode instances. For every firing of the graph,
> the input drivers are polled to feed the instrument pfields and input
> buffers; this may cause new instrument instances to be created from a
> template and patched into the graph. After that, each node in the
> graph is fired by calling its subroutine, which consumes tokens from
> the in edges (input variables), processes data, and produces tokens
> into the out edges (output variables).
>
> Similar data flow graphs are used not only in SWSS but also in a vast
> variety of other signal processing applications in our culture. They
> are used because they are well understood, and because there are
> refined tools for translating block designs into data flow graphs and
> from data flow graphs into code, even machine code. In particular,
> modern software tools for data flow graphs are adept at scheduling
> multi-rate graphs, determining an optimal static schedule, sharing
> buffers, and so on.
> SWSS in particular use data flow graphs because historically they have
> been about the only way known to get decent runtime performance out of
> a dynamic system (i.e. one that does not require static compilation).
>
> The data flow graph design is, of course, not only only way to make a
> SWSS. For example, it is possible to use an appropriately nested set
> of closures in a functional programming language, then repeatedly call
> the outermost closure. In that case, the function call graph performs
> the same work on the same schedule as a data flow graph. It is also
> possible to do the same work as a graph by explicitly coding all the
> function calls in predetermined order in an ordinary procedural
> programming language.
> It is useful to note at this point that there now exist high-level
> programming languages with just-in-time compilers whose programs run
> as fast as statically compiled C programs. This obsoletes one of the
> major reasons for using data flow graphs in SWSS.
>
> Now, about the question of embedding.
>
> In most data flow graphs, the nodes are written as blocks in a general
> purpose programming language, and statically compiled as a library of
> routines that the actual data flow graph will invoke upon firing. The
> graph itself is often dynamically compiled. We can call this the
> dynamic graph, static blocks design. Csound works this way (in Csound,
> we call the blocks “opcodes”).
> In other data flow graphs, the data flow graph itself is written as a
> set of functions or as a class library in the same language as the
> blocks for the nodes. We can call this the static graph, static blocks
> design. CLAM and the Synthesis Toolkit in C++ (STK) work this way.
>
> Logically one additional permutation is possible: the dynamic graph,
> dynamic blocks design. It could be implemented in an efficient dynamic
> language such as LuaJIT or JavaScript, and both the data flow graph
> code and the block code would be written in the same language.
>
> This last permutation is what I was trying to describe to Steven. The
> existing Csound opcodes would be retained as C and C++ blocks. But it
> would be possible to write efficient new blocks in the same dynamic
> language as used for the data flow graph itself, such as LuaJIT. The
> Lua opcodes in Csound have proved this concept. Such a language would
> come with all the desirable features of a modern general-purpose, high
> level programming language. There would be no need to create a new
> type system as the dynamic language would already have an
> industrial-strength type system. It should even be possible to change
> the semantic actions in the new Csound compiler to wire up the data
> flow graph in the dynamic language. It would also be possible to
> directly code graphs in the dynamic language.
>
> Now, let me list what I see as the pros and cons of each design permutation.
>
> Dynamic graph, static blocks
>
> Pro: Reasonably fast run time for the graph.
> Pro: optimal run time for the blocks.
> Pro: No external compiler required for graphs.
> Con: External compiler required for blocks.
> Con: The graph is one language, the blocks are another language. This
> creates a perhaps unnecessary degree of complexity for both users and
> programmers.
>
> Static graph, static blocks
>
> Pro: Optimal run time for graphs.
> Pro: Optimal run time for blocks.
> Con: External compiler required for graph.
> Con: External compiler required for blocks.
> Pro: Graph and blocks may easily be written in the same language
> without a speed penalty. The design and use of the system are
> therefore simpler.
>
> Dynamic graph, dynamic blocks
>
> Pro: Reasonably fast run time for graphs.
> Pro: Reasonably fast run time for blocks.
> Pro: No external compiler required for graphs.
> Pro: No external compiler required for blocks.
> Pro: Graph and blocks may easily be written in the same language
> without a speed penalty. The design and use of the system are
> therefore simpler.
>
> I must add one more factor to be considered. As compiler technology
> has matured and standardized, it has become easier and easier to
> obtain free, open source compilers for most programming languages. I
> recently have created dozens of compositions written directly in C++
> using CsoundAC as an algorithmic composition and Csound as a SWSS
> library. Simple batch commands in my text editor replace make or other
> build systems. Compile times are quite reasonable, from a second or so
> to a few dozen seconds. So it may be that not only are dynamic
> languages now fast enough for SWSS, but external compilation also is
> now fast enough and easy enough for SWSS.
>
>
>
>
> --
> Michael Gogins
> Irreducible Productions
> http://www.michael-gogins.com
> Michael dot Gogins at gmail dot com
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Responding to your response... in line.

I'd like to clear some things up first, though. I'm 100% committed to
complete backward compatibility for all existing Csound compositions.
So, I'm not proposing to replace the existing orc language. But I am
proposing to implement the orc compiler by generating the Csound data
flow graph using a "host language," which could _also_ be used for
writing pieces, instruments, or opcodes. Being able to write Csound
pieces, instruments, and opcodes in other languages is a goal of
Csound 6, no? Here I am simply exploring what seems to me the simplest
and most efficient way of doing this.

Perhaps the best way to understand this proposal is by analogy with
C++. When C++ first came along, it was implemented with a compiler
that did not generate machine code, but rather special C code, which
was then compiled by the regular C compiler. This actually worked
rather well. So, what I'm proposing is taking a "host language" and
writing a SWSS in it that uses the Csound opcodes, then changing the
Csound orc compiler so that it compiles Csound pieces written in the
traditional Csound language and syntax into programs in the host
language. As part of this compilation, there would be a synchronous
data flow control graph, written in the host language, that reflects
the current state of the art in such things.

But then, the host language could also be used to write pieces,
instruments, or opcodes. For some people, that would be a better way
of writing music. The host language would be a widely used, powerful
language such as C++, Lua, or whatever. As long as it's fast, robust,
and genuinely widely used.

On Tue, Jun 5, 2012 at 4:04 PM, Steven Yi  wrote:

> As I mentioned before, I do not think the type system proposal hinders
> embedding, but rather helps enable it more easily.  You mention not
> needing a type system,

Not so, I proposed using the type system of an existing general
purpose language.

>but I would argue as before that there already
> is an intrinsic, hard-coded type system in csound.  A host language,
> if creating a graph, would still need a means to connect the opcodes
> (nodes) to each other through variables (edges).  Each edge has a type
> in Csound, whether that is a-, k-, i-, f-, S-, w-, etc.

The graph would be rewritten in the host language such that it could
connect the Csound types as well as user-defined types.
This could be done in C++ for example with template metaprogramming,
or in Java with generics.

>With a
> generic type system, one can generate wrappers for each variable type
> by querying.

As I said, in C++ at compile time by templates, in dynamic languages
by querying as you say.

>With the current type system, one has to create wrappers
> manually for each language. This is all for assembling a graph from a
> host language.
>

I think it might be simpler to imagine replacing the current semantic
actions in the Csound orchestra language with new actions that
generate code in the host language.

> For writing blocks or graphs from a host language, I would imagine
> this to be clearer:
>
> 1. Instrument class, which host language can subclass.  In classic
> Csound, we would instantiate a sub-class CsoundInstrument (well, it'd
> be a struct with function pointers).  Pointers would have init,
> perform, destroy.  Host languages could then just override methods.
> This would allow creating instruments that runs in Csound's engine
> that are completely written in a host-language.

Yes, but in what I envision the Csound instrument base class is
declared in the host language. Either the Csound orchestra compiler,
or the host language compiler, would both be able to declare,
subclass, and instantiate Csound instruments deriving from this type.

Let me try to make this a bit clearer... the opcodes would be wrapped
in the host language, and the Csound data flow graph would not be
wrapped, it would be replaced by a new data flow graph in the host
language that could perfectly emulate the topology of the existing
Csound graph, or be constructed in new topologies.

In other words... supposing that the host language is LuaJIT, the
Csound graph would be written in Lua and the existing opcodes would
remain as C and C++ code, but would be called by LuaJIT's foreign
function interface (not wrapped via SWIG, as that's not as efficient).
Or, supposing the host language is C++, the Csound graph would be
written in C++ and the existing opcodes would remain as C and C++
code.

> 2. Opcode class, which host language can subclass.  Host language
> could then subclass to introduce opcodes written in host-language.

Yes, but there would be 2 ways or writing opcodes: as traditional
C/C++ Csound opcodes, or in the host language. The Csound graph would
know how to connect and call both kinds, probably through some kind of
abstract wrapper class. Host language opcodes would simply implement
the opcode interface, and traditional Csound opcodes would be wrapped
in a simple implementation of the interface. The existing C++ opcode
base class shows how this can be done in C++ with no, I repeat no,
runtime overhead and considerably easier to read code... similar
things could be done in LuaJIT.

The examples you give below work fine with what I am saying, if the
host language is Python.

> The coding styles I see from this (pseudo-python):
>
> # Instrument with all methods in python
> class MyInstr(Instrument):
>  def init(self):
>     super.init(self)
>     ... insert standard python code ...
>
>  def perform(self):
>     ... do audio code in python, write output for csound to pickup ...
>
>  def destroy(self):
>    .. free up large resources ...
>
> #instrument with Csound graph built in Python
> class MyInstr2(Instrument):
>  def init(self):
>    super.init(self)
>      ... create graph of csound opcodes...
>      self.rootOp = someOpcode(1,2,3)
>
>  def perform(self):
>      writeOutput(CsoundGraphPerform(self.rootOp))
>
> #Opcode
> class MyOp(Opcode):
>  def getInArgs(self):
>    return "ak"
>  def getOutArgs(self):
>    return "a"
>  def init(self):
>    super.init(self)
>  def process(self, csound):
>    ...etc...
>
> NOTE: MyInstr2 in the above does not perform the graph within the host
> language, it only sets up the graph, and uses a utility method to
> perform the graph within C code.  Otherwise, to perform manually, you
> could do something like:
>
> def perform(self):
>  self.generator.perform()
>  self.filterPerform.perform()
>  pyVal = self.filterOutput.getValue() // self.filterOutput would be a
> saved variable of a-type
>  pyVal = doSomethingInPython(pyVal)
>  writeOutput(pyVal)

> One other thing to note: I see having a separate orc language as a
> pro, not a con.  It has to do with point of view; I think
> historically, it is important that the existing orc language has
> survived some 20+ years as a means to describe and keep alive a body
> of musical code.  I think practically, having a domain specific
> language for audio is a good thing and allows sharing audio code
> across different host languages.

I am not proposing to replace the existing orc language! Horrors! I am
100% committed to complete backward compatibility. As I have said, the
Csound compiler would be able to create the new graph from existing
orc code.

At the same time, the style in your example could be used to write
orchestras completely in the host language, including new opcodes. I
think this would be a tremendous advantage -- it is why I am spending
so much time explaining this.

In other words, I think we can have our cake, and eat it too. We can
keep the backward compatibility, and we can create a new SWSS in which
compositions and opcodes can be written in the same language, which
would be a real language that more people would know.

> So, I'll say again, I understand the desire to make more of Csound
> available to a host language to have ways to implement in the host.
> However, I do not see how one gets around the existing type-system in
> Csound, nor how a newer type-system would not be an advantage for the
> goals of more embedding possibilities.

Again, a newer type system is definitely an advantage and I welcome
your desire to add one, but they already exist, there is no need to
invent one.

Best,
Mike

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Hi Michael,

I think I understand a bit more of what you are proposing.  However, I
tried to think it through and I have reservations about mapping Csound
into another language's language/runtime.  I see the following steps
in current csound orc:

1. ORC Parser reads in ORC, translates to TREE
2. Compiler reads in TREE, creates graph of structs and allocates
variables using hardcoded type system, to create an instruments and
UDO's
3. Csound engine performs

with your proposal, I understand it as:

1a. ORC Parser reads in ORC, translates to Engine Language (Lua, C++, etc.)
1b. User directly creates graph in Engine Language
2. Csound engine runs, using Engine Language Runtime

My gut feeling on this is that it would complicate things in a few ways.

1. This would add a very large dependency.  I would imagine that
choices for embedding a language would be Lua (popular for games) and
Scheme (GNU's official scripting language).   LuaJIT is a non-starter:
it doesn't work on Mac 64-bit without compiling the application with
special compiler settings and we can't control that for things like
Python or other scripting systems.  We would have to find a language
that would run fine on all platforms (including Android and iOS).
Then we have issues about how these might interact in a multi-threaded
environment.  It may be fine, it may not, but I imagine it'd take some
time to test/explore.

2. Runtimes could collide. For example, running a python-based
application would have the python runtime in addition to the Engine
Language runtime.  With current Csound, the engine is quite small, and
straight C.  With introducing another language's runtime as the basis
of the engine, I don't know what to expect. If you code in Lua, then
run Csound, which then uses Lua itself, could there be clashes in
globals, signal handlers, etc?  Again, something I'm not sure about,
but it's something of a red-flag. And what of TCL, or Java, or Common
Lisp?  It's just a gut feeling, but I feel more sure of straight C
code that is in Csound now that with a new engine runtime based on
another language.

3. As I see it, even if we did define the graph in another language,
we still need to wrap the current variable types, to convert between
the engine language's types into Csound's variable types. If we
rewrote all opcodes to use engine languages types, that'd be expensive
in development time.  If we wrap types and use convertors, that would
introduce non-negligible processing load, and we'd still want a
generic type system to ease introduction of new types in Csound ORC,
as well as wrapping.

I can see benefits to be able to define new opcode by the user in a
different language.  I can see benefits to creating new instruments
that do all processing in a host-language.  I think the path to
modifying Csound to allow these by wrappers and other things makes
sense, but I'm not sure we need to introduce a language runtime to do
this.  If the goal is to be able to write a piece in one language, as
well as create instrument and opcodes in that language, as well as to
control the graph and modify it at runtime, I think moving forward
with the generic type system and continuing modifying the API will be
easier to implement, and be simpler architecturally.

I also don't think this is so simple as just reusing some other
language runtime's type system.   There is a lot of unknowns I see,
especially with non-desktop platforms, and I don't think the benefits
outweigh the concerns.

Additionally, I don't think a generic type-system will be so big a
thing to develop.  One other thing too is that I think it is a safer
option to go this route at this time, because the work involved will
only make Csound more amenable to future changes that could involve
other runtimes.

For now, I will resume work on a generic type-system, but will do so
locally or in a branch.   We can then integrate if it's desirable, and
we can continue the conversation.  I'd also propose a larger change
such as introducing a language runtime change might be something we
look at for Csound 7 (then I'd rather look at LLVM and hope it only
stabilizes further across platforms).

Thanks! (And please correct if I've misinterpreted anything!)
steven


On Tue, Jun 5, 2012 at 8:23 PM, Michael Gogins  wrote:
> Responding to your response... in line.
>
> I'd like to clear some things up first, though. I'm 100% committed to
> complete backward compatibility for all existing Csound compositions.
> So, I'm not proposing to replace the existing orc language. But I am
> proposing to implement the orc compiler by generating the Csound data
> flow graph using a "host language," which could _also_ be used for
> writing pieces, instruments, or opcodes. Being able to write Csound
> pieces, instruments, and opcodes in other languages is a goal of
> Csound 6, no? Here I am simply exploring what seems to me the simplest
> and most efficient way of doing this.
>
> Perhaps the best way to understand this proposal is by analogy with
> C++. When C++ first came along, it was implemented with a compiler
> that did not generate machine code, but rather special C code, which
> was then compiled by the regular C compiler. This actually worked
> rather well. So, what I'm proposing is taking a "host language" and
> writing a SWSS in it that uses the Csound opcodes, then changing the
> Csound orc compiler so that it compiles Csound pieces written in the
> traditional Csound language and syntax into programs in the host
> language. As part of this compilation, there would be a synchronous
> data flow control graph, written in the host language, that reflects
> the current state of the art in such things.
>
> But then, the host language could also be used to write pieces,
> instruments, or opcodes. For some people, that would be a better way
> of writing music. The host language would be a widely used, powerful
> language such as C++, Lua, or whatever. As long as it's fast, robust,
> and genuinely widely used.
>
> On Tue, Jun 5, 2012 at 4:04 PM, Steven Yi  wrote:
>
>> As I mentioned before, I do not think the type system proposal hinders
>> embedding, but rather helps enable it more easily.  You mention not
>> needing a type system,
>
> Not so, I proposed using the type system of an existing general
> purpose language.
>
>>but I would argue as before that there already
>> is an intrinsic, hard-coded type system in csound.  A host language,
>> if creating a graph, would still need a means to connect the opcodes
>> (nodes) to each other through variables (edges).  Each edge has a type
>> in Csound, whether that is a-, k-, i-, f-, S-, w-, etc.
>
> The graph would be rewritten in the host language such that it could
> connect the Csound types as well as user-defined types.
> This could be done in C++ for example with template metaprogramming,
> or in Java with generics.
>
>>With a
>> generic type system, one can generate wrappers for each variable type
>> by querying.
>
> As I said, in C++ at compile time by templates, in dynamic languages
> by querying as you say.
>
>>With the current type system, one has to create wrappers
>> manually for each language. This is all for assembling a graph from a
>> host language.
>>
>
> I think it might be simpler to imagine replacing the current semantic
> actions in the Csound orchestra language with new actions that
> generate code in the host language.
>
>> For writing blocks or graphs from a host language, I would imagine
>> this to be clearer:
>>
>> 1. Instrument class, which host language can subclass.  In classic
>> Csound, we would instantiate a sub-class CsoundInstrument (well, it'd
>> be a struct with function pointers).  Pointers would have init,
>> perform, destroy.  Host languages could then just override methods.
>> This would allow creating instruments that runs in Csound's engine
>> that are completely written in a host-language.
>
> Yes, but in what I envision the Csound instrument base class is
> declared in the host language. Either the Csound orchestra compiler,
> or the host language compiler, would both be able to declare,
> subclass, and instantiate Csound instruments deriving from this type.
>
> Let me try to make this a bit clearer... the opcodes would be wrapped
> in the host language, and the Csound data flow graph would not be
> wrapped, it would be replaced by a new data flow graph in the host
> language that could perfectly emulate the topology of the existing
> Csound graph, or be constructed in new topologies.
>
> In other words... supposing that the host language is LuaJIT, the
> Csound graph would be written in Lua and the existing opcodes would
> remain as C and C++ code, but would be called by LuaJIT's foreign
> function interface (not wrapped via SWIG, as that's not as efficient).
> Or, supposing the host language is C++, the Csound graph would be
> written in C++ and the existing opcodes would remain as C and C++
> code.
>
>> 2. Opcode class, which host language can subclass.  Host language
>> could then subclass to introduce opcodes written in host-language.
>
> Yes, but there would be 2 ways or writing opcodes: as traditional
> C/C++ Csound opcodes, or in the host language. The Csound graph would
> know how to connect and call both kinds, probably through some kind of
> abstract wrapper class. Host language opcodes would simply implement
> the opcode interface, and traditional Csound opcodes would be wrapped
> in a simple implementation of the interface. The existing C++ opcode
> base class shows how this can be done in C++ with no, I repeat no,
> runtime overhead and considerably easier to read code... similar
> things could be done in LuaJIT.
>
> The examples you give below work fine with what I am saying, if the
> host language is Python.
>
>> The coding styles I see from this (pseudo-python):
>>
>> # Instrument with all methods in python
>> class MyInstr(Instrument):
>>  def init(self):
>>     super.init(self)
>>     ... insert standard python code ...
>>
>>  def perform(self):
>>     ... do audio code in python, write output for csound to pickup ...
>>
>>  def destroy(self):
>>    .. free up large resources ...
>>
>> #instrument with Csound graph built in Python
>> class MyInstr2(Instrument):
>>  def init(self):
>>    super.init(self)
>>      ... create graph of csound opcodes...
>>      self.rootOp = someOpcode(1,2,3)
>>
>>  def perform(self):
>>      writeOutput(CsoundGraphPerform(self.rootOp))
>>
>> #Opcode
>> class MyOp(Opcode):
>>  def getInArgs(self):
>>    return "ak"
>>  def getOutArgs(self):
>>    return "a"
>>  def init(self):
>>    super.init(self)
>>  def process(self, csound):
>>    ...etc...
>>
>> NOTE: MyInstr2 in the above does not perform the graph within the host
>> language, it only sets up the graph, and uses a utility method to
>> perform the graph within C code.  Otherwise, to perform manually, you
>> could do something like:
>>
>> def perform(self):
>>  self.generator.perform()
>>  self.filterPerform.perform()
>>  pyVal = self.filterOutput.getValue() // self.filterOutput would be a
>> saved variable of a-type
>>  pyVal = doSomethingInPython(pyVal)
>>  writeOutput(pyVal)
>
>> One other thing to note: I see having a separate orc language as a
>> pro, not a con.  It has to do with point of view; I think
>> historically, it is important that the existing orc language has
>> survived some 20+ years as a means to describe and keep alive a body
>> of musical code.  I think practically, having a domain specific
>> language for audio is a good thing and allows sharing audio code
>> across different host languages.
>
> I am not proposing to replace the existing orc language! Horrors! I am
> 100% committed to complete backward compatibility. As I have said, the
> Csound compiler would be able to create the new graph from existing
> orc code.
>
> At the same time, the style in your example could be used to write
> orchestras completely in the host language, including new opcodes. I
> think this would be a tremendous advantage -- it is why I am spending
> so much time explaining this.
>
> In other words, I think we can have our cake, and eat it too. We can
> keep the backward compatibility, and we can create a new SWSS in which
> compositions and opcodes can be written in the same language, which
> would be a real language that more people would know.
>
>> So, I'll say again, I understand the desire to make more of Csound
>> available to a host language to have ways to implement in the host.
>> However, I do not see how one gets around the existing type-system in
>> Csound, nor how a newer type-system would not be an advantage for the
>> goals of more embedding possibilities.
>
> Again, a newer type system is definitely an advantage and I welcome
> your desire to add one, but they already exist, there is no need to
> invent one.
>
> Best,
> Mike
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Csound-devel mailing list
> Csound-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/csound-devel

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net

Thanks for your thoughts. As before, I'll comment in line.

> with your proposal, I understand it as:
>
> 1a. ORC Parser reads in ORC, translates to Engine Language (Lua, C++, etc.)
> 1b. User directly creates graph in Engine Language
> 2. Csound engine runs, using Engine Language Runtime

Yes, with qualifications. 1a could mean use an API in the engine
language to instantiate the graph, not necessarily generating engine
language source code. I'm not actually sure which approach would be
best.

> My gut feeling on this is that it would complicate things in a few ways.
>
> 1. This would add a very large dependency.  I would imagine that
> choices for embedding a language would be Lua (popular for games) and
> Scheme (GNU's official scripting language).   LuaJIT is a non-starter:
> it doesn't work on Mac 64-bit without compiling the application with
> special compiler settings and we can't control that for things like
> Python or other scripting systems.  We would have to find a language
> that would run fine on all platforms (including Android and iOS).
> Then we have issues about how these might interact in a multi-threaded
> environment.  It may be fine, it may not, but I imagine it'd take some
> time to test/explore.

For engine languages I was thinking about LuaJIT, Scheme, JavaScript,
and C++. Of course C++ would have to be statically compiled but I have
come to think that this is not really a serious problem. The compiler
can be bundled with Csound as is the case in several other systems
that I have seen.

I actually think C++ with an encapsulated compiler is the best choice.
I think JavaScript is also a possibility.

You are correct that LuaJIT is not a good choice. I think this
language is extremely promising, but it is neither cross-platform
enough nor reliable enough for Csound.

> 2. Runtimes could collide. For example, running a python-based
> application would have the python runtime in addition to the Engine
> Language runtime.  With current Csound, the engine is quite small, and
> straight C.

Yes, I agree that this is a problem.

> With introducing another language's runtime as the basis
> of the engine, I don't know what to expect. If you code in Lua, then
> run Csound, which then uses Lua itself, could there be clashes in
> globals, signal handlers, etc?  Again, something I'm not sure about,
> but it's something of a red-flag. And what of TCL, or Java, or Common
> Lisp?

I don't think this is a problem.

> 3. As I see it, even if we did define the graph in another language,
> we still need to wrap the current variable types, to convert between
> the engine language's types into Csound's variable types.

As I have said before, this wrapping only needs to occur within the
semantic action functions of the orc compiler. This is very
straightforward.

> If we
> rewrote all opcodes to use engine languages types, that'd be expensive
> in development time.

This is completely unnecessary. The existing opcodes would simply, and
automatically, and efficiently, be wrapped. I am certain that this is
quite feasible.

> If we wrap types and use convertors, that would
> introduce non-negligible processing load,

No, because the type mapping occurs ONLY in the semantic actions, i.e.
at the time the csound orc compiler compiles the orc. It does not
occur at all during run time.

> and we'd still want a
> generic type system to ease introduction of new types in Csound ORC,
> as well as wrapping.

Yes, but the use of an engine language would provide exactly this
without requiring additional work. These types could be declared in
the csound orc language, and the Csound language would be provided
with new rules and actions to actually define these new types in the
engine language. This is mapping, but I think it is much easier than
you think it is.

> I can see benefits to be able to define new opcode by the user in a
> different language.  I can see benefits to creating new instruments
> that do all processing in a host-language.  I think the path to
> modifying Csound to allow these by wrappers and other things makes
> sense, but I'm not sure we need to introduce a language runtime to do
> this.  If the goal is to be able to write a piece in one language, as
> well as create instrument and opcodes in that language, as well as to
> control the graph and modify it at runtime, I think moving forward
> with the generic type system and continuing modifying the API will be
> easier to implement, and be simpler architecturally.

I think you under-estimate the difficulty of what you propose.

> Additionally, I don't think a generic type-system will be so big a
> thing to develop.  One other thing too is that I think it is a safer
> option to go this route at this time, because the work involved will
> only make Csound more amenable to future changes that could involve
> other runtimes.

Well, it's not going to stop with a type system... I fear that we will
pull Csound in the direction of re-inventing a new wheel every year
or so, rather than focusing on audio and music processing.

I want to stress this, it's the main reason for my proposal.

> For now, I will resume work on a generic type-system, but will do so
> locally or in a branch.   We can then integrate if it's desirable, and
> we can continue the conversation.  I'd also propose a larger change
> such as introducing a language runtime change might be something we
> look at for Csound 7 (then I'd rather look at LLVM and hope it only
> stabilizes further across platforms).

As an alternative to my proposal, let us consider using LLVM to
implement the Csound orc compiler. This would accomplish much of what
I propose because the "lower half" of my proposal and of the LLVM
setup would be pretty much the
same. A lot of the programming language features that we want are
already implemented in the LLVM "toolkit."

Again, thanks for your thoughts.

I've held off from actually writing any code about this, because my
time is limited and I'm trying to make music. I may take a look at
implementing the "embedding" of the Csound opcodes in a C++ data flow
graph and also "embedding" the C++ compiler. Pieces would be written
in C++ not Csound orc language, but it would develop better
understanding of the problem and permit some empirical comparisons
(speed etc.), perhaps also develop some ideas about the actual data
flow graph, however that ends up being implemented.

Best,
Mike

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Csound-devel mailing list
Csound-devel@lists.sourceforge.net