Csound Csound-dev Csound-tekno Search About

[Csnd] C opcodes idea

Date2021-09-07 10:10
FromVictor Lazzarini
Subject[Csnd] C opcodes idea
Just following up Mike’s C/C++ opcodes idea for a bit, because that maybe a stepping stone for
an UDO JIT (or better On-Request-Compilation, ORC) system.

I see that it seems straightforward to leverage the LLVM stack to write an ORC code that may
generate native code to be inserted as an opcode Csound. Examples are there to do this
from an LLVM intermediate representation (IR). 

What I can’t easily locate is a means to produce the IR without having to use the clang
frontend. That is, to do it programmatically, taking the C code, producing an IR module
that can be put to the ORC code. So I asked a question

https://stackoverflow.com/questions/69084986/c-c-to-llvm-ir-api-is-there-a-programmatic-way-to-compile

Here’s a possible outline for a (init-time) opcode:

1) C code is given consisting of a data structure and two functions, plus the localops
static data.
2) That code is compiled to an IR.
3) The IR is ORC compiled to native
4) The module is stored in the Csound object
5) AppendOpcodes is called with the relevant symbols taken from localops.

The opcode is then available to Csound.

In a further iteration, we could leverage the Lexer/Parser to turn a UDO into a C opcode,
then re-use the above. Finally we could potentially go from UDO to IR if it makes sense.

All in a matter of time ;)
========================
Prof. Victor Lazzarini
Maynooth University
Ireland


Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2021-09-07 10:22
FromRory Walsh
SubjectRe: [Csnd] C opcodes idea
I'm way out of my depth on the low level details, but would this mean having to include LLVM JIT components with Csound?  

On Tue, 7 Sept 2021 at 10:11, Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
Just following up Mike’s C/C++ opcodes idea for a bit, because that maybe a stepping stone for
an UDO JIT (or better On-Request-Compilation, ORC) system.

I see that it seems straightforward to leverage the LLVM stack to write an ORC code that may
generate native code to be inserted as an opcode Csound. Examples are there to do this
from an LLVM intermediate representation (IR).

What I can’t easily locate is a means to produce the IR without having to use the clang
frontend. That is, to do it programmatically, taking the C code, producing an IR module
that can be put to the ORC code. So I asked a question

https://stackoverflow.com/questions/69084986/c-c-to-llvm-ir-api-is-there-a-programmatic-way-to-compile

Here’s a possible outline for a (init-time) opcode:

1) C code is given consisting of a data structure and two functions, plus the localops
static data.
2) That code is compiled to an IR.
3) The IR is ORC compiled to native
4) The module is stored in the Csound object
5) AppendOpcodes is called with the relevant symbols taken from localops.

The opcode is then available to Csound.

In a further iteration, we could leverage the Lexer/Parser to turn a UDO into a C opcode,
then re-use the above. Finally we could potentially go from UDO to IR if it makes sense.

All in a matter of time ;)
========================
Prof. Victor Lazzarini
Maynooth University
Ireland


Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Date2021-09-07 10:24
FromVictor Lazzarini
SubjectRe: [Csnd] [EXTERNAL] [Csnd] C opcodes idea
No, it would be a plugin opcode linked to LLVM (statically or dynamically), like the faust opcodes.
========================
Prof. Victor Lazzarini
Maynooth University
Ireland

> On 7 Sep 2021, at 10:22, Rory Walsh  wrote:
> 
> WARNINGThis email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
> I'm way out of my depth on the low level details, but would this mean having to include LLVM JIT components with Csound?  
> 
> On Tue, 7 Sept 2021 at 10:11, Victor Lazzarini  wrote:
> Just following up Mike’s C/C++ opcodes idea for a bit, because that maybe a stepping stone for
> an UDO JIT (or better On-Request-Compilation, ORC) system.
> 
> I see that it seems straightforward to leverage the LLVM stack to write an ORC code that may
> generate native code to be inserted as an opcode Csound. Examples are there to do this
> from an LLVM intermediate representation (IR). 
> 
> What I can’t easily locate is a means to produce the IR without having to use the clang
> frontend. That is, to do it programmatically, taking the C code, producing an IR module
> that can be put to the ORC code. So I asked a question
> 
> https://stackoverflow.com/questions/69084986/c-c-to-llvm-ir-api-is-there-a-programmatic-way-to-compile
> 
> Here’s a possible outline for a (init-time) opcode:
> 
> 1) C code is given consisting of a data structure and two functions, plus the localops
> static data.
> 2) That code is compiled to an IR.
> 3) The IR is ORC compiled to native
> 4) The module is stored in the Csound object
> 5) AppendOpcodes is called with the relevant symbols taken from localops.
> 
> The opcode is then available to Csound.
> 
> In a further iteration, we could leverage the Lexer/Parser to turn a UDO into a C opcode,
> then re-use the above. Finally we could potentially go from UDO to IR if it makes sense.
> 
> All in a matter of time ;)
> ========================
> Prof. Victor Lazzarini
> Maynooth University
> Ireland
> 
> 
> Csound mailing list
> Csound@listserv.heanet.ie
> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
> Send bugs reports to
>         https://github.com/csound/csound/issues
> Discussions of bugs and features can be posted here
> Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here


Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2021-09-07 10:32
FromRory Walsh
SubjectRe: [Csnd] [EXTERNAL] [Csnd] C opcodes idea
Oh nice. I assume static linking is permitted?

On Tue, 7 Sept 2021 at 10:24, Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
No, it would be a plugin opcode linked to LLVM (statically or dynamically), like the faust opcodes.
========================
Prof. Victor Lazzarini
Maynooth University
Ireland

> On 7 Sep 2021, at 10:22, Rory Walsh <rorywalsh@ear.ie> wrote:
>
> WARNINGThis email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
> I'm way out of my depth on the low level details, but would this mean having to include LLVM JIT components with Csound? 
>
> On Tue, 7 Sept 2021 at 10:11, Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
> Just following up Mike’s C/C++ opcodes idea for a bit, because that maybe a stepping stone for
> an UDO JIT (or better On-Request-Compilation, ORC) system.
>
> I see that it seems straightforward to leverage the LLVM stack to write an ORC code that may
> generate native code to be inserted as an opcode Csound. Examples are there to do this
> from an LLVM intermediate representation (IR).
>
> What I can’t easily locate is a means to produce the IR without having to use the clang
> frontend. That is, to do it programmatically, taking the C code, producing an IR module
> that can be put to the ORC code. So I asked a question
>
> https://stackoverflow.com/questions/69084986/c-c-to-llvm-ir-api-is-there-a-programmatic-way-to-compile
>
> Here’s a possible outline for a (init-time) opcode:
>
> 1) C code is given consisting of a data structure and two functions, plus the localops
> static data.
> 2) That code is compiled to an IR.
> 3) The IR is ORC compiled to native
> 4) The module is stored in the Csound object
> 5) AppendOpcodes is called with the relevant symbols taken from localops.
>
> The opcode is then available to Csound.
>
> In a further iteration, we could leverage the Lexer/Parser to turn a UDO into a C opcode,
> then re-use the above. Finally we could potentially go from UDO to IR if it makes sense.
>
> All in a matter of time ;)
> ========================
> Prof. Victor Lazzarini
> Maynooth University
> Ireland
>
>
> Csound mailing list
> Csound@listserv.heanet.ie
> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
> Send bugs reports to
>         https://github.com/csound/csound/issues
> Discussions of bugs and features can be posted here
> Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here


Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Date2021-09-07 10:33
FromVictor Lazzarini
SubjectRe: [Csnd] [EXTERNAL] [Csnd] C opcodes idea
yes, the released faust opcodes are linked statically to libfaust and LLVM. That’s common with LLVM stuff.
========================
Prof. Victor Lazzarini
Maynooth University
Ireland

> On 7 Sep 2021, at 10:32, Rory Walsh  wrote:
> 
> Oh nice. I assume static linking is permitted?
> 
> On Tue, 7 Sept 2021 at 10:24, Victor Lazzarini  wrote:
> No, it would be a plugin opcode linked to LLVM (statically or dynamically), like the faust opcodes.
> ========================
> Prof. Victor Lazzarini
> Maynooth University
> Ireland
> 
> > On 7 Sep 2021, at 10:22, Rory Walsh  wrote:
> > 
> > WARNINGThis email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
> > I'm way out of my depth on the low level details, but would this mean having to include LLVM JIT components with Csound?  
> > 
> > On Tue, 7 Sept 2021 at 10:11, Victor Lazzarini  wrote:
> > Just following up Mike’s C/C++ opcodes idea for a bit, because that maybe a stepping stone for
> > an UDO JIT (or better On-Request-Compilation, ORC) system.
> > 
> > I see that it seems straightforward to leverage the LLVM stack to write an ORC code that may
> > generate native code to be inserted as an opcode Csound. Examples are there to do this
> > from an LLVM intermediate representation (IR). 
> > 
> > What I can’t easily locate is a means to produce the IR without having to use the clang
> > frontend. That is, to do it programmatically, taking the C code, producing an IR module
> > that can be put to the ORC code. So I asked a question
> > 
> > https://stackoverflow.com/questions/69084986/c-c-to-llvm-ir-api-is-there-a-programmatic-way-to-compile
> > 
> > Here’s a possible outline for a (init-time) opcode:
> > 
> > 1) C code is given consisting of a data structure and two functions, plus the localops
> > static data.
> > 2) That code is compiled to an IR.
> > 3) The IR is ORC compiled to native
> > 4) The module is stored in the Csound object
> > 5) AppendOpcodes is called with the relevant symbols taken from localops.
> > 
> > The opcode is then available to Csound.
> > 
> > In a further iteration, we could leverage the Lexer/Parser to turn a UDO into a C opcode,
> > then re-use the above. Finally we could potentially go from UDO to IR if it makes sense.
> > 
> > All in a matter of time ;)
> > ========================
> > Prof. Victor Lazzarini
> > Maynooth University
> > Ireland
> > 
> > 
> > Csound mailing list
> > Csound@listserv.heanet.ie
> > https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
> > Send bugs reports to
> >         https://github.com/csound/csound/issues
> > Discussions of bugs and features can be posted here
> > Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here
> 
> 
> Csound mailing list
> Csound@listserv.heanet.ie
> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
> Send bugs reports to
>         https://github.com/csound/csound/issues
> Discussions of bugs and features can be posted here
> Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here


Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2021-09-07 12:12
FromMichael Gogins
SubjectRe: [Csnd] C opcodes idea
Not necessarily.

It could mean another set of plugin opcodes, call them the "cgen" opcodes, modeled after the "faustgen" opcodes. These would be used in the orchestra header, and then the C/C++ opcodes defined by the "cgen" opcodes would be available in instr code.

That way there would be no build dependencies of Csound itself upon the LLVM toolchain. Of course to actually use the "cgen" opcodes, they themselves would have to be compiled with LLVM and used with LLVM.

Regards,
Mike

On Tue, Sep 7, 2021, 05:23 Rory Walsh <rorywalsh@ear.ie> wrote:
I'm way out of my depth on the low level details, but would this mean having to include LLVM JIT components with Csound?  

On Tue, 7 Sept 2021 at 10:11, Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
Just following up Mike’s C/C++ opcodes idea for a bit, because that maybe a stepping stone for
an UDO JIT (or better On-Request-Compilation, ORC) system.

I see that it seems straightforward to leverage the LLVM stack to write an ORC code that may
generate native code to be inserted as an opcode Csound. Examples are there to do this
from an LLVM intermediate representation (IR).

What I can’t easily locate is a means to produce the IR without having to use the clang
frontend. That is, to do it programmatically, taking the C code, producing an IR module
that can be put to the ORC code. So I asked a question

https://stackoverflow.com/questions/69084986/c-c-to-llvm-ir-api-is-there-a-programmatic-way-to-compile

Here’s a possible outline for a (init-time) opcode:

1) C code is given consisting of a data structure and two functions, plus the localops
static data.
2) That code is compiled to an IR.
3) The IR is ORC compiled to native
4) The module is stored in the Csound object
5) AppendOpcodes is called with the relevant symbols taken from localops.

The opcode is then available to Csound.

In a further iteration, we could leverage the Lexer/Parser to turn a UDO into a C opcode,
then re-use the above. Finally we could potentially go from UDO to IR if it makes sense.

All in a matter of time ;)
========================
Prof. Victor Lazzarini
Maynooth University
Ireland


Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Date2021-09-07 14:19
FromJohann Philippe <000002c2c0737523-dmarc-request@LISTSERV.HEANET.IE>
SubjectRe: [Csnd] C opcodes idea
I enjoy the idea.
It could be interesting to have this as an option (a flag) so that the same csd could be compile with or without this  UDO JIT build system.

Johann Philippe

Le mardi 7 septembre 2021 à 13:12:29 UTC+2, Michael Gogins <michael.gogins@gmail.com> a écrit :


Not necessarily.

It could mean another set of plugin opcodes, call them the "cgen" opcodes, modeled after the "faustgen" opcodes. These would be used in the orchestra header, and then the C/C++ opcodes defined by the "cgen" opcodes would be available in instr code.

That way there would be no build dependencies of Csound itself upon the LLVM toolchain. Of course to actually use the "cgen" opcodes, they themselves would have to be compiled with LLVM and used with LLVM.

Regards,
Mike

On Tue, Sep 7, 2021, 05:23 Rory Walsh <rorywalsh@ear.ie> wrote:
I'm way out of my depth on the low level details, but would this mean having to include LLVM JIT components with Csound?  

On Tue, 7 Sept 2021 at 10:11, Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
Just following up Mike’s C/C++ opcodes idea for a bit, because that maybe a stepping stone for
an UDO JIT (or better On-Request-Compilation, ORC) system.

I see that it seems straightforward to leverage the LLVM stack to write an ORC code that may
generate native code to be inserted as an opcode Csound. Examples are there to do this
from an LLVM intermediate representation (IR).

What I can’t easily locate is a means to produce the IR without having to use the clang
frontend. That is, to do it programmatically, taking the C code, producing an IR module
that can be put to the ORC code. So I asked a question

https://stackoverflow.com/questions/69084986/c-c-to-llvm-ir-api-is-there-a-programmatic-way-to-compile

Here’s a possible outline for a (init-time) opcode:

1) C code is given consisting of a data structure and two functions, plus the localops
static data.
2) That code is compiled to an IR.
3) The IR is ORC compiled to native
4) The module is stored in the Csound object
5) AppendOpcodes is called with the relevant symbols taken from localops.

The opcode is then available to Csound.

In a further iteration, we could leverage the Lexer/Parser to turn a UDO into a C opcode,
then re-use the above. Finally we could potentially go from UDO to IR if it makes sense.

All in a matter of time ;)
========================
Prof. Victor Lazzarini
Maynooth University
Ireland


Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here