Hi Michael:

Nice proposal. I think the real issue is that nobody seems to have the time to properly maintain a new package manager. That's why some time ago we thought about re-using existing package managers. Sadly, we couldn't find a really good candidate (some aren't multi-platform, some are too complex for our needs, some are GUI-only, some weren't properly designed from the beginning, etc).

Having said that, maybe a step forward in order to improve our current situation could be:

1) to set-up (if it already isn't?) some sort of template project in github for third-party developers to imitate.

2) to have a web page inside csound.com that links to these github projects, or better to their corresponding github.io page where at least two things are located: the download link to the latest version and a help page in the style of the reference manual.

3) to have a clearly defined path inside user-space where users could download-and-drop these new plugins, UDOs, instruments defs., etc. Then maybe an associated "version" file could contain the necessary information for front-ends to check for current compatibility, or maybe even the aforementioned front-ends could make a quick online check to let the users know when there's a new version available for them to go download and replace the old ones with. Something like that.

Anyway, that's my two cents.

Cheers.

On 1/9/21 12:50, Michael Gogins wrote:

I think it's time to do this. It would solve most of the vexing issues with a plugins repository.

TL;DR: This is built into Csound. It automates UPLOADING extensions, and it automates FINDING and INSTALLING extensions. Unlike Steven Yi's proposal this does not automate BUILDING extensions, contributors are still responsible for that. But that could certainly be added later in the form of GitHub actions.
 

The details...  

Here's the current status of Csound versus its main alternatives regarding contributed extensions:

-- Csound (open source), contributors maintain and distribute their own extensions. Partial exceptions: plugins and risset.

-- Max/MSP (proprietary but most widely used), built-in package manager for contributed extensions.

-- ChucK (open source), plugins repository for contributed extensions.

-- Pure Data (open source), built-in package manager for contributed extensions.

-- Purr-data (open source), many extensions are contributed to the main repository by pull request and maintained there.

-- Supercollider (open source), plugins repository for contributed extensions, SuperCollider language extensions are in https://github.com/supercollider-quarks.

Proposal:

Csound gets a built-in package manager (cspm) for plugin opcodes, fgens, and extensions written in Csound languages (e.g. UDOs and instrument definitions). 

This is a csound utility program/opcode. It is loosely modeled on the existing risset package manager, vcpkg, and deken. Simply using vcpkg would work, but would force contributors to jump through unnecessary hoops.

The package manager maintains contributions in a GitHub repository named csound-extensions, licensed LGPLv2.1.

The package manager commands are:

--contribute - upload a contribution to the contributor's account in the csound-extensions repository. The contribution must consist of a directory in a standard format containing a package manifest and pre-built package contents. The contributor must have a GitHub account.

--list - list packages in the package repository.
--search - search in the package repository manifests.
--information - print the package manifest.
--install - install packages if the platform is supported.

--dependencies - list dependencies of packages.
--remove - remove packages.

A Csound extension package is a zip file with a .cse filename extension in the following structure based on Csound environment variables qualified where necessary by runtime architecture (specified as for vcpkg triplets, e.g. x64-windows, x64-osx, x64-linux).

manifest.json

SFDIR/ for soundfiles.
SSDIR/ for samples.
SADIR/ for analysis files.
INCDIR/ for .csd, orc., and include files.
OPCODE6DIR/triplet/ for loadable modules.
OPCODE6DIR64/triplet/ for loadable modules.
MFDIR/ for MIDI files.

deb/ for Debian packages.

exe/ for Windows installers.

dmg/ for MacOS installers.

apk/ for Android installers.

npm/ for NPM packages.

NODE/arch for binary Node.js and JW.js addons.

Extensions with third party dependencies are the main problem. Rather than solve this problem, either (a) the package contains an inner package that manages its own dependencies such as a vcpkg package or a Debian package, or (b) the manifest simply lists all third-party dependencies.

The manifest format is:

{

  name: "package_name";

  version: "1.2.3.alpha";

  author: "author name";

  license: "standard license name";

  description: "What is this extension for?";

  repository: "where if anywhere is the source code repository?";

  tags: "comma-delimited list of searchable tags.";

  dependencies: {

    "triplet_1:" [list of dependencies];

    "triplet_2": [list of dependencies];

    ...

  }; 

 };

It is not necessary to list the contents of the package, that can be inferred from the archive.

As I mentioned, it would be possible to add automated builds for all supported platforms later, and this of course would also require creating Git submodules or source directories for each contribution, but what I have outlined should be doable pretty quickly.

Regards,

Mike

The package manager uses the GitHub API to upload contributions from a contributor's package directory as a pull request. And it uses the GitHub API to download packages, which it then installs according to the manifest.

Regards,

Mike

-----------------------------------------------------

Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com

I get it, although personally I prefer the idea of not having a single repo tied to the csound organization where all pre-built packages end up. However, I believe the main issue remains: who's going to built and maintain this new multi-plaform package manager you propose?

On 1/9/21 14:56, Michael Gogins wrote:

Thanks for your comments!

By omitting the requirement of compiling extensions within the repository, and putting the onus for that on the contributors to the repository, I have omitted all the complexities that have bedeviled similar efforts in the past.

By making the package manager a builtin part of Csound, and documenting it, that would make installing extensions crystal clear to all users.

Perhaps you missed the part where I specified that the builtin package manager would also be capable of uploading new extensions?...

Regards,

Mike

-----------------------------------------------------

Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com

On Wed, Sep 1, 2021 at 1:14 PM Guillermo Senna <gsenna@gmail.com> wrote:

Hi Michael:

Nice proposal. I think the real issue is that nobody seems to have the time to properly maintain a new package manager. That's why some time ago we thought about re-using existing package managers. Sadly, we couldn't find a really good candidate (some aren't multi-platform, some are too complex for our needs, some are GUI-only, some weren't properly designed from the beginning, etc).

Having said that, maybe a step forward in order to improve our current situation could be:

1) to set-up (if it already isn't?) some sort of template project in github for third-party developers to imitate.

2) to have a web page inside csound.com that links to these github projects, or better to their corresponding github.io page where at least two things are located: the download link to the latest version and a help page in the style of the reference manual.

3) to have a clearly defined path inside user-space where users could download-and-drop these new plugins, UDOs, instruments defs., etc. Then maybe an associated "version" file could contain the necessary information for front-ends to check for current compatibility, or maybe even the aforementioned front-ends could make a quick online check to let the users know when there's a new version available for them to go download and replace the old ones with. Something like that.

Anyway, that's my two cents.

Cheers.

On 1/9/21 12:50, Michael Gogins wrote:

I think it's time to do this. It would solve most of the vexing issues with a plugins repository.

TL;DR: This is built into Csound. It automates UPLOADING extensions, and it automates FINDING and INSTALLING extensions. Unlike Steven Yi's proposal this does not automate BUILDING extensions, contributors are still responsible for that. But that could certainly be added later in the form of GitHub actions.
 

The details...  

Here's the current status of Csound versus its main alternatives regarding contributed extensions:

-- Csound (open source), contributors maintain and distribute their own extensions. Partial exceptions: plugins and risset.

-- Max/MSP (proprietary but most widely used), built-in package manager for contributed extensions.

-- ChucK (open source), plugins repository for contributed extensions.

-- Pure Data (open source), built-in package manager for contributed extensions.

-- Purr-data (open source), many extensions are contributed to the main repository by pull request and maintained there.

-- Supercollider (open source), plugins repository for contributed extensions, SuperCollider language extensions are in https://github.com/supercollider-quarks.

Proposal:

Csound gets a built-in package manager (cspm) for plugin opcodes, fgens, and extensions written in Csound languages (e.g. UDOs and instrument definitions). 

This is a csound utility program/opcode. It is loosely modeled on the existing risset package manager, vcpkg, and deken. Simply using vcpkg would work, but would force contributors to jump through unnecessary hoops.

The package manager maintains contributions in a GitHub repository named csound-extensions, licensed LGPLv2.1.

The package manager commands are:

--contribute - upload a contribution to the contributor's account in the csound-extensions repository. The contribution must consist of a directory in a standard format containing a package manifest and pre-built package contents. The contributor must have a GitHub account.

--list - list packages in the package repository.
--search - search in the package repository manifests.
--information - print the package manifest.
--install - install packages if the platform is supported.

--dependencies - list dependencies of packages.
--remove - remove packages.

A Csound extension package is a zip file with a .cse filename extension in the following structure based on Csound environment variables qualified where necessary by runtime architecture (specified as for vcpkg triplets, e.g. x64-windows, x64-osx, x64-linux).

manifest.json

SFDIR/ for soundfiles.
SSDIR/ for samples.
SADIR/ for analysis files.
INCDIR/ for .csd, orc., and include files.
OPCODE6DIR/triplet/ for loadable modules.
OPCODE6DIR64/triplet/ for loadable modules.
MFDIR/ for MIDI files.

deb/ for Debian packages.

exe/ for Windows installers.

dmg/ for MacOS installers.

apk/ for Android installers.

npm/ for NPM packages.

NODE/arch for binary Node.js and JW.js addons.

Extensions with third party dependencies are the main problem. Rather than solve this problem, either (a) the package contains an inner package that manages its own dependencies such as a vcpkg package or a Debian package, or (b) the manifest simply lists all third-party dependencies.

The manifest format is:

{

  name: "package_name";

  version: "1.2.3.alpha";

  author: "author name";

  license: "standard license name";

  description: "What is this extension for?";

  repository: "where if anywhere is the source code repository?";

  tags: "comma-delimited list of searchable tags.";

  dependencies: {

    "triplet_1:" [list of dependencies];

    "triplet_2": [list of dependencies];

    ...

  }; 

 };

It is not necessary to list the contents of the package, that can be inferred from the archive.

As I mentioned, it would be possible to add automated builds for all supported platforms later, and this of course would also require creating Git submodules or source directories for each contribution, but what I have outlined should be doable pretty quickly.

Regards,

Mike

The package manager uses the GitHub API to upload contributions from a contributor's package directory as a pull request. And it uses the GitHub API to download packages, which it then installs according to the manifest.

Regards,

Mike

-----------------------------------------------------

Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com

I take your point about not tying users of the package manager to one official extensions repository. I amend my proposal to be configurable by users with additional extensions repositories.

About maintenance, there would be two aspects:

-- Maintaining, in the core Csound repository, the package manager program, which both uploads and downloads extensions.

-- Maintaining the extensions repository, which is essentially just a database of archives. That doesn't sound like much work to me.

Regards,

Mike

-----------------------------------------------------

Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com

On Wed, Sep 1, 2021 at 2:29 PM Guillermo Senna <gsenna@gmail.com> wrote:

I get it, although personally I prefer the idea of not having a single repo tied to the csound organization where all pre-built packages end up. However, I believe the main issue remains: who's going to built and maintain this new multi-plaform package manager you propose?

On 1/9/21 14:56, Michael Gogins wrote:

Thanks for your comments!

By omitting the requirement of compiling extensions within the repository, and putting the onus for that on the contributors to the repository, I have omitted all the complexities that have bedeviled similar efforts in the past.

By making the package manager a builtin part of Csound, and documenting it, that would make installing extensions crystal clear to all users.

Perhaps you missed the part where I specified that the builtin package manager would also be capable of uploading new extensions?...

Regards,

Mike

-----------------------------------------------------

Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com

On Wed, Sep 1, 2021 at 1:14 PM Guillermo Senna <gsenna@gmail.com> wrote:

Hi Michael:

Nice proposal. I think the real issue is that nobody seems to have the time to properly maintain a new package manager. That's why some time ago we thought about re-using existing package managers. Sadly, we couldn't find a really good candidate (some aren't multi-platform, some are too complex for our needs, some are GUI-only, some weren't properly designed from the beginning, etc).

Having said that, maybe a step forward in order to improve our current situation could be:

1) to set-up (if it already isn't?) some sort of template project in github for third-party developers to imitate.

2) to have a web page inside csound.com that links to these github projects, or better to their corresponding github.io page where at least two things are located: the download link to the latest version and a help page in the style of the reference manual.

3) to have a clearly defined path inside user-space where users could download-and-drop these new plugins, UDOs, instruments defs., etc. Then maybe an associated "version" file could contain the necessary information for front-ends to check for current compatibility, or maybe even the aforementioned front-ends could make a quick online check to let the users know when there's a new version available for them to go download and replace the old ones with. Something like that.

Anyway, that's my two cents.

Cheers.

On 1/9/21 12:50, Michael Gogins wrote:

I think it's time to do this. It would solve most of the vexing issues with a plugins repository.

TL;DR: This is built into Csound. It automates UPLOADING extensions, and it automates FINDING and INSTALLING extensions. Unlike Steven Yi's proposal this does not automate BUILDING extensions, contributors are still responsible for that. But that could certainly be added later in the form of GitHub actions.
 

The details...  

Here's the current status of Csound versus its main alternatives regarding contributed extensions:

-- Csound (open source), contributors maintain and distribute their own extensions. Partial exceptions: plugins and risset.

-- Max/MSP (proprietary but most widely used), built-in package manager for contributed extensions.

-- ChucK (open source), plugins repository for contributed extensions.

-- Pure Data (open source), built-in package manager for contributed extensions.

-- Purr-data (open source), many extensions are contributed to the main repository by pull request and maintained there.

-- Supercollider (open source), plugins repository for contributed extensions, SuperCollider language extensions are in https://github.com/supercollider-quarks.

Proposal:

Csound gets a built-in package manager (cspm) for plugin opcodes, fgens, and extensions written in Csound languages (e.g. UDOs and instrument definitions). 

This is a csound utility program/opcode. It is loosely modeled on the existing risset package manager, vcpkg, and deken. Simply using vcpkg would work, but would force contributors to jump through unnecessary hoops.

The package manager maintains contributions in a GitHub repository named csound-extensions, licensed LGPLv2.1.

The package manager commands are:

--contribute - upload a contribution to the contributor's account in the csound-extensions repository. The contribution must consist of a directory in a standard format containing a package manifest and pre-built package contents. The contributor must have a GitHub account.

--list - list packages in the package repository.
--search - search in the package repository manifests.
--information - print the package manifest.
--install - install packages if the platform is supported.

--dependencies - list dependencies of packages.
--remove - remove packages.

A Csound extension package is a zip file with a .cse filename extension in the following structure based on Csound environment variables qualified where necessary by runtime architecture (specified as for vcpkg triplets, e.g. x64-windows, x64-osx, x64-linux).

manifest.json

SFDIR/ for soundfiles.
SSDIR/ for samples.
SADIR/ for analysis files.
INCDIR/ for .csd, orc., and include files.
OPCODE6DIR/triplet/ for loadable modules.
OPCODE6DIR64/triplet/ for loadable modules.
MFDIR/ for MIDI files.

deb/ for Debian packages.

exe/ for Windows installers.

dmg/ for MacOS installers.

apk/ for Android installers.

npm/ for NPM packages.

NODE/arch for binary Node.js and JW.js addons.

Extensions with third party dependencies are the main problem. Rather than solve this problem, either (a) the package contains an inner package that manages its own dependencies such as a vcpkg package or a Debian package, or (b) the manifest simply lists all third-party dependencies.

The manifest format is:

{

  name: "package_name";

  version: "1.2.3.alpha";

  author: "author name";

  license: "standard license name";

  description: "What is this extension for?";

  repository: "where if anywhere is the source code repository?";

  tags: "comma-delimited list of searchable tags.";

  dependencies: {

    "triplet_1:" [list of dependencies];

    "triplet_2": [list of dependencies];

    ...

  }; 

 };

It is not necessary to list the contents of the package, that can be inferred from the archive.

As I mentioned, it would be possible to add automated builds for all supported platforms later, and this of course would also require creating Git submodules or source directories for each contribution, but what I have outlined should be doable pretty quickly.

Regards,

Mike

The package manager uses the GitHub API to upload contributions from a contributor's package directory as a pull request. And it uses the GitHub API to download packages, which it then installs according to the manifest.

Regards,

Mike

-----------------------------------------------------

Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com

Michael,

I have a couple of concerns about this which I'd like to raise.

One is about sharing code which depends on these packages. If I get a
.csd file which depends on a number of external packages, how do I
know which ones I need, and which versions in order to run it? Would
the .csd need some metadata which specifies packages and version? If
it does, should it be enforced (so the modules don't get loaded unless
they are specified, like python's import system).

The second concern is about the volatility. If there are many
extensions with varying degrees of maintenance, particularly if they
have external dependencies, which in turn must be maintained, then the
promise that csound has always kept that I can write a piece now and
perform it in a decade seems difficult to keep.

I don't think either of these are reasons not to proceed but rather to
do so carefully.

 Joel

On Wed, 1 Sept 2021 at 19:54, Guillermo Senna <gsenna@gmail.com> wrote:
>
> Funny you say this, Victor. I apologize, but I haven't been able to follow all the recent threads in this list. However, I see now that not only 3) has been recently covered by Eduardo, but also that Steven, François and others have already suggested similar things to 1) and 2) over the past few days!
>
> On 1/9/21 15:01, Victor Lazzarini wrote:
>
> We already have 3), introduced by Eduardo. The plugins installers target that.
>
> Prof. Victor Lazzarini
> Maynooth University
> Ireland
>
> On 1 Sep 2021, at 18:14, Guillermo Senna <gsenna@gmail.com> wrote:
>
> 
> *Warning* This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
>
> Hi Michael:
>
> Nice proposal. I think the real issue is that nobody seems to have the time to properly maintain a new package manager. That's why some time ago we thought about re-using existing package managers. Sadly, we couldn't find a really good candidate (some aren't multi-platform, some are too complex for our needs, some are GUI-only, some weren't properly designed from the beginning, etc).
>
> Having said that, maybe a step forward in order to improve our current situation could be:
>
> 1) to set-up (if it already isn't?) some sort of template project in github for third-party developers to imitate.
>
> 2) to have a web page inside csound.com that links to these github projects, or better to their corresponding github.io page where at least two things are located: the download link to the latest version and a help page in the style of the reference manual.
>
> 3) to have a clearly defined path inside user-space where users could download-and-drop these new plugins, UDOs, instruments defs., etc. Then maybe an associated "version" file could contain the necessary information for front-ends to check for current compatibility, or maybe even the aforementioned front-ends could make a quick online check to let the users know when there's a new version available for them to go download and replace the old ones with. Something like that.
>
> Anyway, that's my two cents.
>
> Cheers.
>
>
>
> On 1/9/21 12:50, Michael Gogins wrote:
>
> I think it's time to do this. It would solve most of the vexing issues with a plugins repository.
>
> TL;DR: This is built into Csound. It automates UPLOADING extensions, and it automates FINDING and INSTALLING extensions. Unlike Steven Yi's proposal this does not automate BUILDING extensions, contributors are still responsible for that. But that could certainly be added later in the form of GitHub actions.
>
> The details...
>
> Here's the current status of Csound versus its main alternatives regarding contributed extensions:
>
> -- Csound (open source), contributors maintain and distribute their own extensions. Partial exceptions: plugins and risset.
>
> -- Max/MSP (proprietary but most widely used), built-in package manager for contributed extensions.
>
> -- ChucK (open source), plugins repository for contributed extensions.
>
> -- Pure Data (open source), built-in package manager for contributed extensions.
>
> -- Purr-data (open source), many extensions are contributed to the main repository by pull request and maintained there.
>
> -- Supercollider (open source), plugins repository for contributed extensions, SuperCollider language extensions are in https://github.com/supercollider-quarks.
>
> Proposal:
>
> Csound gets a built-in package manager (cspm) for plugin opcodes, fgens, and extensions written in Csound languages (e.g. UDOs and instrument definitions).
>
> This is a csound utility program/opcode. It is loosely modeled on the existing risset package manager, vcpkg, and deken. Simply using vcpkg would work, but would force contributors to jump through unnecessary hoops.
>
> The package manager maintains contributions in a GitHub repository named csound-extensions, licensed LGPLv2.1.
>
> The package manager commands are:
>
> --contribute - upload a contribution to the contributor's account in the csound-extensions repository. The contribution must consist of a directory in a standard format containing a package manifest and pre-built package contents. The contributor must have a GitHub account.
> --list - list packages in the package repository.
> --search - search in the package repository manifests.
> --information - print the package manifest.
> --install - install packages if the platform is supported.
> --dependencies - list dependencies of packages.
> --remove - remove packages.
>
> A Csound extension package is a zip file with a .cse filename extension in the following structure based on Csound environment variables qualified where necessary by runtime architecture (specified as for vcpkg triplets, e.g. x64-windows, x64-osx, x64-linux).
>
> manifest.json
> SFDIR/ for soundfiles.
> SSDIR/ for samples.
> SADIR/ for analysis files.
> INCDIR/ for .csd, orc., and include files.
> OPCODE6DIR/triplet/ for loadable modules.
> OPCODE6DIR64/triplet/ for loadable modules.
> MFDIR/ for MIDI files.
> deb/ for Debian packages.
> exe/ for Windows installers.
> dmg/ for MacOS installers.
> apk/ for Android installers.
> npm/ for NPM packages.
> NODE/arch for binary Node.js and JW.js addons.
>
> Extensions with third party dependencies are the main problem. Rather than solve this problem, either (a) the package contains an inner package that manages its own dependencies such as a vcpkg package or a Debian package, or (b) the manifest simply lists all third-party dependencies.
>
> The manifest format is:
> {
>   name: "package_name";
>   version: "1.2.3.alpha";
>   author: "author name";
>   license: "standard license name";
>   description: "What is this extension for?";
>   repository: "where if anywhere is the source code repository?";
>   tags: "comma-delimited list of searchable tags.";
>   dependencies: {
>     "triplet_1:" [list of dependencies];
>     "triplet_2": [list of dependencies];
>     ...
>   };
>  };
>
> It is not necessary to list the contents of the package, that can be inferred from the archive.
>
> As I mentioned, it would be possible to add automated builds for all supported platforms later, and this of course would also require creating Git submodules or source directories for each contribution, but what I have outlined should be doable pretty quickly.
>
> Regards,
> Mike
>
>
>
>
>
> The package manager uses the GitHub API to upload contributions from a contributor's package directory as a pull request. And it uses the GitHub API to download packages, which it then installs according to the manifest.
>
> Regards,
> Mike
>
>
> -----------------------------------------------------
> Michael Gogins
> Irreducible Productions
> http://michaelgogins.tumblr.com
> Michael dot Gogins at gmail dot com

Hi Mike,

 

Sound good…I think that Cabbage is running in this way since there is a automated build for Mac, Linux and Windows.

 

Regards,

 

Marc

 

 

Envoyé à partir de Courrier pour Windows

 

De : Michael Gogins
Envoyé le :mercredi 1 septembre 2021 11:50
À : CSOUND@LISTSERV.HEANET.IE
Objet :[Csnd] Package Manager for Csound

 

I think it's time to do this. It would solve most of the vexing issues with a plugins repository.

 

TL;DR: This is built into Csound. It automates UPLOADING extensions, and it automates FINDING and INSTALLING extensions. Unlike Steven Yi's proposal this does not automate BUILDING extensions, contributors are still responsible for that. But that could certainly be added later in the form of GitHub actions.
 

The details...  

 

Here's the current status of Csound versus its main alternatives regarding contributed extensions:

 

-- Csound (open source), contributors maintain and distribute their own extensions. Partial exceptions: plugins and risset.

 

-- Max/MSP (proprietary but most widely used), built-in package manager for contributed extensions.

 

-- ChucK (open source), plugins repository for contributed extensions.

 

-- Pure Data (open source), built-in package manager for contributed extensions.

 

-- Purr-data (open source), many extensions are contributed to the main repository by pull request and maintained there.

 

-- Supercollider (open source), plugins repository for contributed extensions, SuperCollider language extensions are in https://github.com/supercollider-quarks.

 

Proposal:

 

Csound gets a built-in package manager (cspm) for plugin opcodes, fgens, and extensions written in Csound languages (e.g. UDOs and instrument definitions). 

 

This is a csound utility program/opcode. It is loosely modeled on the existing risset package manager, vcpkg, and deken. Simply using vcpkg would work, but would force contributors to jump through unnecessary hoops.

 

The package manager maintains contributions in a GitHub repository named csound-extensions, licensed LGPLv2.1.

 

The package manager commands are:

 

--contribute - upload a contribution to the contributor's account in the csound-extensions repository. The contribution must consist of a directory in a standard format containing a package manifest and pre-built package contents. The contributor must have a GitHub account.

--list - list packages in the package repository.
--search - search in the package repository manifests.
--information - print the package manifest.
--install - install packages if the platform is supported.

--dependencies - list dependencies of packages.
--remove - remove packages.

 

A Csound extension package is a zip file with a .cse filename extension in the following structure based on Csound environment variables qualified where necessary by runtime architecture (specified as for vcpkg triplets, e.g. x64-windows, x64-osx, x64-linux).

 

manifest.json

SFDIR/ for soundfiles.
SSDIR/ for samples.
SADIR/ for analysis files.
INCDIR/ for .csd, orc., and include files.
OPCODE6DIR/triplet/ for loadable modules.
OPCODE6DIR64/triplet/ for loadable modules.
MFDIR/ for MIDI files.

deb/ for Debian packages.

exe/ for Windows installers.

dmg/ for MacOS installers.

apk/ for Android installers.

npm/ for NPM packages.

NODE/arch for binary Node.js and JW.js addons.

 

Extensions with third party dependencies are the main problem. Rather than solve this problem, either (a) the package contains an inner package that manages its own dependencies such as a vcpkg package or a Debian package, or (b) the manifest simply lists all third-party dependencies.

 

The manifest format is:

{

  name: "package_name";

  version: "1.2.3.alpha";

  author: "author name";

  license: "standard license name";

  description: "What is this extension for?";

  repository: "where if anywhere is the source code repository?";

  tags: "comma-delimited list of searchable tags.";

  dependencies: {

    "triplet_1:" [list of dependencies];

    "triplet_2": [list of dependencies];

    ...

  }; 

 };

 

It is not necessary to list the contents of the package, that can be inferred from the archive.

 

As I mentioned, it would be possible to add automated builds for all supported platforms later, and this of course would also require creating Git submodules or source directories for each contribution, but what I have outlined should be doable pretty quickly.

 

Regards,

Mike

 

 

 

 

 

The package manager uses the GitHub API to upload contributions from a contributor's package directory as a pull request. And it uses the GitHub API to download packages, which it then installs according to the manifest.

 

Regards,

Mike

 

-----------------------------------------------------

Michael Gogins
Irreducible Productions
http://michaelgogins.tumblr.com
Michael dot Gogins at gmail dot com

Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

 

Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Also, if the package manager itself is part of Csound, it becomes an extra element to keep maintained
and kept running correctly. It increases the complexity of the system as a whole.
========================
Prof. Victor Lazzarini
Maynooth University
Ireland

> On 1 Sep 2021, at 19:41, Rory Walsh <rorywalsh@ear.ie> wrote:
>
> WARNINGThis email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
> Maintaining this is one thing, but who is going to develop and build the system?
>
> On Wed, 1 Sept 2021 at 19:36, Michael Gogins <michael.gogins@gmail.com> wrote:
> I take your point about not tying users of the package manager to one official extensions repository. I amend my proposal to be configurable by users with additional extensions repositories.
>
> About maintenance, there would be two aspects:
>
> -- Maintaining, in the core Csound repository, the package manager program, which both uploads and downloads extensions.
>
> -- Maintaining the extensions repository, which is essentially just a database of archives. That doesn't sound like much work to me.
>
> Regards,
> Mike
>
> -----------------------------------------------------
> Michael Gogins
> Irreducible Productions
> http://michaelgogins.tumblr.com
> Michael dot Gogins at gmail dot com
>
>
> On Wed, Sep 1, 2021 at 2:29 PM Guillermo Senna <gsenna@gmail.com> wrote:
> I get it, although personally I prefer the idea of not having a single repo tied to the csound organization where all pre-built packages end up. However, I believe the main issue remains: who's going to built and maintain this new multi-plaform package manager you propose?
>
> On 1/9/21 14:56, Michael Gogins wrote:
>> Thanks for your comments!
>>
>> By omitting the requirement of compiling extensions within the repository, and putting the onus for that on the contributors to the repository, I have omitted all the complexities that have bedeviled similar efforts in the past.
>>
>> By making the package manager a builtin part of Csound, and documenting it, that would make installing extensions crystal clear to all users.
>>
>> Perhaps you missed the part where I specified that the builtin package manager would also be capable of uploading new extensions?...
>>
>> Regards,
>> Mike
>>
>> -----------------------------------------------------
>> Michael Gogins
>> Irreducible Productions
>> http://michaelgogins.tumblr.com
>> Michael dot Gogins at gmail dot com
>>
>>
>> On Wed, Sep 1, 2021 at 1:14 PM Guillermo Senna <gsenna@gmail.com> wrote:
>> Hi Michael:
>>
>> Nice proposal. I think the real issue is that nobody seems to have the time to properly maintain a new package manager. That's why some time ago we thought about re-using existing package managers. Sadly, we couldn't find a really good candidate (some aren't multi-platform, some are too complex for our needs, some are GUI-only, some weren't properly designed from the beginning, etc).
>>
>> Having said that, maybe a step forward in order to improve our current situation could be:
>>
>> 1) to set-up (if it already isn't?) some sort of template project in github for third-party developers to imitate.
>>
>> 2) to have a web page inside csound.com that links to these github projects, or better to their corresponding github.io page where at least two things are located: the download link to the latest version and a help page in the style of the reference manual.
>>
>> 3) to have a clearly defined path inside user-space where users could download-and-drop these new plugins, UDOs, instruments defs., etc. Then maybe an associated "version" file could contain the necessary information for front-ends to check for current compatibility, or maybe even the aforementioned front-ends could make a quick online check to let the users know when there's a new version available for them to go download and replace the old ones with. Something like that.
>>
>> Anyway, that's my two cents.
>>
>> Cheers.
>>
>>
>>
>>
>>
>> On 1/9/21 12:50, Michael Gogins wrote:
>>> I think it's time to do this. It would solve most of the vexing issues with a plugins repository.
>>>
>>> TL;DR: This is built into Csound. It automates UPLOADING extensions, and it automates FINDING and INSTALLING extensions. Unlike Steven Yi's proposal this does not automate BUILDING extensions, contributors are still responsible for that. But that could certainly be added later in the form of GitHub actions.
>>> 
>>> The details... 
>>>
>>> Here's the current status of Csound versus its main alternatives regarding contributed extensions:
>>>
>>> -- Csound (open source), contributors maintain and distribute their own extensions. Partial exceptions: plugins and risset.
>>>
>>> -- Max/MSP (proprietary but most widely used), built-in package manager for contributed extensions.
>>>
>>> -- ChucK (open source), plugins repository for contributed extensions.
>>>
>>> -- Pure Data (open source), built-in package manager for contributed extensions.
>>>
>>> -- Purr-data (open source), many extensions are contributed to the main repository by pull request and maintained there.
>>>
>>> -- Supercollider (open source), plugins repository for contributed extensions, SuperCollider language extensions are in https://github.com/supercollider-quarks.
>>>
>>> Proposal:
>>>
>>> Csound gets a built-in package manager (cspm) for plugin opcodes, fgens, and extensions written in Csound languages (e.g. UDOs and instrument definitions).
>>>
>>> This is a csound utility program/opcode. It is loosely modeled on the existing risset package manager, vcpkg, and deken. Simply using vcpkg would work, but would force contributors to jump through unnecessary hoops.
>>>
>>> The package manager maintains contributions in a GitHub repository named csound-extensions, licensed LGPLv2.1.
>>>
>>> The package manager commands are:
>>>
>>> --contribute - upload a contribution to the contributor's account in the csound-extensions repository. The contribution must consist of a directory in a standard format containing a package manifest and pre-built package contents. The contributor must have a GitHub account.
>>> --list - list packages in the package repository.
>>> --search - search in the package repository manifests.
>>> --information - print the package manifest.
>>> --install - install packages if the platform is supported.
>>> --dependencies - list dependencies of packages.
>>> --remove - remove packages.
>>>
>>> A Csound extension package is a zip file with a .cse filename extension in the following structure based on Csound environment variables qualified where necessary by runtime architecture (specified as for vcpkg triplets, e.g. x64-windows, x64-osx, x64-linux).
>>>
>>> manifest.json
>>> SFDIR/ for soundfiles.
>>> SSDIR/ for samples.
>>> SADIR/ for analysis files.
>>> INCDIR/ for .csd, orc., and include files.
>>> OPCODE6DIR/triplet/ for loadable modules.
>>> OPCODE6DIR64/triplet/ for loadable modules.
>>> MFDIR/ for MIDI files.
>>> deb/ for Debian packages.
>>> exe/ for Windows installers.
>>> dmg/ for MacOS installers.
>>> apk/ for Android installers.
>>> npm/ for NPM packages.
>>> NODE/arch for binary Node.js and JW.js addons.
>>>
>>> Extensions with third party dependencies are the main problem. Rather than solve this problem, either (a) the package contains an inner package that manages its own dependencies such as a vcpkg package or a Debian package, or (b) the manifest simply lists all third-party dependencies.
>>>
>>> The manifest format is:
>>> {
>>>   name: "package_name";
>>>   version: "1.2.3.alpha";
>>>   author: "author name";
>>>   license: "standard license name";
>>>   description: "What is this extension for?";
>>>   repository: "where if anywhere is the source code repository?";
>>>   tags: "comma-delimited list of searchable tags.";
>>>   dependencies: {
>>>     "triplet_1:" [list of dependencies];
>>>     "triplet_2": [list of dependencies];
>>>     ...
>>>   };
>>>  };
>>>
>>> It is not necessary to list the contents of the package, that can be inferred from the archive.
>>>
>>> As I mentioned, it would be possible to add automated builds for all supported platforms later, and this of course would also require creating Git submodules or source directories for each contribution, but what I have outlined should be doable pretty quickly.
>>>
>>> Regards,
>>> Mike
>>>
>>>
>>>
>>>
>>>
>>> The package manager uses the GitHub API to upload contributions from a contributor's package directory as a pull request. And it uses the GitHub API to download packages, which it then installs according to the manifest.
>>>
>>> Regards,
>>> Mike
>>>
>>>
>>> -----------------------------------------------------
>>> Michael Gogins
>>> Irreducible Productions
>>> http://michaelgogins.tumblr.com
>>> Michael dot Gogins at gmail dot com