Csound Csound-dev Csound-tekno Search About

[Csnd-dev] CI Build Changes / Windows changes

Date2021-08-25 23:36
FromStephen Kyne
Subject[Csnd-dev] CI Build Changes / Windows changes
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 02:05
FromDave Seidel
SubjectRe: [Csnd-dev] CI Build Changes / Windows changes
I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 08:48
FromVictor Lazzarini
SubjectRe: [Csnd-dev] [EXTERNAL] [Csnd-dev] CI Build Changes / Windows changes
This is absolutely great, thanks for your work on it.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 25 Aug 2021, at 23:36, Stephen Kyne <stevek@outlook.ie> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 09:00
FromVictor Lazzarini
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 10:48
FromRory Walsh
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
Csound for windows would be in dire straits without your contribution Stephen. I'll try a local build next week once I'm back at my windows machine. I'll probably have a few questions then. I'll reach out over discord 👍 

On Thu 26 Aug 2021, 9:00 a.m. Victor Lazzarini, <Victor.Lazzarini@mu.ie> wrote:
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 12:43
FromDave Seidel
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
I'm running "git pull --recurse-submodules" and it seems to be doing the right thing. Thanks, Victor!

On Thu, Aug 26, 2021 at 4:00 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 13:16
FromVictor Lazzarini
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
yes, that's right, no need to clone again if you have already done it.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 12:45, Dave Seidel <dave.seidel@gmail.com> wrote:


I'm running "git pull --recurse-submodules" and it seems to be doing the right thing. Thanks, Victor!

On Thu, Aug 26, 2021 at 4:00 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 13:33
FromDave Seidel
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
Closer, but now (this is during the cmake configuration step):

CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!


On Thu, Aug 26, 2021 at 8:16 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
yes, that's right, no need to clone again if you have already done it.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 12:45, Dave Seidel <dave.seidel@gmail.com> wrote:


I'm running "git pull --recurse-submodules" and it seems to be doing the right thing. Thanks, Victor!

On Thu, Aug 26, 2021 at 4:00 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 13:35
FromDave Seidel
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
I was able to run the bootstrap script after pulling in the vcpkg submodules, but there was no visible sign of it pulling down any dependencies other than vcpkg.exe.

On Thu, Aug 26, 2021 at 8:33 AM Dave Seidel <dave.seidel@gmail.com> wrote:
Closer, but now (this is during the cmake configuration step):

CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!


On Thu, Aug 26, 2021 at 8:16 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
yes, that's right, no need to clone again if you have already done it.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 12:45, Dave Seidel <dave.seidel@gmail.com> wrote:


I'm running "git pull --recurse-submodules" and it seems to be doing the right thing. Thanks, Victor!

On Thu, Aug 26, 2021 at 4:00 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 14:21
FromStephen Kyne
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
Hi Dave,

Did you add -DUSE_VCPKG=1 to the cmake configure step? You should see something like: 

[cmake] -- VCPKG_TARGET_TRIPLET: x64-windows-csound
[cmake] -- CMAKE_TOOLCHAIN_FILE: E:/dev/github/csound/vcpkg/scripts/buildsystems/vcpkg.cmake

At the top of the configure stage which shows that it's using the correct triplet and toolchain. 

Thanks,
Stephen

From: Csound-developers <CSOUND-DEV@LISTSERV.HEANET.IE> on behalf of Dave Seidel <dave.seidel@GMAIL.COM>
Sent: Thursday 26 August 2021 13:35
To: CSOUND-DEV@LISTSERV.HEANET.IE <CSOUND-DEV@LISTSERV.HEANET.IE>
Subject: Re: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
 
I was able to run the bootstrap script after pulling in the vcpkg submodules, but there was no visible sign of it pulling down any dependencies other than vcpkg.exe.

On Thu, Aug 26, 2021 at 8:33 AM Dave Seidel <dave.seidel@gmail.com> wrote:
Closer, but now (this is during the cmake configuration step):

CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!


On Thu, Aug 26, 2021 at 8:16 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
yes, that's right, no need to clone again if you have already done it.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 12:45, Dave Seidel <dave.seidel@gmail.com> wrote:


I'm running "git pull --recurse-submodules" and it seems to be doing the right thing. Thanks, Victor!

On Thu, Aug 26, 2021 at 4:00 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 15:03
FromDave Seidel
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
Hi Stephen

It may be relevant that I've done builds in here before using the older scheme, Maybe there's something left over that I need to remove? Here's a complete transcript, just ran it again:

C:\Users\dave\Documents\GitHub\csound>cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
-- VCPKG_TARGET_TRIPLET: x64-windows-csound
-- CMAKE_TOOLCHAIN_FILE: C:/Users/dave/Documents/GitHub/csound/vcpkg/scripts/buildsystems/vcpkg.cmake
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19041.
-- Csound directory: C:/Users/dave/Documents/GitHub/csound
-- Csound version: 6.17.0
-- CMake module path: C:/Users/dave/Documents/GitHub/csound/cmake/Modules/
-- Including Custom.cmake file: C:/Users/dave/Documents/GitHub/csound/platform/windows/Custom-vs.cmake
-- LIBRARY INSTALL DIR: C:/Program Files (x86)/Csound/lib64
-- Building with 64-bit floats
-- BUILD_BIN_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
-- BUILD_LIB_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
-- BUILD_PLUGINS_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!
See also "C:/Users/dave/Documents/GitHub/csound/build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/dave/Documents/GitHub/csound/build/CMakeFiles/CMakeError.log".



On Thu, Aug 26, 2021 at 9:21 AM Stephen Kyne <stevek@outlook.ie> wrote:
Hi Dave,

Did you add -DUSE_VCPKG=1 to the cmake configure step? You should see something like: 

[cmake] -- VCPKG_TARGET_TRIPLET: x64-windows-csound
[cmake] -- CMAKE_TOOLCHAIN_FILE: E:/dev/github/csound/vcpkg/scripts/buildsystems/vcpkg.cmake

At the top of the configure stage which shows that it's using the correct triplet and toolchain. 

Thanks,
Stephen

From: Csound-developers <CSOUND-DEV@LISTSERV.HEANET.IE> on behalf of Dave Seidel <dave.seidel@GMAIL.COM>
Sent: Thursday 26 August 2021 13:35
To: CSOUND-DEV@LISTSERV.HEANET.IE <CSOUND-DEV@LISTSERV.HEANET.IE>
Subject: Re: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
 
I was able to run the bootstrap script after pulling in the vcpkg submodules, but there was no visible sign of it pulling down any dependencies other than vcpkg.exe.

On Thu, Aug 26, 2021 at 8:33 AM Dave Seidel <dave.seidel@gmail.com> wrote:
Closer, but now (this is during the cmake configuration step):

CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!


On Thu, Aug 26, 2021 at 8:16 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
yes, that's right, no need to clone again if you have already done it.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 12:45, Dave Seidel <dave.seidel@gmail.com> wrote:


I'm running "git pull --recurse-submodules" and it seems to be doing the right thing. Thanks, Victor!

On Thu, Aug 26, 2021 at 4:00 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 15:05
FromDave Seidel
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
I'm willing to blow this away and get a fresh clone if that's the way to resolve the issue. I may try that later (but right now trying to get work done).

On Thu, Aug 26, 2021 at 10:03 AM Dave Seidel <dave.seidel@gmail.com> wrote:
Hi Stephen

It may be relevant that I've done builds in here before using the older scheme, Maybe there's something left over that I need to remove? Here's a complete transcript, just ran it again:

C:\Users\dave\Documents\GitHub\csound>cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
-- VCPKG_TARGET_TRIPLET: x64-windows-csound
-- CMAKE_TOOLCHAIN_FILE: C:/Users/dave/Documents/GitHub/csound/vcpkg/scripts/buildsystems/vcpkg.cmake
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19041.
-- Csound directory: C:/Users/dave/Documents/GitHub/csound
-- Csound version: 6.17.0
-- CMake module path: C:/Users/dave/Documents/GitHub/csound/cmake/Modules/
-- Including Custom.cmake file: C:/Users/dave/Documents/GitHub/csound/platform/windows/Custom-vs.cmake
-- LIBRARY INSTALL DIR: C:/Program Files (x86)/Csound/lib64
-- Building with 64-bit floats
-- BUILD_BIN_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
-- BUILD_LIB_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
-- BUILD_PLUGINS_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!
See also "C:/Users/dave/Documents/GitHub/csound/build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/dave/Documents/GitHub/csound/build/CMakeFiles/CMakeError.log".



On Thu, Aug 26, 2021 at 9:21 AM Stephen Kyne <stevek@outlook.ie> wrote:
Hi Dave,

Did you add -DUSE_VCPKG=1 to the cmake configure step? You should see something like: 

[cmake] -- VCPKG_TARGET_TRIPLET: x64-windows-csound
[cmake] -- CMAKE_TOOLCHAIN_FILE: E:/dev/github/csound/vcpkg/scripts/buildsystems/vcpkg.cmake

At the top of the configure stage which shows that it's using the correct triplet and toolchain. 

Thanks,
Stephen

From: Csound-developers <CSOUND-DEV@LISTSERV.HEANET.IE> on behalf of Dave Seidel <dave.seidel@GMAIL.COM>
Sent: Thursday 26 August 2021 13:35
To: CSOUND-DEV@LISTSERV.HEANET.IE <CSOUND-DEV@LISTSERV.HEANET.IE>
Subject: Re: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
 
I was able to run the bootstrap script after pulling in the vcpkg submodules, but there was no visible sign of it pulling down any dependencies other than vcpkg.exe.

On Thu, Aug 26, 2021 at 8:33 AM Dave Seidel <dave.seidel@gmail.com> wrote:
Closer, but now (this is during the cmake configuration step):

CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!


On Thu, Aug 26, 2021 at 8:16 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
yes, that's right, no need to clone again if you have already done it.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 12:45, Dave Seidel <dave.seidel@gmail.com> wrote:


I'm running "git pull --recurse-submodules" and it seems to be doing the right thing. Thanks, Victor!

On Thu, Aug 26, 2021 at 4:00 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 15:58
FromStephen Kyne
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
Hmm that's kind of strange. It looks mostly correct but it's missing the manifest install part. It should look like this:

PS E:\dev\github\csound> cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
-- Building for: Visual Studio 16 2019
-- VCPKG_TARGET_TRIPLET: x64-windows-csound
-- CMAKE_TOOLCHAIN_FILE: E:/dev/github/csound/vcpkg/scripts/buildsystems/vcpkg.cmake
-- Running vcpkg install
Detecting compiler hash for triplet x64-windows-csound...
Detecting compiler hash for triplet x64-windows...
The following packages will be built and installed:
    dirent[core]:x64-windows-csound -> 1.23.2 -- E:\dev\github\csound\vcpkg\buildtrees\versioning\versions\dirent\b1ac2911c352ec0e8485c4cb49f544d0f05d749b
... Remaining packages ...
-- Running vcpkg install - done
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19043.
-- The C compiler identification is MSVC 19.29.30133.0
-- The CXX compiler identification is MSVC 19.29.30133.0

I'd make sure that no other VCPKG is on the path in case it is there. That could be a reason. 

Stephen
From: Csound-developers <CSOUND-DEV@LISTSERV.HEANET.IE> on behalf of Dave Seidel <dave.seidel@GMAIL.COM>
Sent: Thursday 26 August 2021 15:05
To: CSOUND-DEV@LISTSERV.HEANET.IE <CSOUND-DEV@LISTSERV.HEANET.IE>
Subject: Re: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
 
I'm willing to blow this away and get a fresh clone if that's the way to resolve the issue. I may try that later (but right now trying to get work done).

On Thu, Aug 26, 2021 at 10:03 AM Dave Seidel <dave.seidel@gmail.com> wrote:
Hi Stephen

It may be relevant that I've done builds in here before using the older scheme, Maybe there's something left over that I need to remove? Here's a complete transcript, just ran it again:

C:\Users\dave\Documents\GitHub\csound>cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
-- VCPKG_TARGET_TRIPLET: x64-windows-csound
-- CMAKE_TOOLCHAIN_FILE: C:/Users/dave/Documents/GitHub/csound/vcpkg/scripts/buildsystems/vcpkg.cmake
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19041.
-- Csound directory: C:/Users/dave/Documents/GitHub/csound
-- Csound version: 6.17.0
-- CMake module path: C:/Users/dave/Documents/GitHub/csound/cmake/Modules/
-- Including Custom.cmake file: C:/Users/dave/Documents/GitHub/csound/platform/windows/Custom-vs.cmake
-- LIBRARY INSTALL DIR: C:/Program Files (x86)/Csound/lib64
-- Building with 64-bit floats
-- BUILD_BIN_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
-- BUILD_LIB_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
-- BUILD_PLUGINS_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!
See also "C:/Users/dave/Documents/GitHub/csound/build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/dave/Documents/GitHub/csound/build/CMakeFiles/CMakeError.log".



On Thu, Aug 26, 2021 at 9:21 AM Stephen Kyne <stevek@outlook.ie> wrote:
Hi Dave,

Did you add -DUSE_VCPKG=1 to the cmake configure step? You should see something like: 

[cmake] -- VCPKG_TARGET_TRIPLET: x64-windows-csound
[cmake] -- CMAKE_TOOLCHAIN_FILE: E:/dev/github/csound/vcpkg/scripts/buildsystems/vcpkg.cmake

At the top of the configure stage which shows that it's using the correct triplet and toolchain. 

Thanks,
Stephen

From: Csound-developers <CSOUND-DEV@LISTSERV.HEANET.IE> on behalf of Dave Seidel <dave.seidel@GMAIL.COM>
Sent: Thursday 26 August 2021 13:35
To: CSOUND-DEV@LISTSERV.HEANET.IE <CSOUND-DEV@LISTSERV.HEANET.IE>
Subject: Re: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
 
I was able to run the bootstrap script after pulling in the vcpkg submodules, but there was no visible sign of it pulling down any dependencies other than vcpkg.exe.

On Thu, Aug 26, 2021 at 8:33 AM Dave Seidel <dave.seidel@gmail.com> wrote:
Closer, but now (this is during the cmake configuration step):

CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!


On Thu, Aug 26, 2021 at 8:16 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
yes, that's right, no need to clone again if you have already done it.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 12:45, Dave Seidel <dave.seidel@gmail.com> wrote:


I'm running "git pull --recurse-submodules" and it seems to be doing the right thing. Thanks, Victor!

On Thu, Aug 26, 2021 at 4:00 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen

Date2021-08-26 22:25
FromDave Seidel
SubjectRe: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
I did a fresh clone and it looks like it's doing the right thing. Clearly there was some leftover gunk from the previous regime. Looking forward to working with this improved system!

On Thu, Aug 26, 2021 at 10:58 AM Stephen Kyne <stevek@outlook.ie> wrote:
Hmm that's kind of strange. It looks mostly correct but it's missing the manifest install part. It should look like this:

PS E:\dev\github\csound> cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
-- Building for: Visual Studio 16 2019
-- VCPKG_TARGET_TRIPLET: x64-windows-csound
-- CMAKE_TOOLCHAIN_FILE: E:/dev/github/csound/vcpkg/scripts/buildsystems/vcpkg.cmake
-- Running vcpkg install
Detecting compiler hash for triplet x64-windows-csound...
Detecting compiler hash for triplet x64-windows...
The following packages will be built and installed:
    dirent[core]:x64-windows-csound -> 1.23.2 -- E:\dev\github\csound\vcpkg\buildtrees\versioning\versions\dirent\b1ac2911c352ec0e8485c4cb49f544d0f05d749b
... Remaining packages ...
-- Running vcpkg install - done
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19043.
-- The C compiler identification is MSVC 19.29.30133.0
-- The CXX compiler identification is MSVC 19.29.30133.0

I'd make sure that no other VCPKG is on the path in case it is there. That could be a reason. 

Stephen
From: Csound-developers <CSOUND-DEV@LISTSERV.HEANET.IE> on behalf of Dave Seidel <dave.seidel@GMAIL.COM>
Sent: Thursday 26 August 2021 15:05
To: CSOUND-DEV@LISTSERV.HEANET.IE <CSOUND-DEV@LISTSERV.HEANET.IE>
Subject: Re: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
 
I'm willing to blow this away and get a fresh clone if that's the way to resolve the issue. I may try that later (but right now trying to get work done).

On Thu, Aug 26, 2021 at 10:03 AM Dave Seidel <dave.seidel@gmail.com> wrote:
Hi Stephen

It may be relevant that I've done builds in here before using the older scheme, Maybe there's something left over that I need to remove? Here's a complete transcript, just ran it again:

C:\Users\dave\Documents\GitHub\csound>cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
-- VCPKG_TARGET_TRIPLET: x64-windows-csound
-- CMAKE_TOOLCHAIN_FILE: C:/Users/dave/Documents/GitHub/csound/vcpkg/scripts/buildsystems/vcpkg.cmake
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19041.
-- Csound directory: C:/Users/dave/Documents/GitHub/csound
-- Csound version: 6.17.0
-- CMake module path: C:/Users/dave/Documents/GitHub/csound/cmake/Modules/
-- Including Custom.cmake file: C:/Users/dave/Documents/GitHub/csound/platform/windows/Custom-vs.cmake
-- LIBRARY INSTALL DIR: C:/Program Files (x86)/Csound/lib64
-- Building with 64-bit floats
-- BUILD_BIN_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
-- BUILD_LIB_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
-- BUILD_PLUGINS_DIR set to C:/Users/dave/Documents/GitHub/csound/build.
CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!
See also "C:/Users/dave/Documents/GitHub/csound/build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/dave/Documents/GitHub/csound/build/CMakeFiles/CMakeError.log".



On Thu, Aug 26, 2021 at 9:21 AM Stephen Kyne <stevek@outlook.ie> wrote:
Hi Dave,

Did you add -DUSE_VCPKG=1 to the cmake configure step? You should see something like: 

[cmake] -- VCPKG_TARGET_TRIPLET: x64-windows-csound
[cmake] -- CMAKE_TOOLCHAIN_FILE: E:/dev/github/csound/vcpkg/scripts/buildsystems/vcpkg.cmake

At the top of the configure stage which shows that it's using the correct triplet and toolchain. 

Thanks,
Stephen

From: Csound-developers <CSOUND-DEV@LISTSERV.HEANET.IE> on behalf of Dave Seidel <dave.seidel@GMAIL.COM>
Sent: Thursday 26 August 2021 13:35
To: CSOUND-DEV@LISTSERV.HEANET.IE <CSOUND-DEV@LISTSERV.HEANET.IE>
Subject: Re: [Csnd-dev] [EXTERNAL] Re: [Csnd-dev] CI Build Changes / Windows changes
 
I was able to run the bootstrap script after pulling in the vcpkg submodules, but there was no visible sign of it pulling down any dependencies other than vcpkg.exe.

On Thu, Aug 26, 2021 at 8:33 AM Dave Seidel <dave.seidel@gmail.com> wrote:
Closer, but now (this is during the cmake configuration step):

CMake Error at CMakeLists.txt:454 (find_package):
  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!


On Thu, Aug 26, 2021 at 8:16 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
yes, that's right, no need to clone again if you have already done it.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 12:45, Dave Seidel <dave.seidel@gmail.com> wrote:


I'm running "git pull --recurse-submodules" and it seems to be doing the right thing. Thanks, Victor!

On Thu, Aug 26, 2021 at 4:00 AM Victor Lazzarini <Victor.Lazzarini@mu.ie> wrote:
you can clone the repo by using --recurse-submodules.

Prof. Victor Lazzarini
Maynooth University
Ireland

On 26 Aug 2021, at 02:06, Dave Seidel <dave.seidel@gmail.com> wrote:



*Warning*

This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.

I'm trying to follow these steps in Windows. I have pulled the latest develop. There is nothing in my vcpkg subdirectory and the file bootstrap-bcpkg.bat exists nowhere in my source tree.

- Dave.

On Wed, Aug 25, 2021 at 6:36 PM Stephen Kyne <stevek@outlook.ie> wrote:
Hi all,

A PR of mine has been merged today which does the following: 
  1. Uses Github Actions to build the Windows binaries
  2. VCPKG now uses a manifest instead of manually installing dependencies via shell script.
  3. VCPKG now uses binary caching which stores output as Nuget packages in our Csound org for Github. These will be updated any time the version changes. This speeds up builds considerably.
  4. VCPKG dependencies are versioned now so will be consistent across builds unless we manually update them. 
  5. VCPKG is now a submodule of our repo as per the recommendations from the authors. Anyone who is using the Windows build will have to make sure they pull the submodule as well.
  6. Finally, the MSVC folder and associated scripts are no longer needed anymore. 
The build process for local Windows builds should be:
  1. Ensure external dependencies are satisfied
    1. choco install swig winflexbison
  2. Ensure VCPKG submodule is pulled down and bootstrapped
    1. .\vcpkg\bootstrap-vcpkg.bat
  3. Configure cmake 
    1. cmake -B build -S . -DUSE_VCPKG=1 -DCUSTOM_CMAKE="./platform/windows/Custom-vs.cmake"
    2. The custom cmake isn't needed but if you want to replicate the CI then leave it in. The import part is the new "USE_VCPKG" flag which will install dependencies.
  4. Build csound
    1. cmake --build build --config Release|Debug
So, it should be a lot more streamlined now and not require any additional script files. Any changes to the VCPKG manifest will be picked up by running the cmake configure again. 

All VCPKG package builds are cached locally so you can have multiple VCPKG folders and they will reuse this cache instead of residing in the folder like before.

Appveyor is now no longer needed as all our CI is on Github now. Associated files will be removed shortly.

I'd like to expand this work to Linux/MacOS once VCPKG support is working fully. 

Thanks,
Stephen