Csound Csound-dev Csound-tekno Search About

[Csnd] [semi-OT] "synthesizing" graphics from audio

Date2025-04-23 00:07
From"Jeanette C."
Subject[Csnd] [semi-OT] "synthesizing" graphics from audio
Hey hey,
just today someone asked, if I couldn't add a little more interesting image 
content to my music on Youtube. I had been thinking about that before.

Does anyone here know of a text programmable tool that can create abstract 
images from music? Based on some kind of audio analysis perhaps? Can Csound be 
used to achieve that? I was thinking of abstract geometric shapes in varying 
colours. It's been a while since I've seen a desktop audio player app on an 
actual screen, but some of them used to have similar features...

If you can think of something, it could be a Python library, that can load 
audio and produce and save moving images, I'd be grateful.x!

Thanks and best wishes,

Jeanette

-- 
  * Website: http://juliencoder.de - for summer is a state of sound
  * Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
  * Audiobombs: https://www.audiobombs.com/users/jeanette_c
  * GitHub: https://github.com/jeanette-c

I saw your smile
Stay with me a while <3
(Britney Spears)

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2025-04-23 00:37
FromRichard Knight
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
Hi,

I don't know any out-of-the-box solutions but you can use PIL to draw 
graphics as frames and then string those together into video using 
ffmpeg or similar. I used this synchronised with Csound for a couple of 
experimental videos - https://www.youtube.com/watch?v=BTTvcESpaq4 ; 
https://www.youtube.com/watch?v=qW-Igs75fEE

The image opcodes in Csound 
(https://csound.com/docs/manual/imagecreate.html) could be used 
similarly to create individual frames, but are very low level - not used 
them much.

moviepy (https://zulko.github.io/moviepy/) is a useful python library 
for creating more detailed videos as well but more geared towards 
manipulating existing video.

Also I've created an experimental plugin for SDL graphics within Csound 
- this outputs to the screen rather than a file, but can be quite easily 
used for basic shapes etc.

With all these, there's work to do in mapping audio features to visual 
features though..


On 2025-04-23 00:07, Jeanette C. wrote:
> Hey hey,
> just today someone asked, if I couldn't add a little more interesting
> image content to my music on Youtube. I had been thinking about that
> before.
> 
> Does anyone here know of a text programmable tool that can create
> abstract images from music? Based on some kind of audio analysis
> perhaps? Can Csound be used to achieve that? I was thinking of
> abstract geometric shapes in varying colours. It's been a while since
> I've seen a desktop audio player app on an actual screen, but some of
> them used to have similar features...
> 
> If you can think of something, it could be a Python library, that can
> load audio and produce and save moving images, I'd be grateful.x!
> 
> Thanks and best wishes,
> 
> Jeanette
> 
> --
>  * Website: http://juliencoder.de - for summer is a state of sound
>  * Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
>  * Audiobombs: https://www.audiobombs.com/users/jeanette_c
>  * GitHub: https://github.com/jeanette-c
> 
> I saw your smile
> Stay with me a while <3
> (Britney Spears)
> 
> Csound mailing list
> Csound@listserv.heanet.ie
> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
> Send bugs reports to
>        https://github.com/csound/csound/issues
> Discussions of bugs and features can be posted here

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2025-04-23 13:46
FromSi Mills
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
Hi Jeanette

A few web-based options:

You could explore p5.Csound (https://rorywalsh.github.io/p5.Csound/#/) so you can run your Csound patches and have bi-directional communication with p5.js

Another Csound wasm option to explore is CABLES.gl if you fancy something a bit more visual programming like Pure Data.

Hydrasynth is really cool, though I’ve yet to see a direct integration, but I think something could be done. You can however send midi events to have visuals react :) 

Other than that:

Openframeworks (bit more involved), Touchdesigner, Game engines: Unity, Unreal  — but a bit more involved

+++

I think web might be the route to faster results though. Simply sending MIDI/OSC events to said options, or in some cases audio analysis

Cheers, S

On 23 Apr 2025, at 00:37, Richard Knight <richard@1BPM.NET> wrote:

Hi,

I don't know any out-of-the-box solutions but you can use PIL to draw graphics as frames and then string those together into video using ffmpeg or similar. I used this synchronised with Csound for a couple of experimental videos - https://www.youtube.com/watch?v=BTTvcESpaq4 ; https://www.youtube.com/watch?v=qW-Igs75fEE

The image opcodes in Csound (https://csound.com/docs/manual/imagecreate.html) could be used similarly to create individual frames, but are very low level - not used them much.

moviepy (https://zulko.github.io/moviepy/) is a useful python library for creating more detailed videos as well but more geared towards manipulating existing video.

Also I've created an experimental plugin for SDL graphics within Csound - this outputs to the screen rather than a file, but can be quite easily used for basic shapes etc.

With all these, there's work to do in mapping audio features to visual features though..


On 2025-04-23 00:07, Jeanette C. wrote:
Hey hey,
just today someone asked, if I couldn't add a little more interesting
image content to my music on Youtube. I had been thinking about that
before.
Does anyone here know of a text programmable tool that can create
abstract images from music? Based on some kind of audio analysis
perhaps? Can Csound be used to achieve that? I was thinking of
abstract geometric shapes in varying colours. It's been a while since
I've seen a desktop audio player app on an actual screen, but some of
them used to have similar features...
If you can think of something, it could be a Python library, that can
load audio and produce and save moving images, I'd be grateful.x!
Thanks and best wishes,
Jeanette
--
* Website: http://juliencoder.de - for summer is a state of sound
* Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
* Audiobombs: https://www.audiobombs.com/users/jeanette_c
* GitHub: https://github.com/jeanette-c
I saw your smile
Stay with me a while <3
(Britney Spears)
Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
      https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
      https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here


Date2025-04-23 13:48
FromSi Mills
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
Sorry — Hydrasynth  = Hydra (https://hydra.ojack.xyz/)

The former is an actual synth

On 23 Apr 2025, at 13:46, Si Mills <smills@rootsix.net> wrote:

Hi Jeanette

A few web-based options:

You could explore p5.Csound (https://rorywalsh.github.io/p5.Csound/#/) so you can run your Csound patches and have bi-directional communication with p5.js

Another Csound wasm option to explore is CABLES.gl if you fancy something a bit more visual programming like Pure Data.

Hydrasynth is really cool, though I’ve yet to see a direct integration, but I think something could be done. You can however send midi events to have visuals react :) 

Other than that:

Openframeworks (bit more involved), Touchdesigner, Game engines: Unity, Unreal  — but a bit more involved

+++

I think web might be the route to faster results though. Simply sending MIDI/OSC events to said options, or in some cases audio analysis

Cheers, S

On 23 Apr 2025, at 00:37, Richard Knight <richard@1BPM.NET> wrote:

Hi,

I don't know any out-of-the-box solutions but you can use PIL to draw graphics as frames and then string those together into video using ffmpeg or similar. I used this synchronised with Csound for a couple of experimental videos - https://www.youtube.com/watch?v=BTTvcESpaq4 ; https://www.youtube.com/watch?v=qW-Igs75fEE

The image opcodes in Csound (https://csound.com/docs/manual/imagecreate.html) could be used similarly to create individual frames, but are very low level - not used them much.

moviepy (https://zulko.github.io/moviepy/) is a useful python library for creating more detailed videos as well but more geared towards manipulating existing video.

Also I've created an experimental plugin for SDL graphics within Csound - this outputs to the screen rather than a file, but can be quite easily used for basic shapes etc.

With all these, there's work to do in mapping audio features to visual features though..


On 2025-04-23 00:07, Jeanette C. wrote:
Hey hey,
just today someone asked, if I couldn't add a little more interesting
image content to my music on Youtube. I had been thinking about that
before.
Does anyone here know of a text programmable tool that can create
abstract images from music? Based on some kind of audio analysis
perhaps? Can Csound be used to achieve that? I was thinking of
abstract geometric shapes in varying colours. It's been a while since
I've seen a desktop audio player app on an actual screen, but some of
them used to have similar features...
If you can think of something, it could be a Python library, that can
load audio and produce and save moving images, I'd be grateful.x!
Thanks and best wishes,
Jeanette
--
* Website: http://juliencoder.de - for summer is a state of sound
* Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
* Audiobombs: https://www.audiobombs.com/users/jeanette_c
* GitHub: https://github.com/jeanette-c
I saw your smile
Stay with me a while <3
(Britney Spears)
Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
      https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
      https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here


Date2025-04-23 13:53
From"Jeanette C."
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
Hi Richard and S.,
thanks both of your for your suggestions.

@Richard: I've taken a look at PIL (Python Pillow), which comes closest. 
Though I think with that particular interface, I might almost return to 
POV-Ray and work that way. Perhaps not that reactive in the end. :)

@smills: sorry all those are out. Can't use that GUI stuff, which is why I 
particularly asked for text-programmable stuff. I should have made myself 
clearer, in saying that I need something that can be developed and run purely 
from the commandline.

I love the Hydrasynth, though haven't come across the other, even though I'm 
aware of a few video synthesizers, mostly in hardware, they get shown at synth 
events now, which is great in itself.

Best wishes,

Jeanette

-- 
  * Website: http://juliencoder.de - for summer is a state of sound
  * Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
  * Audiobombs: https://www.audiobombs.com/users/jeanette_c
  * GitHub: https://github.com/jeanette-c

I believe
We all have one true love
Somewhere in this world <3
(Britney Spears)

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2025-04-23 14:09
FromPhilipp von Neumann <0000119f78f3a4f9-dmarc-request@LISTSERV.HEANET.IE>
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
I did not know that cables has a csound implementation yet. Are there some example patches?

Am 23.04.2025 um 14:46 schrieb Si Mills <smills@rootsix.net>:

Another Csound wasm option to explore is CABLES.gl if you fancy something a bit more visual programming like Pure Data.

Date2025-04-23 14:40
FromRichard Knight
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
Hydrasynth does look quite cool.

I do actually love POV-ray - in a nostalgic and perhaps slightly 
masochistic way!
Incidentally there's a Python interface for it - 
https://github.com/Zulko/vapory - which may be of use.

On 2025-04-23 13:53, Jeanette C. wrote:
> Hi Richard and S.,
> thanks both of your for your suggestions.
> 
> @Richard: I've taken a look at PIL (Python Pillow), which comes
> closest. Though I think with that particular interface, I might almost
> return to POV-Ray and work that way. Perhaps not that reactive in the
> end. :)
> 
> @smills: sorry all those are out. Can't use that GUI stuff, which is
> why I particularly asked for text-programmable stuff. I should have
> made myself clearer, in saying that I need something that can be
> developed and run purely from the commandline.
> 
> I love the Hydrasynth, though haven't come across the other, even
> though I'm aware of a few video synthesizers, mostly in hardware, they
> get shown at synth events now, which is great in itself.
> 
> Best wishes,
> 
> Jeanette
> 
> --
>  * Website: http://juliencoder.de - for summer is a state of sound
>  * Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
>  * Audiobombs: https://www.audiobombs.com/users/jeanette_c
>  * GitHub: https://github.com/jeanette-c
> 
> I believe
> We all have one true love
> Somewhere in this world <3
> (Britney Spears)
> 
> Csound mailing list
> Csound@listserv.heanet.ie
> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
> Send bugs reports to
>        https://github.com/csound/csound/issues
> Discussions of bugs and features can be posted here

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2025-04-23 14:57
Fromluis antunes pena <00000fd5d0d20dbf-dmarc-request@LISTSERV.HEANET.IE>
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
Attachmentsvisualmusic.zip  
Hi Jeanette,
you could also try processing + csound via OSC. I've attached a simple example that should work on the fly.
Rory Walsh made some years ago an integration of processing and csound.
Hope it helps.
Best,
Luís


On 23.04.25 01:07, Jeanette C. wrote:
Hey hey,
just today someone asked, if I couldn't add a little more interesting image content to my music on Youtube. I had been thinking about that before.

Does anyone here know of a text programmable tool that can create abstract images from music? Based on some kind of audio analysis perhaps? Can Csound be used to achieve that? I was thinking of abstract geometric shapes in varying colours. It's been a while since I've seen a desktop audio player app on an actual screen, but some of them used to have similar features...

If you can think of something, it could be a Python library, that can load audio and produce and save moving images, I'd be grateful.x!

Thanks and best wishes,

Jeanette


Date2025-04-23 15:10
From"Jeanette C."
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
Hi Richard,
thanks for the Python POV-Ray library. that might really be the way forwards. 
So I can combiine two things that I know. I see why you might remember it with 
some frustration. But if you want to do some graphics design while being blind 
- or as close as makes no difference - POV-Ray is absolutely ACE! :)

Best wishes,

Jeanette

-- 
  * Website: http://juliencoder.de - for summer is a state of sound
  * Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
  * Audiobombs: https://www.audiobombs.com/users/jeanette_c
  * GitHub: https://github.com/jeanette-c

I believe
We all have one true love
Somewhere in this world <3
(Britney Spears)

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2025-04-23 15:14
FromRichard Knight
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
Ah great, I am glad that may be helpful.
And yes, I can absolutely understand, while 3D modelling has moved to be 
quite visual in design, for text scene description and scripting, 
POV-Ray is fantastic!


On 2025-04-23 15:10, Jeanette C. wrote:
> Hi Richard,
> thanks for the Python POV-Ray library. that might really be the way
> forwards. So I can combiine two things that I know. I see why you
> might remember it with some frustration. But if you want to do some
> graphics design while being blind - or as close as makes no difference
> - POV-Ray is absolutely ACE! :)
> 
> Best wishes,
> 
> Jeanette
> 
> --
>  * Website: http://juliencoder.de - for summer is a state of sound
>  * Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
>  * Audiobombs: https://www.audiobombs.com/users/jeanette_c
>  * GitHub: https://github.com/jeanette-c
> 
> I believe
> We all have one true love
> Somewhere in this world <3
> (Britney Spears)
> 
> Csound mailing list
> Csound@listserv.heanet.ie
> https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
> Send bugs reports to
>        https://github.com/csound/csound/issues
> Discussions of bugs and features can be posted here

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2025-04-23 15:18
From"Jeanette C."
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
Hi Luis,
thanks for the example. Do you know how you could run Processing on the 
commandline? The OSC control is a nice feature.

Best wishes,

Jeanette

-- 
  * Website: http://juliencoder.de - for summer is a state of sound
  * Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
  * Audiobombs: https://www.audiobombs.com/users/jeanette_c
  * GitHub: https://github.com/jeanette-c

I believe
We all have one true love
Somewhere in this world <3
(Britney Spears)

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2025-04-23 15:41
FromSi Mills
SubjectRe: [Csnd] [semi-OT] "synthesizing" graphics from audio
Yes of course, sorry https://cables.gl/p/tjO3wp

It took some finagling - mainly importing the csound js 

I just need to abstract out the setup to make the op less tied to the patch

On 23 Apr 2025, at 14:09, Philipp von Neumann <0000119f78f3a4f9-dmarc-request@LISTSERV.HEANET.IE> wrote:

I did not know that cables has a csound implementation yet. Are there some example patches?

Am 23.04.2025 um 14:46 schrieb Si Mills <smills@rootsix.net>:

Another Csound wasm option to explore is CABLES.gl if you fancy something a bit more visual programming like Pure Data.
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here