[Csnd] audiovisual synthesis via programming
Date | 2013-03-15 15:16 |
From | Adam Puckett |
Subject | [Csnd] audiovisual synthesis via programming |
I'm interested in putting videos on YouTube (as I mentioned in a previous thread a long time ago). I know how to work with the audio end of things, and I am trying to learn (using FFMpeg sources) how to manipulate the video side, and so far I'm not making as much progress as I'd hoped, unfortunately. Is there something simple, like STK is for audio, for learning video programming? Or a language like Csound, for drawing and animating and saving it to files? |
Date | 2013-03-15 16:06 |
From | Justin Smith |
Subject | Re: [Csnd] audiovisual synthesis via programming |
I have in the past used mplayer / mencoder to break a video into individual images, then used the gimp in script mode to edit the images programmatically. At the time the gimp included a guile scheme interpreter that could access all program functionality. This may have been replaced by python, but the functionality would be similar. I had started out using the csound image processing opcodes, but it was easier to write guile code that interpreted lines in the same format csound uses and generates images (and generating both the csound score lines and the guile/gimp score lines from the same program).
On Fri, Mar 15, 2013 at 8:16 AM, Adam Puckett <adotsdothmusic@gmail.com> wrote: I'm interested in putting videos on YouTube (as I mentioned in a |
Date | 2013-03-15 16:08 |
From | Justin Smith |
Subject | Re: [Csnd] audiovisual synthesis via programming |
Also, the best way to do this is probably the processing environment. On Fri, Mar 15, 2013 at 9:06 AM, Justin Smith <noisesmith@gmail.com> wrote:
|
Date | 2013-03-15 16:15 |
From | Adam Puckett |
Subject | Re: [Csnd] audiovisual synthesis via programming |
I cannot use processing as it is inaccessible to my screen reader. The ideal environment would be a text editor. On 3/15/13, Justin Smith |
Date | 2013-03-15 16:29 |
From | Rory Walsh |
Subject | Re: [Csnd] audiovisual synthesis via programming |
Hi Adam. you don't need to use Processing's IDE, you can use another IDE, netbeans or eclipse?. I would have thought the Processing IDE would be accessible to screen reading software, it's just a plain old java app? On 15 March 2013 16:15, Adam Puckett |
Date | 2013-03-15 16:32 |
From | Adam Puckett |
Subject | Re: [Csnd] audiovisual synthesis via programming |
Java apps aren't accessible. So how do I get started with just a text editor and Processing? On 3/15/13, Rory Walsh |
Date | 2013-03-15 16:34 |
From | Justin Smith |
Subject | Re: [Csnd] audiovisual synthesis via programming |
Processing supposedly dropped support for using external editors (so they claim, at least). Also there is no documented way to run a sketch outside the processing app that I could find.
If using the processing UI is a dealbreaker, then gimp works just find headless via scheme or python (and for obvious reasons is much more powerful in this usage than the normal GUI, if less discoverable).
On Fri, Mar 15, 2013 at 9:32 AM, Adam Puckett <adotsdothmusic@gmail.com> wrote: Java apps aren't accessible. So how do I get started with just a text |
Date | 2013-03-15 16:34 |
From | Rory Walsh |
Subject | Re: [Csnd] audiovisual synthesis via programming |
If java apps aren't accessible then you could have a problem. Let me take a quick look... On 15 March 2013 16:32, Adam Puckett |
Date | 2013-03-15 16:38 |
From | Rory Walsh |
Subject | Re: [Csnd] audiovisual synthesis via programming |
That's a pity. Might any of the online interfaces work? http://sketch.processing.org/ http://sketchpad.cc On 15 March 2013 16:34, Justin Smith |
Date | 2013-03-15 16:54 |
From | Adam Puckett |
Subject | Re: [Csnd] audiovisual synthesis via programming |
Unfortunately they don't work. I'll try GIMP. On 3/15/13, Rory Walsh |
Date | 2013-03-15 16:58 |
From | Justin Smith |
Subject | Re: [Csnd] audiovisual synthesis via programming |
Another option is the Imagemagick tools, which unlike the gimp operating as a shell, are commands that run, do some modification, output a file, then exit. I think they lack much of GIMP's functionality, but they can do quite a bit.
On Fri, Mar 15, 2013 at 9:54 AM, Adam Puckett <adotsdothmusic@gmail.com> wrote: Unfortunately they don't work. I'll try GIMP. |
Date | 2013-03-15 17:33 |
From | Adam Puckett |
Subject | Re: [Csnd] audiovisual synthesis via programming |
Can it save video and audio inside a container? On 3/15/13, Justin Smith |
Date | 2013-03-15 17:39 |
From | Justin Smith |
Subject | Re: [Csnd] audiovisual synthesis via programming |
I used mplayer / mencoder for the tasks of breaking a movie into images / audio, then reconstituting afterward. That said, there are gimp plugins for working with video, but I have not tried them, because mplayer / mencoder is quite mature and reliable from the command line. The downside: massive disk usage for all the separate video frames. The upside: improved quality, as you are not re-encoding to the lossy video codec at each processing stage (imagine working with audio and saving everything as mp3 at each intermediate stage).
On Fri, Mar 15, 2013 at 10:33 AM, Adam Puckett <adotsdothmusic@gmail.com> wrote: Can it save video and audio inside a container? |
Date | 2013-03-15 17:44 |
From | Adam Puckett |
Subject | Re: [Csnd] audiovisual synthesis via programming |
So what I would do would basically be something like this, right?: Do whatever audio processing I need. Script the images with GIMP (frame by frame, or however you do it). Use mencoder to encode the images. Am I leaving anything out? On 3/15/13, Justin Smith |
Date | 2013-03-15 17:47 |
From | Justin Smith |
Subject | Re: [Csnd] audiovisual synthesis via programming |
that is pretty much it: mplayer to break up the movie into frames / audio csound to manipulate the audio in parallel with gimp manipulating the images (the images will be in files named after the frame number 000000.png 000001.png 000002.png etc. so you can generate the image names in a for loop, and calculate frame number based on time if you know the fps)
mencoder to bundle the outputs all together On Fri, Mar 15, 2013 at 10:44 AM, Adam Puckett <adotsdothmusic@gmail.com> wrote: So what I would do would basically be something like this, right?: |
Date | 2013-03-15 17:53 |
From | Adam Puckett |
Subject | Re: [Csnd] audiovisual synthesis via programming |
Ok. But I won't be doing any Roger Rabbit type stuff (yet). Which means if I'm generating the images I'll name them 0000000.png 0000000n.png etc? On 3/15/13, Justin Smith |
Date | 2013-03-15 18:00 |
From | Justin Smith |
Subject | Re: [Csnd] audiovisual synthesis via programming |
The image names are automatically generated by mplayer when you use it to split an input video into frames. Since you are not using input video, all you need to know is that mencoder simply gets the images as a glob in the command line argument, so you want to make sure that they collate in the right order when the glob is expanded.
this is the example from the mencoder manual: mencoder "mf://*.jpg" -mf fps=25 -o output.avi -ovc lavc -lavcopts vcodec=mpeg4 "mf://*.jpg" is expanded by your shell to all the files ending in .jpg in the current directory, and you simply need to name them such that the shell puts them in the right order. though most algorithmically generated images, I recommend encoding to png rather than jpg. Another trick is that if you frequently redisplay the same frame, you can make symbolic links rather than duplicate frames (saving on processing time and disk usage).
On Fri, Mar 15, 2013 at 10:53 AM, Adam Puckett <adotsdothmusic@gmail.com> wrote: Ok. But I won't be doing any Roger Rabbit type stuff (yet). Which |
Date | 2013-03-15 18:17 |
From | Adam Puckett |
Subject | Re: [Csnd] audiovisual synthesis via programming |
Am I correct in assuming the scripting can be done outside of an IDE? On 3/15/13, Justin Smith |
Date | 2013-03-15 19:05 |
From | Justin Smith |
Subject | Re: [Csnd] audiovisual synthesis via programming |
The scripting is either a scheme or python file that is presented as a command line argument to GIMP, or input to GIMP in console mode, neither of which uses an IDE or GUI in any way. On Fri, Mar 15, 2013 at 11:17 AM, Adam Puckett <adotsdothmusic@gmail.com> wrote: Am I correct in assuming the scripting can be done outside of an IDE? |