Csound Csound-dev Csound-tekno Search About

Re: a simple perceptron

Date1998-07-01 19:22
FromPedro Batista
SubjectRe: a simple perceptron
Matt Southall wrote

>This is something I've been interested in for a while.  As you point out,
>using a perceptron for each individual sample is a bit of a pain.  One
>thing you might like to look at is recurrent networks.

yes, that was my first thought too. But recursive nets can hold AT MOST the 
past six values, which is fine if youre training a 3 binary digit sequence, 
but its impracticable for audio at 44.1 kHz (some reduction of the input 
data can be performed, but I dont believe you can reduce audio data to the 
point where 6 past values would be enough...)
But if you consider applying this line of thought to _midi_ data, instead of 
sample data, then its a whole different scenario...

>I've just had a paper accepted for 'Simulated Adaptive Behaviour 98' on
>the subject of 'Temporal Pattern Learning in a Spiking Neural Network',

is it online? (would like to see it)

>and I'd be interested to discuss the issues involved (privately, it's a
>little off topic) with anyone else who has attempted anything along these
>lines

sure, I'd like that

>I think the first step in improving your network
>would be to move to a freq/time domain.

actually thats what I'm doing now :) this example was primarly meant to 
assert how far could I go in computational power using csound; no practical 
use whatsoever

concerning csound, some leads I'm working on include combining delay units 
to give the net some recursiveness (several points of the sample are 
presented to the net at each instant, so the net aint recursive, but the 
input is presented in a recurrent form); other hipotesis are using the 
csound utilities to extract audio info that would be more suitable to be 
presented to the net; maybe pvanal can be used for that, but I'm still 
studying the implications.
Of course my next attempts will use back-prop and hidden layers, but I must 
reduce the presented audio info so that I dont have to deal with such big 
arrays

One thing I know, is that this will still hold many challanges for my future 
csound programming

pedro

Date1998-07-02 22:23
FromEli Brandt
SubjectRe: a simple perceptron
Pedro Batista wrote:
> yes, that was my first thought too. But recursive nets can hold AT MOST the 
> past six values, [...]

Why's that?  A bucket-brigade delay is a possible configuration of a
one-layer recurrent net.  Admittedly a rather specific one -- when I
played with recurrent nets for signal generation (using annealing and
Hinton's recurrent backprop; had better luck with the former) I had a
hard time training them to set enough weights to zero.  A quadrature
osc, for example, was hard: a net can do one sine osc just fine, but a
little interleakage entrains the two oscs together.

(Getting rather unCsoundy; transpose to music-dsp?)

-- 
     Eli Brandt  |  eli+@cs.cmu.edu  |  http://www.cs.cmu.edu/~eli/