|
here's a little coding example, a very rudimentary atempt at using neural
processing based dsp
This is a very simple neural net, that learns audio patterns, and can
display memory and recognition when presented with learned patterns (even if
the input data is incomplete), or simply respond with similarity to unknown
audio data.
You train the net by presenting two audio files, A and B, respectivelly at
the input and output of the net; this will associate a pattern at least
similar to A, with the corresponding response B. You can have a number of
these pairs, three being a practical maximum, before this particular net
begins forgeting old patterns
To code the audio in net data, the rudimentary approach of calculating the
16-bit value of the sample and using binary neurons was used. This is fine
for visual patterns, but I want to use a more audio oriented mapping, using
the basic audio info like freq and amp (maybe with the aid of pvoc?), and
continuous neurons, instead of binary ones.
Using more advanced neural nets, dsp could be performed. For example,
presenting samples of unfiltered and filtered waves, respectively to the
input and output of the net, it could theoretically learn how to respond as
a filter to other audio data (the acurateness would be determined by the
learning process and data), distinct from what was taught
Problem here is I'm actually training a 16 neuron network for each of the
samples, what makes this instr run sooo slooow! (this example takes about 4
min in my P200, for half-a-second of audio...). I really would apreciate
hints on speeding it up (for instance would table write be faster than zak?
- guess not).
I also hope that you can help me develop a more audio-prone coding
algorithm, but I'll get back on that later
I'm working on a more advanced instr using back propagation, and will post
results when I get it done
In the mean time, hope this rather useless example amuses you :)
pedro
_____________________________
; start of orchestra
sr = 44100
kr = 44100
ksmps = 1
zakinit 1, 295
;--------------------------------------------------;
; A simple single-layer bidirectional perceptron, ;
; using McCulloch-Pitts algorithm ;
; ;
; Coded for CSound by Pedro Batista jun/98 ;
;--------------------------------------------------;
;* * WARNING * * WARNING * * WARNING * * WARNING * *
;* I cannot guarantee bug-free execution,
;* nor can I assure that this instr wont generate
;* high-values, susceptible of causing harm to the
;* listner or listening equipment (it probably will,
;* if unknown patterns are presented, since there's
;* no compression coded yet) - pls use with caution!
;* * WARNING * * WARNING * * WARNING * * WARNING * *
instr 1
;--------------------------------------------------
inump init 3 ; number of patterns (maximum 3)
;--------------------------------------------------
iich1 init 1 ; in channel #1
iich2 init 2 ; in channel #2
iich3 init 3 ; in channel #3
ioch1 init 4 ; out channel #1
ioch2 init 5 ; out channel #2
ioch3 init 6 ; out channel #3
irch init 7 ; stimulus/response channel
ibin init 8; [8-23]
ibout init 24; [24-39]
ipes init 40; [40-295]
ipass init 0
isigi init 0
isigi1 init 0
isigi2 init 0
isigi3 init 0
isigi4 init 0
isigi5 init 0
isigo init 0
isigo1 init 0
isigo2 init 0
isigo3 init 0
isigo4 init 0
isigo5 init 0
indx init 0
indi init 0
indj init 0
ierr init 0
iact init 0
ipow2 init 0
; skip to performance
igoto perf
ini: ;-----------------------------------------------
; initialize weights
indi = 0
loopi:
indj = 0
loopj:
ziw 0, ipes+indi*16+indj
indj = indj+1
if (indj<16) igoto loopj
indi = indi+1
if (indi<16) igoto loopi
if (inump<1) igoto off
isigi1 zir iich1
isigo1 zir ioch1
if (inump<2) igoto nosig
isigi2 zir iich2
isigo2 zir ioch2
if (inump<3) igoto nosig
isigi3 zir iich3
isigo3 zir ioch3
nosig:
istim zir irch
ipass = 1
restrt: ;-----------------------------------------------
; select training signals
if (ipass==1) igoto case1
if (ipass==2) igoto case2
if (ipass==3) igoto case3
igoto skip
case1:
isigi = isigi1
isigo = isigo1
igoto skip
case2:
isigi = isigi2
isigo = isigo2
igoto skip
case3:
isigi = isigi3
isigo = isigo3
skip:
; convert input signal to binary
indx = 0
isigi = isigi+32768
loop1:
ibit = (frac(isigi/2)==0 ? 0:1)
ziw ibit, ibin+indx
isigi = int(isigi/2)
indx = indx+1
if (indx<16) igoto loop1
; convert output signal to binary
indx = 0
isigo = isigo+32768
loop2:
ibit = (frac(isigo/2)==0 ? 0:1)
ziw ibit, ibout+indx
isigo = int(isigo/2)
indx = indx+1
if (indx<16) igoto loop2
; train pattern
train:
indi = 0
loop3:
indj = 0
loop4:
ibi zir ibin+indi
ibo zir ibout+indj
iw zir ipes+indi*16+indj
iw = iw+((ibi*2-1)*(ibo*2-1))
ziw iw, ipes+indi*16+indj
indj = indj+1
if (indj<16) igoto loop4
indi = indi+1
if (indi<16) igoto loop3
ipass = ipass+1
; proceed with next pattern
if (ipass<=inump) igoto restrt
; convert stimulus signal to binary
indx = 0
istim = istim+32768
loop5:
ibit = (frac(istim/2)==0 ? 0:1)
ziw ibit, ibin+indx
istim = int(istim/2)
indx = indx+1
if (indx<16) igoto loop5
; calculate response from stimulus pattern
prop:
ierr = 0
indi = 0
loop6:
ibo zir ibout+indi
iact = 0
indj = 0
loop7:
ibi zir ibin+indj
iw zir ipes+indj*16+indi
iact = iact+iw*ibi
indj = indj+1
if (indj<16) igoto loop7
if (iact==0) igoto skp1
iact = (iact>0 ? 1:0)
ierr = ierr+abs(iact-ibo)
ziw iact, ibout+indi
skp1:
indi = indi+1
if (indi<16) igoto loop6
indi = 0
loop8:
ibi zir ibin+indi
iact = 0
indj = 0
loop9:
ibo zir ibout+indj
iw zir ipes+indj*16+indi
iact = iact+iw*ibo
indj = indj+1
if (indj<16) igoto loop9
if (iact==0) igoto skp2
iact = (iact>0 ? 1:0)
ierr = ierr+abs(iact-ibi)
ziw iact, ibin+indi
skp2:
indi = indi+1
if (indi<16) igoto loop8
if (ierr>0) igoto prop ; until stable
; convert binary response to audio value
indx = 0
isigo = 0
ipow2 = 1
loop10:
ibit zir ibout+indx
isigo = isigo+ibit*ipow2
ipow2 = ipow2*2
indx = indx+1
if (indx<16) igoto loop10
isigo = isigo-32768
; compress the resulting signal
;iclip table ... accepting ideas here :)
off:
; write the output and leave
ziw isigo, irch
rireturn
;============== P E R F O R M A N C E ==============;
perf:
; change input and output soundin # to whatever you like
; (the soundin # could be passed as a p-field, but for some
; reason my setup doesnt recognise teh SSDIR, so full path
; must be given
ain1 soundin "c:\csound\samples\soundin.1"
kin1 downsamp ain1
zkw kin1, iich1
ain2 soundin "c:\csound\samples\soundin.2"
kin2 downsamp ain2
zkw kin2, iich2
ain3 soundin "c:\csound\samples\soundin.3"
kin3 downsamp ain3
zkw kin3, iich3
; Just use the same pattern for input and output of the net
; A different set of samples could be present at the input and
; output, causing the learning of that association.
;aout1 soundin "c:\csound\samples\soundin.#" ; Program three
kout1 = kin1 ; patterns in
zkw kout1, ioch1 ; "recognition"
;aout2 soundin "c:\csound\samples\soundin.#" ; mode.
kout2 = kin2 ; Remember to
zkw kout2, ioch2 ; update var
;aout3 soundin "c:\csound\samples\soundin.#" ; inump
kout3 = kin3 ; at the top
zkw kout3, ioch3 ; of the instr
;astim soundin "c:\csound\samples\soundin.#" ; ask the net
kstim = kin1 ; for pattern
zkw kstim, irch ; number 1
; (try kin2&3)
; This will successfully (well, almost, try just 2) recognise
; any of the three patterns and hopefully reconstruct any of
; those, if it is slightly damaged.
; What "slightly" means is very little, using this simple
; network, and due to the limited effectiveness of the
; method used for the audio coding in binary form
; I'm studying an implementation with hidden layers and back-
; -prop learning, using better audio coding strategies
;
; One can present a completelly diferent input than those
; that were trained, and see how the net responds, but
; *NOTICE* there is no secure against signal clipping
; process sample at i-rate
reinit ini
;-----------------------------------------------
kout zkr irch
aout interp kout
out aout
endin
; start of score
; the instr just has to be running for the intended duration
i1 0 .5
e |