Csound Csound-dev Csound-tekno Search About

Spectral component separation

Date2017-08-08 22:02
FromRoger Kelly
SubjectSpectral component separation
I want to try to split an input signal of sine waves into their various partials.

Basically if the input contains 440HZ and 880HZ I want to be able to generate the midi note(s) for these.

I saw spectrum can a "wsig" of the DFT, but I don't see an opcode other than "spectdisp" that can give me access to the various bins and their magnitudes.

Any ideas on how to get to those frequency separations produced by "spectrum"
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Date2017-08-08 22:16
FromEd Costello
SubjectRe: Spectral component separation
You can get an array of magnitudes from an audio signal using the pvsanal and pvs2tab opcodes, example:

<CsoundSynthesizer>
<CsInstruments>
nchnls = 2
0dbfs = 1
ksmps = 256
sr = 44100

opcode AudioToPolar, k[]k[], aii

  aInput, iFrameSize, iHopSize xin

  kmags[] init iFrameSize/2 + 1
  kfreqs[] init iFrameSize/2 + 1
  fsig   pvsanal aInput, iFrameSize, iHopSize, iFrameSize, 1
  kframe  pvs2tab kmags, kfreqs, fsig
  xout kmags, kfreqs

endop


opcode Sine, a, kk

  kAmplitude, kFrequency xin
  aOut oscil kAmplitude, kFrequency, ftgen(0, 0, 2^10, 10, 1)
  xout aOut

endop

gaOut init 0

instr 1

    iFrameSize = 1024
    iHopSize = 256
    aOut Sine 0.4, 440
    kmagnitudes[], kfrequencies[] AudioToPolar aOut, iFrameSize, iHopSize
    printk2 kmagnitudes[5]
endin

</CsInstruments>
<CsScore>
i1 0 2
</CsScore>
</CsoundSynthesizer>


The kmagnitudes array will contain 513 magnitude values for each frame of the audio signal.
Hope that helps,
Ed


On 8 Aug 2017, at 14:02, Roger Kelly <loraxman@gmail.com> wrote:

I want to try to split an input signal of sine waves into their various partials.

Basically if the input contains 440HZ and 880HZ I want to be able to generate the midi note(s) for these.

I saw spectrum can a "wsig" of the DFT, but I don't see an opcode other than "spectdisp" that can give me access to the various bins and their magnitudes.

Any ideas on how to get to those frequency separations produced by "spectrum"
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here


Date2017-08-08 22:19
From"Jeanette C."
SubjectRe: Spectral component separation
Aug 8 2017, Roger Kelly has written:

> I want to try to split an input signal of sine waves into their various
> partials.
Hi Roger,
is there any particular need to use the spectrum opcode?
Otherwise the PVS family of opcodes allows you access and methods for
manipulation. If the supplied opcodes from the streamable real-time
phase vocoding opcodes - aka PVS - won't do, you have pvsftw and pvsftr
which will write both frequencies and related amplitudes into two tables
and can read them back into an f-rate signal.
...

Best wishes,

Jeanette

--------
* website: http://juliencoder.de - for summer is a state of sound
* SoundCloud: https://soundcloud.com/jeanette_c

And I love the way with just one whisper
You tell me everything <3
(Britney Spears)

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2017-08-09 00:12
FromRoger Kelly
SubjectRe: Spectral component separation
Yes I think this is what I need.  Will give it a try.  Thanks 

On Aug 8, 2017 4:16 PM, "Ed Costello" <phasereset@gmail.com> wrote:
You can get an array of magnitudes from an audio signal using the pvsanal and pvs2tab opcodes, example:

<CsoundSynthesizer>
<CsInstruments>
nchnls = 2
0dbfs = 1
ksmps = 256
sr = 44100

opcode AudioToPolar, k[]k[], aii

  aInput, iFrameSize, iHopSize xin

  kmags[] init iFrameSize/2 + 1
  kfreqs[] init iFrameSize/2 + 1
  fsig   pvsanal aInput, iFrameSize, iHopSize, iFrameSize, 1
  kframe  pvs2tab kmags, kfreqs, fsig
  xout kmags, kfreqs

endop


opcode Sine, a, kk

  kAmplitude, kFrequency xin
  aOut oscil kAmplitude, kFrequency, ftgen(0, 0, 2^10, 10, 1)
  xout aOut

endop

gaOut init 0

instr 1

    iFrameSize = 1024
    iHopSize = 256
    aOut Sine 0.4, 440
    kmagnitudes[], kfrequencies[] AudioToPolar aOut, iFrameSize, iHopSize
    printk2 kmagnitudes[5]
endin

</CsInstruments>
<CsScore>
i1 0 2
</CsScore>
</CsoundSynthesizer>


The kmagnitudes array will contain 513 magnitude values for each frame of the audio signal.
Hope that helps,
Ed


On 8 Aug 2017, at 14:02, Roger Kelly <loraxman@gmail.com> wrote:

I want to try to split an input signal of sine waves into their various partials.

Basically if the input contains 440HZ and 880HZ I want to be able to generate the midi note(s) for these.

I saw spectrum can a "wsig" of the DFT, but I don't see an opcode other than "spectdisp" that can give me access to the various bins and their magnitudes.

Any ideas on how to get to those frequency separations produced by "spectrum"
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here
Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Date2017-08-09 19:34
FromRoger Kelly
SubjectRe: Spectral component separation
I had not considered pvsftw and pvsftr.  I will have a look at those also! In the end I am wanting to send the frequencies out as a "k" signal via OSC.


Thanks for the advice!


On Tue, Aug 8, 2017 at 4:19 PM, Jeanette C. <julien@mail.upb.de> wrote:
Aug 8 2017, Roger Kelly has written:

I want to try to split an input signal of sine waves into their various
partials.
Hi Roger,
is there any particular need to use the spectrum opcode?
Otherwise the PVS family of opcodes allows you access and methods for
manipulation. If the supplied opcodes from the streamable real-time
phase vocoding opcodes - aka PVS - won't do, you have pvsftw and pvsftr
which will write both frequencies and related amplitudes into two tables
and can read them back into an f-rate signal.
...

Best wishes,

Jeanette

--------
* website: http://juliencoder.de - for summer is a state of sound
* SoundCloud: https://soundcloud.com/jeanette_c

And I love the way with just one whisper
You tell me everything <3
(Britney Spears)


Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
       https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Date2017-08-09 20:03
FromRory Walsh
SubjectRe: Spectral component separation
You may wish to check out some of Oeyvind's work. His cross adaptive research using analysis to drive processing. More info can be found here:
http://crossadaptive.hf.ntnu.no/index.php/about-the-project/

On 9 Aug 2017 8:34 p.m., "Roger Kelly" <loraxman+csound@gmail.com> wrote:
I had not considered pvsftw and pvsftr.  I will have a look at those also! In the end I am wanting to send the frequencies out as a "k" signal via OSC.


Thanks for the advice!


On Tue, Aug 8, 2017 at 4:19 PM, Jeanette C. <julien@mail.upb.de> wrote:
Aug 8 2017, Roger Kelly has written:

I want to try to split an input signal of sine waves into their various
partials.
Hi Roger,
is there any particular need to use the spectrum opcode?
Otherwise the PVS family of opcodes allows you access and methods for
manipulation. If the supplied opcodes from the streamable real-time
phase vocoding opcodes - aka PVS - won't do, you have pvsftw and pvsftr
which will write both frequencies and related amplitudes into two tables
and can read them back into an f-rate signal.
...

Best wishes,

Jeanette

--------
* website: http://juliencoder.de - for summer is a state of sound
* SoundCloud: https://soundcloud.com/jeanette_c

And I love the way with just one whisper
You tell me everything <3
(Britney Spears)


Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
       https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Date2017-08-10 18:57
FromRoger Kelly
SubjectRe: Spectral component separation

So my attempt at using pvsanal is only giving me the same results for amplitude as frequency.  What am I doing wrong here?



<CsoundSynthesizer>

<CsOptions>

-odac

</CsOptions>

<CsInstruments>


instr 1

aSin poscil 0dbfs/4, 440


ifftsize = 512

ioverlap = ifftsize / 4

iwinsize = ifftsize

iwinshape = 1



inbins = 512

ifn ftgen 0,0,inbins,10,1 ; make ftable ;von-Hann window

ifq ftgen 0,0,inbins,10,1 ; make ftable


fftin pvsanal aSin, ifftsize, ioverlap, iwinsize, iwinshape ;fft-analysis of the audio-signal

kflag pvsftw fftin,ifn,ifq ; export amps/freq to table,



i_index = 0

isize = ftlen(ifq)

loop_start:

ivalue tablei i_index, ifq

ivalue2 tablei i_index, ifn

prints "freq %d:\t%f\n", i_index, ivalue

prints "amp %d:\t%f\n", i_index, ivalue2


loop_lt i_index, 1, isize, loop_start ;read table 1 with our index

endin

</CsInstruments>

<CsScore>

i 1 0 10

</CsScore>

</CsoundSynthesizer>



On Tue, Aug 8, 2017 at 4:19 PM, Jeanette C. <julien@mail.upb.de> wrote:
Aug 8 2017, Roger Kelly has written:

I want to try to split an input signal of sine waves into their various
partials.
Hi Roger,
is there any particular need to use the spectrum opcode?
Otherwise the PVS family of opcodes allows you access and methods for
manipulation. If the supplied opcodes from the streamable real-time
phase vocoding opcodes - aka PVS - won't do, you have pvsftw and pvsftr
which will write both frequencies and related amplitudes into two tables
and can read them back into an f-rate signal.
...

Best wishes,

Jeanette

--------
* website: http://juliencoder.de - for summer is a state of sound
* SoundCloud: https://soundcloud.com/jeanette_c

And I love the way with just one whisper
You tell me everything <3
(Britney Spears)


Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
       https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Date2017-08-10 20:34
From"Jeanette C."
SubjectRe: Spectral component separation
Aug 10 2017, Roger Kelly has written:

> So my attempt at using pvsanal is only giving me the same results for
> amplitude as frequency.  What am I doing wrong here?
Two things: pvsftw returns the kflag, which is set to one, when a new
FFT-window has been analysed. That doesn't happen every k-cycle. So the
whole loop code should be inside an if-statement:
if kflag == 1 then
; loop stuff
endif

Second: for the loop you use all i-rate variables: i_index, i_value,
i_value2,... That means they can only be assigned once at i-time. Make
them k-rate: kindex, kvalue, kvalue2,...

Two minor points: you don't need to use the ftlen, you know the length
of the function. Secondly, with an FFT-size of 512, you get 256 bins or
maybe 257.

HTH.

Best wishes,

Jeanette

--------
* website: http://juliencoder.de - for summer is a state of sound
* SoundCloud: https://soundcloud.com/jeanette_c

Make all the clouds disappear
Put all your fears to rest <3
(Britney Spears)

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2017-08-10 22:27
FromRoger Kelly
SubjectRe: Spectral component separation

I think I have something wrong with displaying or understanding the krate outputs.  If I just want to show all the bins with printks, I would have thought this would work.   But I don't see all the bins.


What is wrong below?


<CsoundSynthesizer>

<CsOptions>

-odac

</CsOptions>

<CsInstruments>


instr 1

aSin      poscil    0dbfs/4, 440


 

ifftsize  = 512

ioverlap  = ifftsize / 4

iwinsize  = ifftsize

iwinshape = 1



inbins = 512

ifn ftgen 0,0,inbins,10,1 ; make ftable ;von-Hann window

ifq ftgen 0,0,inbins,10,1 ; make ftable


fftin     pvsanal aSin, ifftsize, ioverlap, iwinsize, iwinshape ;fft-analysis of the audio-signal

kflag pvsftw fftin,ifn,ifq ; export amps to table,



kndx init 0

kncr=1

kmax=256


isize = ftlen(ifq)

if kflag == 1 then

loop_start:

    kvalue tablekt kndx, ifq

    kvalue2 tablekt kndx, ifn

    if kvalue2 > 0.0 && kvalue > 00.0 then

       printks "freq %d:\t%f\n",.001, kndx, kvalue

       printks "amp  %d:\t%f\n",.001, kndx, kvalue2

    endif

    kndx  =  kndx + 1

    printks "kndx %d\n", .1,kndx

    printks "kmax %d\n", .1,kmax

    if (kndx < kmax) kgoto loop_start


 endif

 

endin

</CsInstruments>

<CsScore>

i 1 0 .56

</CsScore>

</CsoundSynthesizer>


On Thu, Aug 10, 2017 at 2:34 PM, Jeanette C. <julien@mail.upb.de> wrote:
Aug 10 2017, Roger Kelly has written:

So my attempt at using pvsanal is only giving me the same results for
amplitude as frequency.  What am I doing wrong here?
Two things: pvsftw returns the kflag, which is set to one, when a new
FFT-window has been analysed. That doesn't happen every k-cycle. So the
whole loop code should be inside an if-statement:
if kflag == 1 then
; loop stuff
endif

Second: for the loop you use all i-rate variables: i_index, i_value,
i_value2,... That means they can only be assigned once at i-time. Make
them k-rate: kindex, kvalue, kvalue2,...

Two minor points: you don't need to use the ftlen, you know the length
of the function. Secondly, with an FFT-size of 512, you get 256 bins or
maybe 257.

HTH.

Best wishes,

Jeanette

--------
* website: http://juliencoder.de - for summer is a state of sound
* SoundCloud: https://soundcloud.com/jeanette_c

Make all the clouds disappear
Put all your fears to rest <3

(Britney Spears)

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
       https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here

Date2017-08-10 22:56
From"Jeanette C."
SubjectRe: Spectral component separation
Aug 10 2017, Roger Kelly has written:
...
> kflag pvsftw fftin,ifn,ifq ; export amps to table,
>
>
>
> kndx init 0
kndx = 0 ; init only works once per instrument call
>
> kncr=1
No need for kncr, loop_lt will solve that, at any rate this is a
constant, unless you plan to do interesting things.
>
> kmax=256
That can be imax, it's a constant based on the i-rate FFT size.
>
>
> isize = ftlen(ifq)
No need for isize, you have imax
>
> if kflag == 1 then
>
> loop_start:
>
>    kvalue tablekt kndx, ifq
You can use table or even tab, no need for interpolation, you want the
absolute values at direct indces:
kvalue tab kndx, ifq ;, 0
I believe the reading mode is 0 by default
>
>    kvalue2 tablekt kndx, ifn
Same here for tab
>
>    if kvalue2 > 0.0 && kvalue > 00.0 then
You don't really need to check the frequency value, only one bin will
potentially contain a frequency of 0Hz
>
>       printks "freq %d:\t%f\n",.001, kndx, kvalue
I usually go for something like:
printks "freq %d:\t%f\n", 1/sr, kndx, kvalue
You reach this statement only when analysis data from the FFT is
available. Lower rates can be helpful if you don't need every FFT frame.
>
>       printks "amp  %d:\t%f\n",.001, kndx, kvalue2
>
>    endif
>
Replace the lines below with:
loop_lt kndx, 1, imax, loop_start
>    kndx  =  kndx + 1
>    printks "kndx %d\n", .1,kndx
>
>    printks "kmax %d\n", .1,kmax
>
>    if (kndx < kmax) kgoto loop_start
...
>From here on everything can stay in place.

Best wishes,

Jeanette

--------
* website: http://juliencoder.de - for summer is a state of sound
* SoundCloud: https://soundcloud.com/jeanette_c

Make all the clouds disappear
Put all your fears to rest <3
(Britney Spears)

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
        https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Date2017-08-11 16:34
FromRoger Kelly
SubjectRe: Spectral component separation
Thanks so much! It did exactly what I wanted.


<CsoundSynthesizer>

<CsOptions>

</CsOptions>

<CsInstruments>


sr = 44100

ksmps = 1

nchnls = 1

0dbfs = 1.0



instr 1



gisine   ftgen 1, 0, 16384, 10, 1 ;sine wave

aSin      poscil    0dbfs, 880,gisine

aSin3      poscil    0dbfs, 440,gisine

aSin2      poscil    0dbfs, 110,gisine




ifftsize  = 512

ioverlap  = ifftsize / 4

iwinsize  = ifftsize

iwinshape = 1



inbins  =       512

ifn     ftgen   0,0,inbins,10,1         ; make ftable                                   ;von-Hann window

ifq     ftgen   0,0,inbins,10,1         ; make ftable


fftin     pvsanal aSin+aSin2+aSin3, ifftsize, ioverlap, iwinsize, iwinshape        ;fft-analysis of the audio-signal

kflag  pvsftw  fftin,ifn,ifq           ; export amps to table,


kndx = 0



imax=256


if kflag == 1 then

loop_start:

    kvalue tab kndx+0, ifq

 

 

    kvalue2 tab kndx+0, ifn

    if kvalue2 > 0.8 && kvalue > 00.8 then

;    if kndx%100 == 0 and kvalue2 > .8 then

    event "i", 2, 0, .01, kvalue,kvalue2

;    endif

    endif

    ;if kvalue2 > 0.0 && kvalue > 00.0 then

   ;;    printks "freq %d:\t%f\n", 1/sr, kndx, kvalue

    ;;     printks "amp  %d:\t%f\n",1/sr, kndx, kvalue2

   ; endif

loop_lt kndx, 1, imax, loop_start


 endif

out aSin

endin


instr 2

print p4

print p5

endin


</CsInstruments>

<CsScore>

i 1 0 3.56

</CsScore>

</CsoundSynthesizer>


On Thu, Aug 10, 2017 at 4:56 PM, Jeanette C. <julien@mail.upb.de> wrote:
Aug 10 2017, Roger Kelly has written:
...
kflag pvsftw fftin,ifn,ifq ; export amps to table,



kndx init 0
kndx = 0 ; init only works once per instrument call

kncr=1
No need for kncr, loop_lt will solve that, at any rate this is a
constant, unless you plan to do interesting things.

kmax=256
That can be imax, it's a constant based on the i-rate FFT size.


isize = ftlen(ifq)
No need for isize, you have imax

if kflag == 1 then

loop_start:

   kvalue tablekt kndx, ifq
You can use table or even tab, no need for interpolation, you want the
absolute values at direct indces:
kvalue tab kndx, ifq ;, 0
I believe the reading mode is 0 by default

   kvalue2 tablekt kndx, ifn
Same here for tab

   if kvalue2 > 0.0 && kvalue > 00.0 then
You don't really need to check the frequency value, only one bin will
potentially contain a frequency of 0Hz

      printks "freq %d:\t%f\n",.001, kndx, kvalue
I usually go for something like:
printks "freq %d:\t%f\n", 1/sr, kndx, kvalue
You reach this statement only when analysis data from the FFT is
available. Lower rates can be helpful if you don't need every FFT frame.

      printks "amp  %d:\t%f\n",.001, kndx, kvalue2

   endif

Replace the lines below with:
loop_lt kndx, 1, imax, loop_start
   kndx  =  kndx + 1
   printks "kndx %d\n", .1,kndx

   printks "kmax %d\n", .1,kmax

   if (kndx < kmax) kgoto loop_start
...
>From here on everything can stay in place.


Best wishes,

Jeanette

--------
* website: http://juliencoder.de - for summer is a state of sound
* SoundCloud: https://soundcloud.com/jeanette_c

Make all the clouds disappear
Put all your fears to rest <3
(Britney Spears)

Csound mailing list
Csound@listserv.heanet.ie
https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND
Send bugs reports to
       https://github.com/csound/csound/issues
Discussions of bugs and features can be posted here

Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here