beat detection, sample replacement/peak opcode
Date | 2017-02-24 09:04 |
From | Richard |
Subject | beat detection, sample replacement/peak opcode |
What I try to do is find the start and endpoint of samples in a wave file, in order to reverse them. (I do not know in advance where the samples are) I tried it first by exporting the file to ASCII and then in a Python program find the start and end points. This turns out to be not so easy as it seems. Then I thought about beat detection and read Jim Hearon's article about the subject. He mentions several opcodes there, among them is peak. I tried peak, but it did not work as expected, even with the supplied sample. The peak output values are nowhere near the peaks in the audio sample - verified that with Audacity. I also read that most beat detection algorithms use a moving window (say 1024 samples) to detect the average energy in a sound. My questions are: What opcode should I use to find the start of a sample (say with a fast attack) Does Csound have opcodes for beat detection based on a window? Richard Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here |
Date | 2017-02-24 10:07 |
From | Tarmo Johannes |
Subject | Re: beat detection, sample replacement/peak opcode |
Hi, Oeyvind has a very good onset detection UDO, I think that should serve you well (I can send later), to detect end of a sample, probably follow opcode with relatively long attack time in combination with trigger on a given threshold opcode should do it. Not by computer, sorry for not sensing links but maybe it helps you further. Tarmo 24.02.2017 11:05 kirjutas kuupäeval "Richard" <zappfinger@gmail.com>: What I try to do is find the start and endpoint of samples in a wave file, in order to reverse them. |
Date | 2017-02-24 10:44 |
From | Richard |
Subject | Re: beat detection, sample replacement/peak opcode |
Thanks Tarmo, I would love to see that UDO. Richard On 24/02/17 11:07, Tarmo Johannes
wrote:
|
Date | 2017-02-24 17:38 |
From | Oeyvind Brandtsegg |
Subject | Re: beat detection, sample replacement/peak opcode |
Hi Richard, Here's the UDO. I usually use follow2 with a quick attack (0.01) and a slow decay (around 2 sec) to get the amp envelope. Then downsamp the follow2 output and convert it to dB. Then this is the "kin" signal to the transient detection opcode. The reason for separating our the envelope follower is that I also experimented with detecting transients in other signals (pitch, centroid etc), and in those cases the "preprocessing" of the signal had to be done differently. To get your "segment end" marker, you might use the "kreGate" signal from inside the UDO, as this will indicate that the signal level hass fallen below a certain threshold, X dB below the level you had when the transient was detected. If this is too rough you might implement a similar gate with another threshold. Usage example:: a1 = your input sound a_env follow2 a1, kAttack, kRelease k_env downsamp a_env ktransient, kdiff TransientDetect dbfsamp(k_env), iresponse, ktthresh, klowThresh, kdecThresh, kdoubleLimit ; Transient detection udo, Oeyvind Brandtsegg opcode TransientDetect, kk,kikkkk kin, iresponse, ktthresh, klowThresh, kdecThresh, kdoubleLimit xin /* iresponse = 10 ; response time in milliseconds ktthresh = 6 ; transient trig threshold klowThresh = -60 ; lower threshold for transient detection kdoubleLimit = 0.02 ; minimum duration between events, (double trig limit) kdecThresh = 6 ; retrig threshold, how much must the level decay from its local max before allowing new transient trig */ kinDel delayk kin, iresponse/1000 ; delay with response time for comparision of levels ktrig = ((kin > kinDel + ktthresh) ? 1 : 0) ; if current rms plus threshold is larger than previous rms, set trig signal to current rms klowGate = (kin < klowThresh? 0 : 1) ; gate to remove transient of low level signals ktrig = ktrig * klowGate ; activate gate on trig signal ktransLev init 0 ktransLev samphold kin, 1-ktrig ; read amplitude at transient kreGate init 1 ; retrigger gate, to limit transient double trig before signal has decayed (decThresh) from its local max ktrig = ktrig*kreGate ; activate gate kmaxAmp init -99999 kmaxAmp max kmaxAmp, kin ; find local max amp kdiff = kmaxAmp-kin ; how much the signal has decayed since its local max value kreGate limit kreGate-ktrig, 0, 1 ; mute when trig detected kreGate = (kdiff > kdecThresh ? 1 : kreGate) ; re-enable gate when signal has decayed sufficiently kmaxAmp = (kreGate == 1 ? -99999 : kmaxAmp) ; reset max amp gauge ; avoid closely spaced transient triggers (first trig priority) kdouble init 1 ktrig = ktrig*kdouble if ktrig > 0 then reinit double endif double: idoubleLimit = i(kdoubleLimit) idoubleLimit limit idoubleLimit, 1/kr, 5 kdouble linseg 0, idoubleLimit, 0, 0, 1, 1, 1 rireturn xout ktrig, kdiff endop 2017-02-24 2:44 GMT-08:00 Richard |
Date | 2017-02-24 20:26 |
From | Richard |
Subject | Re: beat detection, sample replacement/peak opcode |
Thanks Oeyvind, I'll try it out soon. Richard On 24/02/17 18:38, Oeyvind Brandtsegg wrote: > Hi Richard, > > Here's the UDO. I usually use follow2 with a quick attack (0.01) and a > slow decay (around 2 sec) to get the amp envelope. Then downsamp the > follow2 output and convert it to dB. Then this is the "kin" signal to > the transient detection opcode. The reason for separating our the > envelope follower is that I also experimented with detecting > transients in other signals (pitch, centroid etc), and in those cases > the "preprocessing" of the signal had to be done differently. > > To get your "segment end" marker, you might use the "kreGate" signal > from inside the UDO, as this will indicate that the signal level hass > fallen below a certain threshold, X dB below the level you had when > the transient was detected. If this is too rough you might implement a > similar gate with another threshold. > > Usage example:: > a1 = your input sound > a_env follow2 a1, kAttack, kRelease > k_env downsamp a_env > ktransient, kdiff TransientDetect dbfsamp(k_env), iresponse, ktthresh, > klowThresh, kdecThresh, kdoubleLimit > > > ; Transient detection udo, Oeyvind Brandtsegg > opcode TransientDetect, kk,kikkkk > kin, iresponse, ktthresh, klowThresh, kdecThresh, kdoubleLimit xin > /* > iresponse = 10 ; response time in milliseconds > ktthresh = 6 ; transient trig threshold > klowThresh = -60 ; lower threshold for transient detection > kdoubleLimit = 0.02 ; minimum duration between events, (double trig limit) > kdecThresh = 6 ; retrig threshold, how much must the level decay from > its local max before allowing new transient trig > */ > kinDel delayk kin, iresponse/1000 ; delay with response time for > comparision of levels > ktrig = ((kin > kinDel + ktthresh) ? 1 : 0) ; if current rms plus > threshold is larger than previous rms, set trig signal to current rms > klowGate = (kin < klowThresh? 0 : 1) ; gate to remove transient of low > level signals > ktrig = ktrig * klowGate ; activate gate on trig signal > ktransLev init 0 > ktransLev samphold kin, 1-ktrig ; read amplitude at transient > kreGate init 1 ; retrigger gate, to limit transient double trig before > signal has decayed (decThresh) from its local max > ktrig = ktrig*kreGate ; activate gate > kmaxAmp init -99999 > kmaxAmp max kmaxAmp, kin ; find local max amp > kdiff = kmaxAmp-kin ; how much the signal has decayed since its local max value > kreGate limit kreGate-ktrig, 0, 1 ; mute when trig detected > kreGate = (kdiff > kdecThresh ? 1 : kreGate) ; re-enable gate when > signal has decayed sufficiently > kmaxAmp = (kreGate == 1 ? -99999 : kmaxAmp) ; reset max amp gauge > > ; avoid closely spaced transient triggers (first trig priority) > kdouble init 1 > ktrig = ktrig*kdouble > if ktrig > 0 then > reinit double > endif > double: > idoubleLimit = i(kdoubleLimit) > idoubleLimit limit idoubleLimit, 1/kr, 5 > kdouble linseg 0, idoubleLimit, 0, 0, 1, 1, 1 > rireturn > > xout ktrig, kdiff > endop > > > > 2017-02-24 2:44 GMT-08:00 Richard |
Date | 2017-02-24 21:08 |
From | Tarmo Johannes |
Subject | Re: beat detection, sample replacement/peak opcode |
Attachments | randomPanner-proov.csd |
Hi, there is aslo an example csd where I use it for random panning. Your idea is getting me excited, I want to try something like that also for live input. If I get it done, I will post it and we can share the experiences! tarmo On Friday 24 February 2017 21:26:17 you wrote: > Thanks Oeyvind, I'll try it out soon. > > Richard > > On 24/02/17 18:38, Oeyvind Brandtsegg wrote: > > Hi Richard, > > > > Here's the UDO. I usually use follow2 with a quick attack (0.01) and a > > slow decay (around 2 sec) to get the amp envelope. Then downsamp the > > follow2 output and convert it to dB. Then this is the "kin" signal to > > the transient detection opcode. The reason for separating our the > > envelope follower is that I also experimented with detecting > > transients in other signals (pitch, centroid etc), and in those cases > > the "preprocessing" of the signal had to be done differently. > > > > To get your "segment end" marker, you might use the "kreGate" signal > > from inside the UDO, as this will indicate that the signal level hass > > fallen below a certain threshold, X dB below the level you had when > > the transient was detected. If this is too rough you might implement a > > similar gate with another threshold. > > > > Usage example:: > > a1 = your input sound > > a_env follow2 a1, kAttack, kRelease > > k_env downsamp a_env > > ktransient, kdiff TransientDetect dbfsamp(k_env), iresponse, ktthresh, > > klowThresh, kdecThresh, kdoubleLimit > > > > > > ; Transient detection udo, Oeyvind Brandtsegg > > opcode TransientDetect, kk,kikkkk > > kin, iresponse, ktthresh, klowThresh, kdecThresh, kdoubleLimit xin > > /* > > iresponse = 10 ; response time in milliseconds > > ktthresh = 6 ; transient trig threshold > > klowThresh = -60 ; lower threshold for transient detection > > kdoubleLimit = 0.02 ; minimum duration between events, (double trig limit) > > kdecThresh = 6 ; retrig threshold, how much must the level decay from > > its local max before allowing new transient trig > > */ > > kinDel delayk kin, iresponse/1000 ; delay with response time for > > comparision of levels > > ktrig = ((kin > kinDel + ktthresh) ? 1 : 0) ; if current rms plus > > threshold is larger than previous rms, set trig signal to current rms > > klowGate = (kin < klowThresh? 0 : 1) ; gate to remove transient of low > > level signals > > ktrig = ktrig * klowGate ; activate gate on trig signal > > ktransLev init 0 > > ktransLev samphold kin, 1-ktrig ; read amplitude at transient > > kreGate init 1 ; retrigger gate, to limit transient double trig before > > signal has decayed (decThresh) from its local max > > ktrig = ktrig*kreGate ; activate gate > > kmaxAmp init -99999 > > kmaxAmp max kmaxAmp, kin ; find local max amp > > kdiff = kmaxAmp-kin ; how much the signal has decayed since its local max > > value kreGate limit kreGate-ktrig, 0, 1 ; mute when trig detected > > kreGate = (kdiff > kdecThresh ? 1 : kreGate) ; re-enable gate when > > signal has decayed sufficiently > > kmaxAmp = (kreGate == 1 ? -99999 : kmaxAmp) ; reset max amp gauge > > > > ; avoid closely spaced transient triggers (first trig priority) > > kdouble init 1 > > ktrig = ktrig*kdouble > > if ktrig > 0 then > > reinit double > > endif > > > > double: > > idoubleLimit = i(kdoubleLimit) > > idoubleLimit limit idoubleLimit, 1/kr, 5 > > kdouble linseg 0, idoubleLimit, 0, 0, 1, 1, 1 > > > > rireturn > > > > xout ktrig, kdiff > > endop > > > > 2017-02-24 2:44 GMT-08:00 Richard |
Date | 2017-02-24 22:10 |
From | Richard |
Subject | Re: beat detection, sample replacement/peak opcode |
I think this particular effect is a bit hard to do live. In order to reverse say a gitar note, the whole note has to be played first. So I am thinking of applying the effect on a recorded solo. It is an old fashioned, Beatlelesque effect... Richard On 24/02/17 22:08, Tarmo Johannes wrote: > Hi, > > there is aslo an example csd where I use it for random panning. > > Your idea is getting me excited, I want to try something like that also for > live input. If I get it done, I will post it and we can share the experiences! > > tarmo > > On Friday 24 February 2017 21:26:17 you wrote: >> Thanks Oeyvind, I'll try it out soon. >> >> Richard >> >> On 24/02/17 18:38, Oeyvind Brandtsegg wrote: >>> Hi Richard, >>> >>> Here's the UDO. I usually use follow2 with a quick attack (0.01) and a >>> slow decay (around 2 sec) to get the amp envelope. Then downsamp the >>> follow2 output and convert it to dB. Then this is the "kin" signal to >>> the transient detection opcode. The reason for separating our the >>> envelope follower is that I also experimented with detecting >>> transients in other signals (pitch, centroid etc), and in those cases >>> the "preprocessing" of the signal had to be done differently. >>> >>> To get your "segment end" marker, you might use the "kreGate" signal >>> from inside the UDO, as this will indicate that the signal level hass >>> fallen below a certain threshold, X dB below the level you had when >>> the transient was detected. If this is too rough you might implement a >>> similar gate with another threshold. >>> >>> Usage example:: >>> a1 = your input sound >>> a_env follow2 a1, kAttack, kRelease >>> k_env downsamp a_env >>> ktransient, kdiff TransientDetect dbfsamp(k_env), iresponse, ktthresh, >>> klowThresh, kdecThresh, kdoubleLimit >>> >>> >>> ; Transient detection udo, Oeyvind Brandtsegg >>> opcode TransientDetect, kk,kikkkk >>> kin, iresponse, ktthresh, klowThresh, kdecThresh, kdoubleLimit xin >>> /* >>> iresponse = 10 ; response time in milliseconds >>> ktthresh = 6 ; transient trig threshold >>> klowThresh = -60 ; lower threshold for transient detection >>> kdoubleLimit = 0.02 ; minimum duration between events, (double trig limit) >>> kdecThresh = 6 ; retrig threshold, how much must the level decay from >>> its local max before allowing new transient trig >>> */ >>> kinDel delayk kin, iresponse/1000 ; delay with response time for >>> comparision of levels >>> ktrig = ((kin > kinDel + ktthresh) ? 1 : 0) ; if current rms plus >>> threshold is larger than previous rms, set trig signal to current rms >>> klowGate = (kin < klowThresh? 0 : 1) ; gate to remove transient of low >>> level signals >>> ktrig = ktrig * klowGate ; activate gate on trig signal >>> ktransLev init 0 >>> ktransLev samphold kin, 1-ktrig ; read amplitude at transient >>> kreGate init 1 ; retrigger gate, to limit transient double trig before >>> signal has decayed (decThresh) from its local max >>> ktrig = ktrig*kreGate ; activate gate >>> kmaxAmp init -99999 >>> kmaxAmp max kmaxAmp, kin ; find local max amp >>> kdiff = kmaxAmp-kin ; how much the signal has decayed since its local max >>> value kreGate limit kreGate-ktrig, 0, 1 ; mute when trig detected >>> kreGate = (kdiff > kdecThresh ? 1 : kreGate) ; re-enable gate when >>> signal has decayed sufficiently >>> kmaxAmp = (kreGate == 1 ? -99999 : kmaxAmp) ; reset max amp gauge >>> >>> ; avoid closely spaced transient triggers (first trig priority) >>> kdouble init 1 >>> ktrig = ktrig*kdouble >>> if ktrig > 0 then >>> reinit double >>> endif >>> >>> double: >>> idoubleLimit = i(kdoubleLimit) >>> idoubleLimit limit idoubleLimit, 1/kr, 5 >>> kdouble linseg 0, idoubleLimit, 0, 0, 1, 1, 1 >>> >>> rireturn >>> >>> xout ktrig, kdiff >>> endop >>> >>> 2017-02-24 2:44 GMT-08:00 Richard |
Date | 2017-02-24 22:13 |
From | Oeyvind Brandtsegg |
Subject | Re: beat detection, sample replacement/peak opcode |
.... unless you can implement it with a negative delay time, allowing it to predict the future... whohaa! 2017-02-24 14:10 GMT-08:00 Richard |
Date | 2017-02-25 16:18 |
From | Vincent Michalke |
Subject | Aw: [Csnd] beat detection, sample replacement/peak opcode |
I use the following UDO, it works good if you adjust the values to match the sound that you want to analyze
<CsoundSynthesizer> <CsOptions> -iadc </CsOptions> <CsInstruments>
sr = 44100 ksmps = 32 nchnls = 2 0dbfs = 1.0
opcode Onset, k, aiiiiii ain,iMinFreq,iMaxFreq,iAboveMed,iOffset,iMinSec,iMedLen xin ifftsize = 1024 iIndexStart limit int(iMinFreq*(ifftsize/sr))*2,0,sr/2 iIndexEnd limit int(iMaxFreq*(ifftsize/sr))*2,0,sr/2 fsrc pvsanal ain,ifftsize,ifftsize/4,ifftsize,1 kArr[] init ifftsize+2 kflag pvs2array kArr, fsrc ksumold init 0 kMedIndex init 0 kMedSum init 0 kMedian[] init iMedLen kMinDist init 0 iMinDist = iMinSec*(sr/ksmps) kMinDist limit kMinDist-1,0,100000 if changed(kflag) == 1 && kMinDist == 0 then ksum = 0 kIndex = iIndexStart until kIndex = iIndexEnd do ksum = ksum+kArr[kIndex] kIndex += 2 od kFLUX = ksum-ksumold ksumold = ksum kOnset = 0 if kFLUX > (kMedSum*iAboveMed)+iOffset then kOnset = 1 kMinDist = iMinDist endif kMedian[kMedIndex] = (kFLUX>=0?kFLUX:0) kMedSum = sumarray(kMedian)/iMedLen kMedIndex = (kMedIndex+1)%iMedLen endif
xout changed(kOnset)==1&&kOnset==1?1:0 endop
instr 1 ain inch 1 ;frequency range to analyze in hz (max samplerate/2) iMinFreq = 1500 iMaxFreq = 20000 ;factor how much stronger the onset amplitude should be, compared to median iAboveMed = 4 ;low level noise offset iAmpOffset = 0.003 ;minimum time between two onsets iMinSec = 0.03 ;how many frames are used to calculate median value iMedLen = 25 ktrigger Onset ain,iMinFreq,iMaxFreq,iAboveMed,iAmpOffset,iMinSec,iMedLen printk2 ktrigger endin
</CsInstruments> <CsScore> i 1 0 36000 </CsScore> </CsoundSynthesizer> Gesendet: Freitag, 24. Februar 2017 um 10:04 Uhr
Von: Richard <zappfinger@GMAIL.COM> An: CSOUND@LISTSERV.HEANET.IE Betreff: [Csnd] beat detection, sample replacement/peak opcode What I try to do is find the start and endpoint of samples in a wave
file, in order to reverse them. (I do not know in advance where the samples are) I tried it first by exporting the file to ASCII and then in a Python program find the start and end points. This turns out to be not so easy as it seems. Then I thought about beat detection and read Jim Hearon's article about the subject. He mentions several opcodes there, among them is peak. I tried peak, but it did not work as expected, even with the supplied sample. The peak output values are nowhere near the peaks in the audio sample - verified that with Audacity. I also read that most beat detection algorithms use a moving window (say 1024 samples) to detect the average energy in a sound. My questions are: What opcode should I use to find the start of a sample (say with a fast attack) Does Csound have opcodes for beat detection based on a window? Richard Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here |
Date | 2017-02-25 17:34 |
From | Oeyvind Brandtsegg |
Subject | Re: Aw: [Csnd] beat detection, sample replacement/peak opcode |
Hi Vincent, thanks for this UDO, it looks interesting. Could you describe a bit how it works, and how it gains from using the frequency domain? all best Oeyvind 2017-02-25 8:18 GMT-08:00 Vincent Michalke |
Date | 2017-02-25 18:21 |
From | Vincent Michalke |
Subject | Aw: Re: [Csnd] [Csnd] beat detection, sample replacement/peak opcode |
Hi Oeyvind,
the UDO sums the amplitudes of bins in a specified frequency range (defined by iMinFreq,iMaxFreq).
A median value of these sums is calculated and it is checked if the current sum is higher than the median by a certain factor (iAboveMed).
If that is the case the UDO outputs 1, otherwise 0.
For me it was useful to detect onsets of an acoustic guitar. The onset sound is noisy for a short moment, so by just looking at amplitude changes in the range ~2000-20000hz i could more or less filter out the swinging strings and just detect that onset noise.
Hope this helps.
Vincent
Gesendet: Samstag, 25. Februar 2017 um 18:34 Uhr
Von: "Oeyvind Brandtsegg" <oyvind.brandtsegg@NTNU.NO> An: CSOUND@LISTSERV.HEANET.IE Betreff: Re: [Csnd] Aw: [Csnd] beat detection, sample replacement/peak opcode Hi Vincent,
thanks for this UDO, it looks interesting. Could you describe a bit how it works, and how it gains from using the frequency domain? all best Oeyvind 2017-02-25 8:18 GMT-08:00 Vincent Michalke <VincentMi@web.de>: > I use the following UDO, it works good if you adjust the values to match the > sound that you want to analyze > > > <CsoundSynthesizer> > > <CsOptions> > > -iadc > > </CsOptions> > > <CsInstruments> > > > > sr = 44100 > > ksmps = 32 > > nchnls = 2 > > 0dbfs = 1.0 > > > > > > opcode Onset, k, aiiiiii > > ain,iMinFreq,iMaxFreq,iAboveMed,iOffset,iMinSec,iMedLen xin > > ifftsize = 1024 > > iIndexStart limit int(iMinFreq*(ifftsize/sr))*2,0,sr/2 > > iIndexEnd limit int(iMaxFreq*(ifftsize/sr))*2,0,sr/2 > > fsrc pvsanal ain,ifftsize,ifftsize/4,ifftsize,1 > > kArr[] init ifftsize+2 > > kflag pvs2array kArr, fsrc > > ksumold init 0 > > kMedIndex init 0 > > kMedSum init 0 > > kMedian[] init iMedLen > > kMinDist init 0 > > iMinDist = iMinSec*(sr/ksmps) > > kMinDist limit kMinDist-1,0,100000 > > if changed(kflag) == 1 && kMinDist == 0 then > > ksum = 0 > > kIndex = iIndexStart > > until kIndex = iIndexEnd do > > ksum = ksum+kArr[kIndex] > > kIndex += 2 > > od > > kFLUX = ksum-ksumold > > ksumold = ksum > > kOnset = 0 > > if kFLUX > (kMedSum*iAboveMed)+iOffset then > > kOnset = 1 > > kMinDist = iMinDist > > endif > > kMedian[kMedIndex] = (kFLUX>=0?kFLUX:0) > > kMedSum = sumarray(kMedian)/iMedLen > > kMedIndex = (kMedIndex+1)%iMedLen > > endif > > > > xout changed(kOnset)==1&&kOnset==1?1:0 > > endop > > > > instr 1 > > ain inch 1 > > ;frequency range to analyze in hz (max samplerate/2) > > iMinFreq = 1500 > > iMaxFreq = 20000 > > ;factor how much stronger the onset amplitude should be, compared to median > > iAboveMed = 4 > > ;low level noise offset > > iAmpOffset = 0.003 > > ;minimum time between two onsets > > iMinSec = 0.03 > > ;how many frames are used to calculate median value > > iMedLen = 25 > > ktrigger Onset ain,iMinFreq,iMaxFreq,iAboveMed,iAmpOffset,iMinSec,iMedLen > > printk2 ktrigger > > endin > > > > > > </CsInstruments> > > <CsScore> > > i 1 0 36000 > > </CsScore> > > </CsoundSynthesizer> > > Gesendet: Freitag, 24. Februar 2017 um 10:04 Uhr > Von: Richard <zappfinger@GMAIL.COM> > An: CSOUND@LISTSERV.HEANET.IE > Betreff: [Csnd] beat detection, sample replacement/peak opcode > What I try to do is find the start and endpoint of samples in a wave > file, in order to reverse them. > (I do not know in advance where the samples are) > I tried it first by exporting the file to ASCII and then in a Python > program find the start and end points. > This turns out to be not so easy as it seems. > Then I thought about beat detection and read Jim Hearon's article about > the subject. He mentions several opcodes there, among them is peak. > I tried peak, but it did not work as expected, even with the supplied > sample. > The peak output values are nowhere near the peaks in the audio sample - > verified that with Audacity. > I also read that most beat detection algorithms use a moving window (say > 1024 samples) to detect the average energy in a sound. > My questions are: > What opcode should I use to find the start of a sample (say with a fast > attack) > Does Csound have opcodes for beat detection based on a window? > > Richard > > Csound mailing list > Csound@listserv.heanet.ie > https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND > Send bugs reports to > https://github.com/csound/csound/issues > Discussions of bugs and features can be posted here > Csound mailing list Csound@listserv.heanet.ie > https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to > https://github.com/csound/csound/issues Discussions of bugs and features can > be posted here -- Oeyvind Brandtsegg Professor of Music Technology NTNU 7491 Trondheim Norway Cell: +47 92 203 205 http://www.partikkelaudio.com/ http://crossadaptive.hf.ntnu.no http://gdsp.hf.ntnu.no/ http://soundcloud.com/brandtsegg http://flyndresang.no/ http://soundcloud.com/t-emp Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here |
Date | 2017-02-25 20:41 |
From | Richard |
Subject | Re: beat detection, sample replacement/peak opcode |
Hi Oeyvind, Works like a charm. Since I am interested in finding the sample positions, I run the instrument at ksmps 1 and have added the following after calling the UDO: if ktransient == 1 then k1 timeinstk printk2 k1 endif Richard On 24/02/17 18:38, Oeyvind Brandtsegg wrote: > Hi Richard, > > Here's the UDO. I usually use follow2 with a quick attack (0.01) and a > slow decay (around 2 sec) to get the amp envelope. Then downsamp the > follow2 output and convert it to dB. Then this is the "kin" signal to > the transient detection opcode. The reason for separating our the > envelope follower is that I also experimented with detecting > transients in other signals (pitch, centroid etc), and in those cases > the "preprocessing" of the signal had to be done differently. > > To get your "segment end" marker, you might use the "kreGate" signal > from inside the UDO, as this will indicate that the signal level hass > fallen below a certain threshold, X dB below the level you had when > the transient was detected. If this is too rough you might implement a > similar gate with another threshold. > > Usage example:: > a1 = your input sound > a_env follow2 a1, kAttack, kRelease > k_env downsamp a_env > ktransient, kdiff TransientDetect dbfsamp(k_env), iresponse, ktthresh, > klowThresh, kdecThresh, kdoubleLimit > > > ; Transient detection udo, Oeyvind Brandtsegg > opcode TransientDetect, kk,kikkkk > kin, iresponse, ktthresh, klowThresh, kdecThresh, kdoubleLimit xin > /* > iresponse = 10 ; response time in milliseconds > ktthresh = 6 ; transient trig threshold > klowThresh = -60 ; lower threshold for transient detection > kdoubleLimit = 0.02 ; minimum duration between events, (double trig limit) > kdecThresh = 6 ; retrig threshold, how much must the level decay from > its local max before allowing new transient trig > */ > kinDel delayk kin, iresponse/1000 ; delay with response time for > comparision of levels > ktrig = ((kin > kinDel + ktthresh) ? 1 : 0) ; if current rms plus > threshold is larger than previous rms, set trig signal to current rms > klowGate = (kin < klowThresh? 0 : 1) ; gate to remove transient of low > level signals > ktrig = ktrig * klowGate ; activate gate on trig signal > ktransLev init 0 > ktransLev samphold kin, 1-ktrig ; read amplitude at transient > kreGate init 1 ; retrigger gate, to limit transient double trig before > signal has decayed (decThresh) from its local max > ktrig = ktrig*kreGate ; activate gate > kmaxAmp init -99999 > kmaxAmp max kmaxAmp, kin ; find local max amp > kdiff = kmaxAmp-kin ; how much the signal has decayed since its local max value > kreGate limit kreGate-ktrig, 0, 1 ; mute when trig detected > kreGate = (kdiff > kdecThresh ? 1 : kreGate) ; re-enable gate when > signal has decayed sufficiently > kmaxAmp = (kreGate == 1 ? -99999 : kmaxAmp) ; reset max amp gauge > > ; avoid closely spaced transient triggers (first trig priority) > kdouble init 1 > ktrig = ktrig*kdouble > if ktrig > 0 then > reinit double > endif > double: > idoubleLimit = i(kdoubleLimit) > idoubleLimit limit idoubleLimit, 1/kr, 5 > kdouble linseg 0, idoubleLimit, 0, 0, 1, 1, 1 > rireturn > > xout ktrig, kdiff > endop > > > > 2017-02-24 2:44 GMT-08:00 Richard |
Date | 2017-02-25 23:27 |
From | luis jure |
Subject | Re: beat detection, sample replacement/peak opcode |
el 2017-02-25 a las 17:18 Vincent Michalke escribió: > I use the following UDO, it works good if you adjust the values to match > the sound that you want to analyze hey, vincent, you finally did it... :-) btw, i haven't followed the thread too closely, but it seems that you're talking about *onset* detection, and not beat detection, which is something completely different. i haven't tried the udo or looked at the code closely, but it seems to be based in something similar to spectral flux, which is one of the most established techniques for onset detection. good work, vincent! lj -- Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here |
Date | 2017-02-26 00:03 |
From | Richard |
Subject | Re: Aw: [Csnd] beat detection, sample replacement/peak opcode |
Thanks, this also seems to work quite well. I have tried it
without any real fine tuning. Richard
On 25/02/17 17:18, Vincent Michalke
wrote:
|
Date | 2017-02-26 05:56 |
From | Oeyvind Brandtsegg |
Subject | Re: Aw: [Csnd] beat detection, sample replacement/peak opcode |
Hi, I think that the later onsets in Vincents UDO is due to the spectral processing and also the median filtering. It is a nice idea though, perhaps it could be implemented with a simpler filter in the time domain, and thus reduce the latency? (It does not seem to use spectral flux though, Louis. That would involve multiplying amps of a frame with the amps of the previous frame and then normalizing) 2017-02-25 16:03 GMT-08:00 Richard |
Date | 2017-02-26 13:10 |
From | luis jure |
Subject | Re: Aw: [Csnd] beat detection, sample replacement/peak opcode |
el 2017-02-25 a las 21:56 Oeyvind Brandtsegg escribió: > (It does not seem to use spectral flux though, Louis. That would > involve multiplying amps of a frame with the amps of the previous > frame and then normalizing) well, i said "something similar" to spectral flux... :-) anyway, i don't understand your description of spectral flux. there are some variations, but generally speaking SF consists in taking the (usually normalized) spectrum and then finding the difference (L1 or L2) between consecutive frames (often half-wave rectified). i don't think it involves multiplying amplitudes between consecutive frames. but then again, i'm not an expert in dsp and i might have got you wrong. Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here |
Date | 2017-02-27 06:23 |
From | Oeyvind Brandtsegg |
Subject | Re: Aw: [Csnd] beat detection, sample replacement/peak opcode |
I would not call myself an expert either, but I've used spectral flux for some resent work, and the definition I've based my analysis on is in the article "The Timbre Toolbox: Extracting audio descriptors from musical signals" by Geoffroy Peeters et.al, in J. Acoust. Soc. Am., Vol. 130, No. 5, November 2011, also available here: https://www.mcgill.ca/mpcl/files/mpcl/peeters_2011_jasa.pdf. There, the spectral flux is defined as "1 minus the normalized correlation between the successive ak" where ak (subscript k) "represents the value at bin k of the magnitude STFT". The correlation is done by multiplying the value of a bin k at time m with the value of bin k at time m-1, summing these across the whole frame. In my expreience, the resulting measure is a good indicator of the balance between noise and tonal/stable components in the timbre. 2017-02-26 5:10 GMT-08:00 luis jure |
Date | 2017-02-27 23:02 |
From | luis jure |
Subject | Re: Aw: [Csnd] beat detection, sample replacement/peak opcode |
el 2017-02-26 a las 22:23 Oeyvind Brandtsegg escribió: > the definition I've based my analysis on is in the article "The Timbre > Toolbox: Extracting audio descriptors from musical signals" by Geoffroy > Peeters et.al, in J. Acoust. Soc. Am., Vol. 130, No. 5, November 2011, > also available here: > https://www.mcgill.ca/mpcl/files/mpcl/peeters_2011_jasa.pdf. interesting reference, oeyvind, thanks for the link. i didn't know that one, i'll definitely have to read it in detail. their definition of spectral flux is definitely different from the "standard" definition in the literature about onset detection. actually, the paper seems to be more oriented toward timbre description, and area where mcadams is an expert. did this approach work well for you for onsets detection? current research in this area use a different definition of spectral flux, as i described in my previous mail. there are countless references, but the following three papers give a good overall picture: Simon Dixon. "Onset detection revisited." In Proc. of the Int. Conf. on Digital Audio Effects (DAFx-06), pages 133–137, Montreal, Quebec, Canada, September 2006. http://dafx.ca/proceedings/papers/p_133.pdf Juan Pablo Bello, Laurent Daudet, Samer Abdallah, Chris Duxbury, Mike Davies, and Mark B. Sandler. “A tutorial on onset detection in musical signals” IEEE Trans. Speech and Audio Proc., vol. 13, no. 5, pp. 1035–1047, 2005. http://hans.fugal.net/comps/papers/bello_2005.pdf Sebastian Böck, Florian Krebs, and Markus Schedl. "Evaluating the online capabilities of onset detection methods." In Proc. of the 13th International Society for Music Information Retrieval Conference (ISMIR 2012), pages 49–54, Porto, Portugal, Aug. 9-13 2012. http://www.cp.jku.at/research/papers/Boeck_etal_ISMIR_2012.pdf BTW, sebastian böck and colleagues released a python library with tools for music information retrieval, including onsets detection: https://github.com/CPJKU/madmom (many more references there) best, lj Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here |
Date | 2017-02-28 08:06 |
From | Oeyvind Brandtsegg |
Subject | Re: Aw: [Csnd] beat detection, sample replacement/peak opcode |
Thanks for bringing this to my attention, I had missed the different definitions used for spectral flux. I had touched on some of these articles, but not thoroughly enough. The measure is called *spectral difference* in the Bello article, which is perhaps a better term for this method(?). Please do correct me, but is not the positive difference as used here dependent on signal amplitude? I might misread the maths, but it looks so to me. The spectral flux as used by Peeters is normalized with regards to signal amplitude, so it seems more robust(?). One thing I find objectionable is that Dixon mentions onset detection functions to have a low sampling rate of 100 Hz. (Böck mentiones an even lower accuracy of +/- 25 ms to be sufficient, as an improvement over the 50 ms accuracy used in earlier studies). For rhythmical analysis, one could say that no rhythms in a musical sense will occur at a frequency faster than around 20 Hz so 100Hz should be ample sampling rate. However, fine rhythmical nuances in phrasing would then be limited to a resolution of 10 milliseconds, and I think a lot of performing musicians would argue that they strive to place their notes more accurately than that. Playing on a realtime instrument with audio buffering shows a clear perceptible difference when adjusting the buffer size from a conservative 256 (approx 6 ms latency) to 128 (approx 3 ms latency), and similarly from 128 to 64. I haven't done quantitative studies of this, but merely experimented with it to determine where I find my perceptual thresholds are, and discussed with musician colleagues to try to verify, or at least find some concensus. According to these experiments, a 10 millisecond resolution is far from sufficient, Using spectral techniques not only introduce latency, but also limits the time precision within where onsets can be placed due to the analysis window. For this reason, I have not used spectral techniques for onset detection, but tried to find time domain methods that performs "sufficiently" well. I do note that the spectral methods can provide reliable onset detection for instruments that have a softer amplitude envelope, like arco strings or flute for example. Perhaps it would be a good idea to combine the time based methods with a "fail-safe" based on spectral methods, and in the case where the time domain method did not detect an onset but the spectral method provided one, we could accept the lower resolution and higher latency (since those onsets would otherwise have been lost). I should perhaps state that my use for the techniques are realtime performance, where the output might be used for triggering and or for rhythm analysis. Thus, my requirements may limit the choice of methods somewhat, 2017-02-27 15:02 GMT-08:00 luis jure |
Date | 2017-03-02 14:30 |
From | luis jure |
Subject | Re: beat detection, sample replacement/peak opcode |
el 2017-02-28 a las 00:06 Oeyvind Brandtsegg escribió: > The measure is called *spectral difference* in the Bello article, which > is perhaps a better term for this method(?). as far as i have seen, both terms are equivalent and used more or less indistinctly. > Please do correct me, but is not the positive difference as used here > dependent on signal amplitude? I might misread the maths, but it looks > so to me. most techniques normalize the spectrum. böck doesn't use normalization because it's not suited for online situations. he compensates this with a novel peak detection algorithm. onset detection systems are considered to have three stages: - pre-processing (e. g. normalization, whitening, etc) - the onset detection function (in this case, spectral flux, there are others) - peak detection that means that the detection function itself is only one step in the process, and the peak detection method has a great impact on the performance (not in terms of temporal accuracy, but rather in terms of not missing onsets, and not returning false positives). > For rhythmical analysis, one could say that no rhythms in a musical > sense will occur at a frequency faster than around 20 Hz so 100Hz should > be ample sampling rate. However, fine rhythmical nuances in phrasing > would then be limited to a resolution of 10 milliseconds, and I think a > lot of performing musicians would argue that they strive to place their > notes more accurately than that. definitely, yes. > Using spectral techniques not only introduce latency, but also limits > the time precision within where onsets can be placed due to the analysis > window. yes, that's true. most onset detection techniques i know seem to have been developed for metric analysis and are not so well suited for microtiming analysis. in that scenario, "robust" means catching all onsets and not getting false positives. > I should perhaps state that my use for the techniques are realtime > performance, where the output might be used for triggering and or for > rhythm analysis. Thus, my requirements may limit the choice of methods > somewhat, definitely, yes. best, lj Csound mailing list Csound@listserv.heanet.ie https://listserv.heanet.ie/cgi-bin/wa?A0=CSOUND Send bugs reports to https://github.com/csound/csound/issues Discussions of bugs and features can be posted here |