|
|
TITLE:: FluidBufMFCC
|
|
|
SUMMARY:: Mel-Frequency Cepstral Coefficients as Spectral Descriptors on a Buffer
|
|
|
CATEGORIES:: Libraries>FluidDecomposition
|
|
|
RELATED:: Guides/FluCoMa, Guides/FluidDecomposition, Classes/FluidBufMelBands
|
|
|
|
|
|
DESCRIPTION::
|
|
|
This class implements a classic spectral descriptor, the Mel-Frequency Cepstral Coefficients (https://en.wikipedia.org/wiki/Mel-frequency_cepstrum). The input is first filtered in to STRONG::numBands:: perceptually-spaced bands, as in LINK::Classes/FluidMelBands::. It is then analysed into STRONG::numCoeffs:: number of cepstral coefficients. It has the avantage of being amplitude invarient, except for the first coefficient. It is part of the Fluid Decomposition Toolkit of the FluCoMa project.FOOTNOTE:: This was made possible thanks to the FluCoMa project ( http://www.flucoma.org/ ) funded by the European Research Council ( https://erc.europa.eu/ ) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 725899).::
|
|
|
|
|
|
The process will return a single multichannel buffer of STRONG::numCoeffs:: per input channel. Each frame represents a value, which is every hopSize.
|
|
|
|
|
|
CLASSMETHODS::
|
|
|
|
|
|
METHOD:: process
|
|
|
This is the method that calls for the spectral shape descriptors to be calculated on a given source buffer.
|
|
|
|
|
|
ARGUMENT:: server
|
|
|
The server on which the buffers to be processed are allocated.
|
|
|
|
|
|
ARGUMENT:: source
|
|
|
The index of the buffer to use as the source material to be described through the various descriptors. The different channels of multichannel buffers will be processing sequentially.
|
|
|
|
|
|
ARGUMENT:: startFrame
|
|
|
Where in the srcBuf should the process start, in sample.
|
|
|
|
|
|
ARGUMENT:: numFrames
|
|
|
How many frames should be processed.
|
|
|
|
|
|
ARGUMENT:: startChan
|
|
|
For multichannel srcBuf, which channel should be processed first.
|
|
|
|
|
|
ARGUMENT:: numChans
|
|
|
For multichannel srcBuf, how many channel should be processed.
|
|
|
|
|
|
ARGUMENT:: features
|
|
|
The destination buffer for the numCoeffs coefficients describing the spectral shape.
|
|
|
|
|
|
ARGUMENT:: numCoeffs
|
|
|
The number of cepstral coefficients to be outputed. It will decide how many channels are produce per channel of the source.
|
|
|
|
|
|
ARGUMENT:: numBands
|
|
|
The number of bands that will be perceptually equally distributed between STRONG::minFreq:: and STRONG::maxFreq::.
|
|
|
|
|
|
ARGUMENT:: minFreq
|
|
|
The lower boundary of the lowest band of the model, in Hz.
|
|
|
|
|
|
ARGUMENT:: maxFreq
|
|
|
The highest boundary of the highest band of the model, in Hz.
|
|
|
|
|
|
ARGUMENT:: windowSize
|
|
|
The window size. As MFCC computation relies on spectral frames, we need to decide what precision we give it spectrally and temporally, in line with Gabor Uncertainty principles. http://www.subsurfwiki.org/wiki/Gabor_uncertainty
|
|
|
|
|
|
ARGUMENT:: hopSize
|
|
|
The window hop size. As MFCC computation relies on spectral frames, we need to move the window forward. It can be any size but low overlap will create audible artefacts.
|
|
|
|
|
|
ARGUMENT:: fftSize
|
|
|
The inner FFT/IFFT size. It should be at least 4 samples long, at least the size of the window, and a power of 2. Making it larger allows an oversampling of the spectral precision.
|
|
|
|
|
|
ARGUMENT:: action
|
|
|
A Function to be evaluated once the offline process has finished and all Buffer's instance variables have been updated on the client side. The function will be passed [features] as an argument.
|
|
|
|
|
|
RETURNS::
|
|
|
Nothing, as the destination buffer is declared in the function call.
|
|
|
|
|
|
EXAMPLES::
|
|
|
|
|
|
code::
|
|
|
// create some buffers
|
|
|
(
|
|
|
b = Buffer.read(s,File.realpath(FluidBufMFCC.class.filenameSymbol).dirname.withTrailingSlash ++ "../AudioFiles/Nicol-LoopE-M.wav");
|
|
|
c = Buffer.new(s);
|
|
|
)
|
|
|
|
|
|
// run the process with basic parameters
|
|
|
(
|
|
|
Routine{
|
|
|
t = Main.elapsedTime;
|
|
|
FluidBufMFCC.process(s, b, features: c);
|
|
|
(Main.elapsedTime - t).postln;
|
|
|
}.play
|
|
|
)
|
|
|
|
|
|
// listen to the source and look at the buffer
|
|
|
b.play;
|
|
|
c.plot(separately:true)
|
|
|
::
|
|
|
|
|
|
STRONG::A stereo buffer example.::
|
|
|
CODE::
|
|
|
|
|
|
// load two very different files
|
|
|
(
|
|
|
b = Buffer.read(s,File.realpath(FluidBufSpectralShape.class.filenameSymbol).dirname.withTrailingSlash ++ "../AudioFiles/Tremblay-SA-UprightPianoPedalWide.wav");
|
|
|
c = Buffer.read(s,File.realpath(FluidBufSpectralShape.class.filenameSymbol).dirname.withTrailingSlash ++ "../AudioFiles/Tremblay-AaS-AcousticStrums-M.wav");
|
|
|
)
|
|
|
|
|
|
// composite one on left one on right as test signals
|
|
|
FluidBufCompose.process(s, c, numFrames:b.numFrames, startFrame:555000,destStartChan:1, destination:b)
|
|
|
b.play
|
|
|
|
|
|
// create a buffer as destinations
|
|
|
c = Buffer.new(s);
|
|
|
|
|
|
//run the process on them
|
|
|
(
|
|
|
Routine{
|
|
|
t = Main.elapsedTime;
|
|
|
FluidBufMFCC.process(s, b, numCoeffs:5, features: c);
|
|
|
(Main.elapsedTime - t).postln;
|
|
|
}.play
|
|
|
)
|
|
|
|
|
|
// look at the buffer: 5 coefs for left, then 5 coefs for right (the first of each is linked to the loudness)
|
|
|
c.plot(separately:true)
|
|
|
:: |