|
|
TITLE:: FluidBufSines
|
|
|
SUMMARY:: Buffer-Based Sinusoidal Modelling and Resynthesis
|
|
|
CATEGORIES:: Libraries>FluidDecomposition, UGens>Buffer
|
|
|
RELATED:: Guides/FluCoMa, Guides/FluidDecomposition
|
|
|
|
|
|
|
|
|
DESCRIPTION::
|
|
|
This class triggers a Sinusoidal Modelling process on buffers on the non-real-time thread of the server. It implements a mix and match algorithms taken from classic papers. It is part of the Fluid Decomposition Toolkit of the FluCoMa project. footnote:: This was made possible thanks to the FluCoMa project ( http://www.flucoma.org/ ) funded by the European Research Council ( https://erc.europa.eu/ ) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 725899).::
|
|
|
|
|
|
The algorithm will take a buffer in, and will divide it in two parts: LIST::
|
|
|
## a reconstruction of what it detects as sinusoidal;
|
|
|
## a residual derived from the previous buffer to allow null-summing::
|
|
|
|
|
|
The whole process is based on the assumption that signal is made of pitched steady components that have a long-enough duration and are periodic enough to be perceived as such, that can be tracked, resynthesised and removed from the original, leaving behind what is considered as non-pitched, noisy, and/or transient. It first tracks the peaks, then checks if they are the continuation of a peak in previous spectral frames, by assigning them a track. More information on this model, and on how it links to musicianly thinking, are availabe in LINK::Guides/FluCoMa:: overview file.
|
|
|
|
|
|
CLASSMETHODS::
|
|
|
|
|
|
METHOD:: process
|
|
|
This is the method that calls for the sinusoidal estimation to be calculated on a given source buffer and to be resynthesised.
|
|
|
|
|
|
ARGUMENT:: server
|
|
|
The server on which the buffers to be processed are allocated.
|
|
|
|
|
|
ARGUMENT:: srcBufNum
|
|
|
The index of the buffer to use as the source material to be decomposed through the NMF process. The different channels of multichannel buffers will be processing sequentially.
|
|
|
|
|
|
ARGUMENT:: startAt
|
|
|
Where in the srcBuf should the process start, in sample.
|
|
|
|
|
|
ARGUMENT:: nFrames
|
|
|
How many frames should be processed.
|
|
|
|
|
|
ARGUMENT:: startChan
|
|
|
For multichannel srcBuf, which channel should be processed first.
|
|
|
|
|
|
ARGUMENT:: nChans
|
|
|
For multichannel srcBuf, how many channel should be processed.
|
|
|
|
|
|
ARGUMENT:: sineBufNum
|
|
|
The index of the buffer where the extracted sinusoidal component will be reconstructed.
|
|
|
|
|
|
ARGUMENT:: resBufNum
|
|
|
The index of the buffer where the residual of the sinusoidal component will be reconstructed.
|
|
|
|
|
|
ARGUMENT:: bandwidth
|
|
|
The width in bins of the fragment of the fft window that is considered a normal deviation for a potential continuous sinusoidal track. It has an effect on CPU cost: the widest is more accurate but more computationally expensive.
|
|
|
|
|
|
ARGUMENT:: thresh
|
|
|
The normalised threshold, between 0 an 1, to consider a peak as a sinusoidal component from the normalized cross-correlation.
|
|
|
|
|
|
ARGUMENT:: minTrackLen
|
|
|
The minimum duration, in spectral frames, for a sinusoidal track to be accepted as a partial. It allows to remove space-monkeys, but is more CPU intensive and might reject quick pitch material.
|
|
|
|
|
|
ARGUMENT:: magWeight
|
|
|
The weight of the magnitude proximity of a peak when trying to associate it to an existing track (relative to freqWeight - suggested between 0 to 1)
|
|
|
|
|
|
ARGUMENT:: freqWeight
|
|
|
The weight of the frequency proximity of a peak when trying to associate it to an existing track (relative to magWeight - suggested between 0 to 1)
|
|
|
|
|
|
ARGUMENT:: winSize
|
|
|
The window size. As sinusoidal estimation relies on spectral frames, we need to decide what precision we give it spectrally and temporally, in line with Gabor Uncertainty principles. http://www.subsurfwiki.org/wiki/Gabor_uncertainty
|
|
|
|
|
|
ARGUMENT:: hopSize
|
|
|
The window hope size. As sinusoidal estimation relies on spectral frames, we need to move the window forward. It can be any size but low overlap will create audible artefacts.
|
|
|
|
|
|
ARGUMENT:: fftSize
|
|
|
The inner FFT/IFFT size. It should be at least 4 samples long, at least the size of the window, and a power of 2. Making it larger allows an oversampling of the spectral precision.
|
|
|
|
|
|
RETURNS::
|
|
|
Nothing, as the various destination buffers are declared in the function call.
|
|
|
|
|
|
EXAMPLES::
|
|
|
|
|
|
code::
|
|
|
// create some buffers
|
|
|
(
|
|
|
b = Buffer.read(s,File.realpath(FluidBufSines.class.filenameSymbol).dirname.withTrailingSlash ++ "../AudioFiles/Tremblay-AaS-SynthTwoVoices-M.wav");
|
|
|
c = Buffer.new(s);
|
|
|
d = Buffer.new(s);
|
|
|
)
|
|
|
|
|
|
// run the process with basic parameters
|
|
|
(
|
|
|
Routine{
|
|
|
t = Main.elapsedTime;
|
|
|
FluidBufSines.process(s, b.bufnum, sineBufNum: c.bufnum, resBufNum:d.bufnum);
|
|
|
s.sync;
|
|
|
(Main.elapsedTime - t).postln;
|
|
|
}.play
|
|
|
)
|
|
|
|
|
|
// listen to each component
|
|
|
c.query;
|
|
|
c.play;
|
|
|
d.query;
|
|
|
d.play;
|
|
|
|
|
|
//nullsumming tests
|
|
|
{(PlayBuf.ar(1,c.bufnum))+(PlayBuf.ar(1,d.bufnum))-(PlayBuf.ar(1,b.bufnum,doneAction:2))}.play
|
|
|
::
|
|
|
|