|
|
TITLE:: FluidBufSines
|
|
|
SUMMARY:: Buffer-Based Sinusoidal Modelling and Resynthesis
|
|
|
CATEGORIES:: Libraries>FluidDecomposition, UGens>Buffer
|
|
|
RELATED:: Guides/FluCoMa, Guides/FluidDecomposition, Guides/FluidBufMultiThreading, Guides/FluidBufMultiThreading
|
|
|
|
|
|
|
|
|
DESCRIPTION::
|
|
|
This class triggers a Sinusoidal Modelling process on buffers on the non-real-time thread of the server. It implements a mix and match algorithms taken from classic papers. It is part of the Fluid Decomposition Toolkit of the FluCoMa project. footnote:: This was made possible thanks to the FluCoMa project ( http://www.flucoma.org/ ) funded by the European Research Council ( https://erc.europa.eu/ ) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 725899).::
|
|
|
|
|
|
The algorithm will take a buffer in, and will divide it in two parts: LIST::
|
|
|
## a reconstruction of what it detects as sinusoidal;
|
|
|
## a residual derived from the previous buffer to allow null-summing::
|
|
|
|
|
|
The whole process is based on the assumption that signal is made of pitched steady components that have a long-enough duration and are periodic enough to be perceived as such, that can be tracked, resynthesised and removed from the original, leaving behind what is considered as non-pitched, noisy, and/or transient. It first tracks the peaks, then checks if they are the continuation of a peak in previous spectral frames, by assigning them a track. More information on this model, and on how it links to musicianly thinking, are availabe in LINK::Guides/FluCoMa:: overview file.
|
|
|
|
|
|
STRONG::Threading::
|
|
|
|
|
|
By default, this UGen spawns a new thread to avoid blocking the server command queue, so it is free to go about with its business. For a more detailed discussion of the available threading and monitoring options, including the two undocumented Class Methods below (.processBlocking and .kr) please read the guide LINK::Guides/FluidBufMultiThreading::.
|
|
|
|
|
|
CLASSMETHODS::
|
|
|
|
|
|
METHOD:: process
|
|
|
This is the method that calls for the sinusoidal estimation to be calculated on a given source buffer and to be resynthesised.
|
|
|
|
|
|
ARGUMENT:: server
|
|
|
The server on which the buffers to be processed are allocated.
|
|
|
|
|
|
ARGUMENT:: source
|
|
|
The index of the buffer to use as the source material to be decomposed through the sinusoidal modelling process. The different channels of multichannel buffers will be processing sequentially.
|
|
|
|
|
|
ARGUMENT:: startFrame
|
|
|
Where in the srcBuf should the process start, in sample.
|
|
|
|
|
|
ARGUMENT:: numFrames
|
|
|
How many frames should be processed.
|
|
|
|
|
|
ARGUMENT:: startChan
|
|
|
For multichannel srcBuf, which channel should be processed first.
|
|
|
|
|
|
ARGUMENT:: numChans
|
|
|
For multichannel srcBuf, how many channel should be processed.
|
|
|
|
|
|
ARGUMENT:: sines
|
|
|
The index of the buffer where the extracted sinusoidal component will be reconstructed.
|
|
|
|
|
|
ARGUMENT:: residual
|
|
|
The index of the buffer where the residual of the sinusoidal component will be reconstructed.
|
|
|
|
|
|
ARGUMENT:: bandwidth
|
|
|
The number of bins used to resynthesises a peak. It has an effect on CPU cost: the widest is more accurate but more computationally expensive. It is capped at (fftSize / 2) + 1.
|
|
|
|
|
|
ARGUMENT:: detectionThreshold
|
|
|
The threshold in dB above which a magnitude peak is considered to be a sinusoidal component.
|
|
|
|
|
|
ARGUMENT:: birthLowThreshold
|
|
|
The threshold in dB above which to consider a peak to start a sinusoidal component tracking, for the low end of the spectrum. It is interpolated across the spectrum until birthHighThreshold at half-Nyquist.
|
|
|
|
|
|
ARGUMENT:: birthHighThreshold
|
|
|
The threshold in dB above which to consider a peak to start a sinusoidal component tracking, for the high end of the spectrum. It is interpolated across the spectrum until birthLowThreshold at DC.
|
|
|
|
|
|
ARGUMENT:: minTrackLen
|
|
|
The minimum duration, in spectral frames, for a sinusoidal track to be accepted as a partial. It allows to remove bubbly pitchy artefacts, but is more CPU intensive and might reject quick pitch material.
|
|
|
|
|
|
ARGUMENT:: trackingMethod
|
|
|
The algorithm used to track the sinusoidal continuity between spectral frames. 0 is the default, "Greedy", and 1 is a more expensive "Hungarian" one. footnote::Neri, J., and Depalle, P., "Fast Partial Tracking of Audio with Real-Time Capability through Linear Programming". Proceedings of DAFx-2018.::
|
|
|
|
|
|
ARGUMENT:: trackMagRange
|
|
|
The amplitude difference allowed for a track to diverge between frames, in dB.
|
|
|
|
|
|
ARGUMENT:: trackFreqRange
|
|
|
The frequency difference allowed for a track to diverge between frames, in Hertz.
|
|
|
|
|
|
ARGUMENT:: trackProb
|
|
|
The probability of the tracking algorithm to find a track.
|
|
|
|
|
|
ARGUMENT:: windowSize
|
|
|
The window size. As spectral differencing relies on spectral frames, we need to decide what precision we give it spectrally and temporally, in line with Gabor Uncertainty principles. http://www.subsurfwiki.org/wiki/Gabor_uncertainty
|
|
|
|
|
|
ARGUMENT:: hopSize
|
|
|
The window hop size. As sinusoidal estimation relies on spectral frames, we need to move the window forward. It can be any size but low overlap will create audible artefacts. The -1 default value will default to half of windowSize (overlap of 2).
|
|
|
|
|
|
ARGUMENT:: fftSize
|
|
|
The inner FFT/IFFT size. It should be at least 4 samples long, at least the size of the window, and a power of 2. Making it larger allows an oversampling of the spectral precision. The -1 default value will use the next power of 2 equal or above the highest of windowSize and (bandwidth - 1) * 2.
|
|
|
|
|
|
ARGUMENT:: action
|
|
|
A Function to be evaluated once the offline process has finished and all Buffer's instance variables have been updated on the client side. The function will be passed [sines, residual] as an argument.
|
|
|
|
|
|
RETURNS::
|
|
|
Nothing, as the various destination buffers are declared in the function call.
|
|
|
|
|
|
EXAMPLES::
|
|
|
|
|
|
code::
|
|
|
// create some buffers
|
|
|
(
|
|
|
b = Buffer.read(s,File.realpath(FluidBufSines.class.filenameSymbol).dirname.withTrailingSlash ++ "../AudioFiles/Tremblay-AaS-SynthTwoVoices-M.wav");
|
|
|
c = Buffer.new(s);
|
|
|
d = Buffer.new(s);
|
|
|
)
|
|
|
|
|
|
// run the process with basic parameters
|
|
|
(
|
|
|
Routine{
|
|
|
t = Main.elapsedTime;
|
|
|
FluidBufSines.process(s, b, sines: c, residual:d);
|
|
|
(Main.elapsedTime - t).postln;
|
|
|
}.play
|
|
|
)
|
|
|
|
|
|
// listen to each component
|
|
|
c.play;
|
|
|
d.play;
|
|
|
|
|
|
//nullsumming tests
|
|
|
{(PlayBuf.ar(1, c)) + (PlayBuf.ar(1,d)) - (PlayBuf.ar(1,b,doneAction:2))}.play
|
|
|
::
|
|
|
|
|
|
STRONG::A stereo buffer example.::
|
|
|
CODE::
|
|
|
|
|
|
// load two very different files
|
|
|
(
|
|
|
b = Buffer.read(s,File.realpath(FluidBufSines.class.filenameSymbol).dirname.withTrailingSlash ++ "../AudioFiles/Tremblay-SA-UprightPianoPedalWide.wav");
|
|
|
c = Buffer.read(s,File.realpath(FluidBufSines.class.filenameSymbol).dirname.withTrailingSlash ++ "../AudioFiles/Tremblay-AaS-AcousticStrums-M.wav");
|
|
|
)
|
|
|
|
|
|
// composite one on left one on right as test signals
|
|
|
FluidBufCompose.process(s, c, numFrames:b.numFrames, startFrame:555000,destStartChan:1, destination:b)
|
|
|
b.play
|
|
|
|
|
|
// create 2 new buffers as destinations
|
|
|
d = Buffer.new(s); e = Buffer.new(s);
|
|
|
|
|
|
//run the process on them
|
|
|
(
|
|
|
Routine{
|
|
|
t = Main.elapsedTime;
|
|
|
FluidBufSines.process(s, b, sines: d, residual:e, windowSize: 2048, hopSize: 256, fftSize: 16384);
|
|
|
(Main.elapsedTime - t).postln;
|
|
|
}.play
|
|
|
)
|
|
|
|
|
|
//listen: stereo preserved!
|
|
|
d.play
|
|
|
e.play
|
|
|
::
|