Update SC help files

nix
Owen Green 5 years ago
parent d0cbf95550
commit c6eec5a671

@ -43,7 +43,7 @@ ARGUMENT:: minSliceLength
The length in samples that the Slice will stay ON. Changes of states during that period will be ignored. The length in samples that the Slice will stay ON. Changes of states during that period will be ignored.
ARGUMENT:: highPassFreq ARGUMENT:: highPassFreq
The frequency of the fourth-order LinkwitzRiley high-pass filter (https://en.wikipedia.org/wiki/Linkwitz%E2%80%93Riley_filter). This is done first on the signal to minimise low frequency intermodulation with very fast ramp lengths. A frequency of 0 bypasses the filter. The frequency of the fourth-order link::https://en.wikipedia.org/wiki/Linkwitz%E2%80%93Riley_filter##LinkwitzRiley high-pass filter::. This is done first on the signal to minimise low frequency intermodulation with very fast ramp lengths. A frequency of 0 bypasses the filter.
RETURNS:: RETURNS::
An audio stream with square envelopes around the slices. The latency between the input and the output is dependant on the relation between the two envelope followers. An audio stream with square envelopes around the slices. The latency between the input and the output is dependant on the relation between the two envelope followers.
@ -54,7 +54,8 @@ code::
// detrending explained // detrending explained
// Our source here is a sinewave that does not go to silence and has sharp-ish amplitude bumps as onsets we try to track // Our source here is a sinewave that does not go to silence and has sharp-ish amplitude bumps as onsets we try to track
( (
{var env, source = SinOsc.ar(320,0,LFSaw.ar(20, 0, -0.4, 0.6)); {
var env, source = SinOsc.ar(320,0,LFSaw.ar(20, 0, -0.4, 0.6));
env = FluidAmpSlice.ar(source,fastRampUp: 5,fastRampDown: 50,slowRampUp: 220,slowRampDown: 220, onThreshold: 10, offThreshold: 10,floor: -60); env = FluidAmpSlice.ar(source,fastRampUp: 5,fastRampDown: 50,slowRampUp: 220,slowRampDown: 220, onThreshold: 10, offThreshold: 10,floor: -60);
[source, env] [source, env]
}.plot(0.08); }.plot(0.08);

@ -71,7 +71,7 @@ ARGUMENT:: lookAhead
The length of the buffer kept after an offset to allow the algorithm, once the Slice is considered finished, to wait further in time (up to that many samples) to find a minimum amplitude as the Slice offset point. This affects the latency of the algorithm. The length of the buffer kept after an offset to allow the algorithm, once the Slice is considered finished, to wait further in time (up to that many samples) to find a minimum amplitude as the Slice offset point. This affects the latency of the algorithm.
ARGUMENT:: highPassFreq ARGUMENT:: highPassFreq
The frequency of the fourth-order LinkwitzRiley high-pass filter (https://en.wikipedia.org/wiki/Linkwitz%E2%80%93Riley_filter). This is done first on the signal to minimise low frequency intermodulation with very fast ramp lengths. A frequency of 0 bypasses the filter. The frequency of the fourth-order link::https://en.wikipedia.org/wiki/Linkwitz%E2%80%93Riley_filter##LinkwitzRiley high-pass filter::. This is done first on the signal to minimise low frequency intermodulation with very fast ramp lengths. A frequency of 0 bypasses the filter.
ARGUMENT:: action ARGUMENT:: action
A Function to be evaluated once the offline process has finished and indices instance variables have been updated on the client side. The metric will be passed indices as an argument. A Function to be evaluated once the offline process has finished and indices instance variables have been updated on the client side. The metric will be passed indices as an argument.
@ -89,7 +89,6 @@ c = Buffer.new(s);
) )
b.play b.play
b.plot b.plot
//basic tests: absThresh sanity //basic tests: absThresh sanity
FluidBufAmpGate.process(s, b, indices:c, rampUp:5, rampDown:25, onThreshold:-12, offThreshold: -12) FluidBufAmpGate.process(s, b, indices:c, rampUp:5, rampDown:25, onThreshold:-12, offThreshold: -12)
c.query c.query
@ -145,12 +144,15 @@ c = Buffer.new(s);
) )
// slice the samples // slice the samples
FluidBufAmpGate.process(s, b, indices:c, rampUp:110, rampDown:2205, onThreshold:-27, offThreshold: -31, minSilenceLength:1100, lookBack:441, highPassFreq:40) (
c.query Routine{
c.getn(0,c.numFrames*2,{|item|item.postln;}) FluidBufAmpGate.process(s, b, indices:c, rampUp:110, rampDown:2205, onThreshold:-27, offThreshold: -31, minSilenceLength:1100, lookBack:441, highPassFreq:40).wait;
c.query;
c.getn(0,c.numFrames*2,{|item|item.postln;});
//reformatting to read the onsets and offsets as pairs //reformatting to read the onsets and offsets as pairs
c.getn(0,c.numFrames*2,{|items|items.reshape(c.numFrames,2).do({|x| x.postln});}) c.getn(0,c.numFrames*2,{|items|items.reshape(c.numFrames,2).do({|x| x.postln});});
}.play
)
//loops over a splice with the MouseX, taking the respective onset and offset of a given slice //loops over a splice with the MouseX, taking the respective onset and offset of a given slice
( (
{ {
@ -178,13 +180,14 @@ b.play
// create a new buffer as destinations // create a new buffer as destinations
c = Buffer.new(s); c = Buffer.new(s);
OSCFunc.trace(false)
//run the process on them //run the process on them
( (
// with basic params // with basic params
Routine{ Routine{
t = Main.elapsedTime; var t = Main.elapsedTime;
FluidBufAmpGate.process(s, b, indices: c, rampUp:1, rampDown:10, onThreshold: -30); var proc= FluidBufAmpGate.process(s, b, indices: c, rampUp:1, rampDown:10, onThreshold: -30);
proc.wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -146,8 +146,10 @@ c = Buffer.new(s);
( (
// with basic params // with basic params
Routine{ Routine{
t = Main.elapsedTime; var t = Main.elapsedTime;
FluidBufAmpSlice.process(s,b, indices: c, fastRampUp: 10,fastRampDown: 2205,slowRampUp: 4410,slowRampDown: 4410, onThreshold: 10,offThreshold: 5); var proc = FluidBufAmpSlice.process(s,b, indices: c, fastRampUp: 10,fastRampDown: 2205,slowRampUp: 4410,slowRampDown: 4410, onThreshold: 10,offThreshold: 5);
proc.wait;
c.query;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -89,6 +89,7 @@ FluidBufAudioTransport.process(s,b,source2:c,destination:d,interpolation:0.5,act
// listen to the source and the result // listen to the source and the result
b.play b.play
c.play c.play
d.updateInfo
d.play d.play
// more interesting sources: two cardboard bowing gestures // more interesting sources: two cardboard bowing gestures

@ -68,29 +68,42 @@ d = Buffer.new(s);
) )
// with basic params (basic summing of each full buffer in all dimensions) // with basic params (basic summing of each full buffer in all dimensions)
FluidBufCompose.process(s, source: b, destination: d); (
FluidBufCompose.process(s, source: c, destination: d, destGain: 1.0); Routine{
FluidBufCompose.process(s, source: b, destination: d).wait;
FluidBufCompose.process(s, source: c, destination: d, destGain: 1.0).wait;
d.query; d.query;
d.play; d.play;
}.play;
)
//constructing a mono buffer, with a quiet punch from the synth, with a choked piano resonance from the left channel //constructing a mono buffer, with a quiet punch from the synth, with a choked piano resonance from the left channel
(
Routine{
d.free; d = Buffer.new(s); d.free; d = Buffer.new(s);
FluidBufCompose.process(s, source: b, numFrames: 9000, gain: 0.5, destination: d); FluidBufCompose.process(s, source: b, numFrames: 9000, gain: 0.5, destination: d).wait;
FluidBufCompose.process(s, source: c, startFrame:30000, numFrames:44100, numChans:1, gain:0.9, destination: d, destGain: 1.0); FluidBufCompose.process(s, source: c, startFrame:30000, numFrames:44100, numChans:1, gain:0.9, destination: d, destGain: 1.0).wait;
d.query; d.query;
d.play; d.play;
}.play
)
//constructing a stereo buffer, with the end of the mono synth in both channels, with a piano resonance in swapped stereo //constructing a stereo buffer, with the end of the mono synth in both channels, with a piano resonance in swapped stereo
(
Routine{
d.free; d = Buffer.new(s); d.free; d = Buffer.new(s);
FluidBufCompose.process(s, source: b, startFrame: 441000, numChans: 2, gain: 0.6, destination: d); FluidBufCompose.process(s, source: b, startFrame: 441000, numChans: 2, gain: 0.6, destination: d).wait;
FluidBufCompose.process(s, source: c, numFrames: 78000, startChan: 1, numChans: 2, gain: 0.5, destStartFrame: 22050, destination: d, destGain: 1.0); FluidBufCompose.process(s, source: c, numFrames: 78000, startChan: 1, numChans: 2, gain: 0.5, destStartFrame: 22050, destination: d, destGain: 1.0).wait;
d.query; d.query;
d.play; d.play;
}.play
)
//constructing a one second buffer: the first second of each buffer, the mono synth on the right, the piano on the left //constructing a one second buffer: the first second of each buffer, the mono synth on the right, the piano on the left
(
Routine{
d.free; d = Buffer.new(s); d.free; d = Buffer.new(s);
FluidBufCompose.process(s, source: b, numFrames: 44100, numChans: 1, destStartChan: 1, destination: d); FluidBufCompose.process(s, source: b, numFrames: 44100, numChans: 1, destStartChan: 1, destination: d).wait;
FluidBufCompose.process(s, source: c, numFrames:44100, numChans:1, destination: d, destGain: 1.0); FluidBufCompose.process(s, source: c, numFrames:44100, numChans:1, destination: d, destGain: 1.0).wait;
d.query; d.query;
d.play; d.play;
}.play
)
:: ::

@ -83,7 +83,7 @@ FluidBufPitch.process(s,b,numFrames:512 * 10,numChans:1,features:~pitchdata,acti
// Flatten and print the flat buffer. We expect to see larger numbers (20-2000) interleaved with smaller (0-1) // Flatten and print the flat buffer. We expect to see larger numbers (20-2000) interleaved with smaller (0-1)
( (
FluidBufFlatten.process(s,~pitchdata,~flatdata,action:{ FluidBufFlatten.process(s,~pitchdata,~flatdata,axis:1,action:{
~flatdata.loadToFloatArray(action:{ |a| ~flatdata.loadToFloatArray(action:{ |a|
a.postln; a.postln;
}) })

@ -123,16 +123,17 @@ code::
d = Buffer.new(s); d = Buffer.new(s);
e = Buffer.new(s); e = Buffer.new(s);
) )
OSCFunc.trace(true, true)
// run with basic parameters // run with basic parameters
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufHPSS.process(s, b, harmonic: c, percussive: d); FluidBufHPSS.process(s, b, harmonic: c, percussive: d).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
c.query
d.query
//play the harmonic //play the harmonic
c.play; c.play;
//play the percussive //play the percussive
@ -145,7 +146,8 @@ d.play;
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufHPSS.process(s, b, harmonic: c, percussive: d, residual:e, harmFilterSize:31, maskingMode:2, harmThreshFreq1: 0.005, harmThreshAmp1: 7.5, harmThreshFreq2: 0.168, harmThreshAmp2: 7.5, percThreshFreq1: 0.004, percThreshAmp1: 26.5, percThreshFreq2: 0.152, percThreshAmp2: 26.5,windowSize:4096,hopSize:512); FluidBufHPSS.process(s, b, harmonic: c, percussive: d, residual:e, harmFilterSize:31, maskingMode:2, harmThreshFreq1: 0.005, harmThreshAmp1: 7.5, harmThreshFreq2: 0.168, harmThreshAmp2: 7.5, percThreshFreq1: 0.004, percThreshAmp1: 26.5, percThreshFreq2: 0.152, percThreshAmp2: 26.5,windowSize:4096,hopSize:512)
.wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -171,9 +173,12 @@ c = Buffer.read(s,File.realpath(FluidBufHPSS.class.filenameSymbol).dirname.withT
) )
// composite one on left one on right as test signals // composite one on left one on right as test signals
FluidBufCompose.process(s, c, numFrames:b.numFrames, startFrame:555000,destStartChan:1, destination:b) (
Routine{
FluidBufCompose.process(s, c, numFrames:b.numFrames, startFrame:555000,destStartChan:1, destination:b).wait;
b.play b.play
}.play
)
// create 2 new buffers as destinations // create 2 new buffers as destinations
d = Buffer.new(s); e = Buffer.new(s); d = Buffer.new(s); e = Buffer.new(s);
@ -181,7 +186,7 @@ d = Buffer.new(s); e = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufHPSS.process(s, b, harmonic: d, percussive:e); FluidBufHPSS.process(s, b, harmonic: d, percussive:e).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -73,7 +73,7 @@ b.play; b.plot;
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufLoudness.process(s, source:b, features: c); FluidBufLoudness.process(s, source:b, features: c).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -109,7 +109,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufLoudness.process(s, b, features: c, windowSize: 17640, hopSize:4410); FluidBufLoudness.process(s, b, features: c, windowSize: 17640, hopSize:4410).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -78,7 +78,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufMFCC.process(s, b, features: c); FluidBufMFCC.process(s, b, features: c).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -108,7 +108,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufMFCC.process(s, b, numCoeffs:5, features: c); FluidBufMFCC.process(s, b, numCoeffs:5, features: c).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -79,7 +79,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufMelBands.process(s, b, features: c, numBands:10); FluidBufMelBands.process(s, b, features: c, numBands:10).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -109,7 +109,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufMelBands.process(s, b, features: c, numBands:10); FluidBufMelBands.process(s, b, features: c, numBands:10).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -142,7 +142,7 @@ d.play //////(beware !!!! loud!!!)
( (
// separate them in 2 components // separate them in 2 components
Routine { Routine {
FluidBufNMF.process(s, d, resynth:e, bases: f, activations:g, components:2); FluidBufNMF.process(s, d, resynth:e, bases: f, activations:g, components:2).wait;
e.query; e.query;
f.query; f.query;
g.query; g.query;
@ -179,16 +179,15 @@ y = Buffer.new(s);
// matrix factorisation, requesting everything - wait for the computation time to appear. // matrix factorisation, requesting everything - wait for the computation time to appear.
( (
Routine{ Routine{
t = Main.elapsedTime; var t = Main.elapsedTime;
FluidBufNMF.process(s,b, 0,-1,0,-1,c,x,0,y,0,5,100,~frame_size,~hop_size,~fft_size); FluidBufNMF.process(s,b, 0,-1,0,-1,c,x,0,y,0,5,100,~frame_size,~hop_size,~fft_size)
.wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
//look at the resynthesised components, the bases and the activations //look at the resynthesised components, the bases and the activations
c.plot; x.plot; y.plot; c.plot; x.plot; y.plot;
//null test of the sum of sources
{(PlayBuf.ar(5,c,doneAction:2).sum)+(-1*PlayBuf.ar(1,b,doneAction:2))}.play
// play the components spread in the stereo field // play the components spread in the stereo field
{Splay.ar(PlayBuf.ar(5,c,doneAction:2))}.play {Splay.ar(PlayBuf.ar(5,c,doneAction:2))}.play
@ -196,6 +195,9 @@ c.plot; x.plot; y.plot;
//play a single source //play a single source
{PlayBuf.ar(5,c,doneAction:2)[~which_component].dup}.play {PlayBuf.ar(5,c,doneAction:2)[~which_component].dup}.play
//null test of the sum of sources
{(PlayBuf.ar(5,c,doneAction:2).sum)+(-1*PlayBuf.ar(1,b,doneAction:2))}.play
//play noise using one of the bases as filter. //play noise using one of the bases as filter.
( (
{ {
@ -251,7 +253,7 @@ b.play
// train using the first 2 seconds of the sound file // train using the first 2 seconds of the sound file
( (
Routine { Routine {
FluidBufNMF.process(s,b,0,44100*5,0,1, ~originalNMF, ~bases, components:10); FluidBufNMF.process(s,b,0,44100*5,0,1, ~originalNMF, ~bases, components:10).wait;
~originalNMF.query; ~originalNMF.query;
}.play; }.play;
) )
@ -296,7 +298,7 @@ z.do({|chan| FluidBufCompose.process(s, ~bases, startChan:chan, numChans: 1, des
//process the whole file, splitting it with the 2 trained bases //process the whole file, splitting it with the 2 trained bases
( (
Routine{ Routine{
FluidBufNMF.process(s, b, resynth: ~sortedNMF, bases: ~trainedBases, basesMode: 2, components:2); FluidBufNMF.process(s, b, resynth: ~sortedNMF, bases: ~trainedBases, basesMode: 2, components:2).wait;
~originalNMF.query; ~originalNMF.query;
}.play; }.play;
) )
@ -354,7 +356,7 @@ e.query
( (
// use the seeding basis, without updating // use the seeding basis, without updating
Routine { Routine {
FluidBufNMF.process(s, d, resynth:f, bases: e, basesMode: 2, activations:g, components:3); FluidBufNMF.process(s, d, resynth:f, bases: e, basesMode: 2, activations:g, components:3).wait;
e.query; e.query;
f.query; f.query;
g.query; g.query;
@ -373,7 +375,7 @@ g.plot;
( (
// use the seeding bases, with updating this time // use the seeding bases, with updating this time
Routine { Routine {
FluidBufNMF.process(s, d, resynth:f, bases: e, basesMode: 1, activations:g, components:3); FluidBufNMF.process(s, d, resynth:f, bases: e, basesMode: 1, activations:g, components:3).wait;
e.query; e.query;
f.query; f.query;
g.query; g.query;

@ -71,16 +71,24 @@ b = Buffer.read(s,File.realpath(FluidBufNNDSVD.class.filenameSymbol).dirname.wit
) )
//how many bases do I need to decompose the buffer with 90% accuracy //how many bases do I need to decompose the buffer with 90% accuracy
FluidBufNNDSVD.process(s, b, ~bases, ~activations, coverage: 0.9, method: 1, action: {\done.postln;}) (
Routine{
FluidBufNNDSVD.process(s, b, ~bases, ~activations, coverage: 0.9, method: 1).wait;
"% bases".format(~bases.numChannels).postln;
}.play;
)
//check how many bases we are returned: //check how many bases we are returned:
~bases.numChannels
//try the same process with less accuracy //try the same process with less accuracy
FluidBufNNDSVD.process(s, b, ~bases, ~activations, coverage: 0.5, action: {\done.postln;}) (
~bases.numChannels Routine{
FluidBufNNDSVD.process(s, b, ~bases, ~activations, coverage: 0.5).wait;
"% bases".format(~bases.numChannels).postln;
}.play
)
//use the bases to run NMF on //use the bases to run NMF on
FluidBufNMF.process(s, b, resynth: ~resynth, bases: ~bases, activations: ~activations,actMode: 2, components: ~bases.numChannels, action: {\done.postln;}) FluidBufNMF.process(s, b, resynth: ~resynth, bases: ~bases, activations: ~activations,actMode: 2, components: ~bases.numChannels, action: {\done.postln;})
{PlayBuf.ar(~resynth.numChannels, ~resynth)[1]}.play {PlayBuf.ar(~resynth.numChannels, ~resynth)[2]}.play
:: ::

@ -87,7 +87,7 @@ c = Buffer.new(s);
// with basic params, with a minimum slight length to avoid over // with basic params, with a minimum slight length to avoid over
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufNoveltySlice.process(s,b, indices: c, threshold:0.4,filterSize: 4, minSliceLength: 8); FluidBufNoveltySlice.process(s,b, indices: c, threshold:0.4,filterSize: 4, minSliceLength: 8).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -120,11 +120,13 @@ c = Buffer.new(s);
) )
// process with a given filterSize // process with a given filterSize
FluidBufNoveltySlice.process(s,b, indices: c, kernelSize:31, threshold:0.1, filterSize:1) (
Routine{
FluidBufNoveltySlice.process(s,b, indices: c, kernelSize:31, threshold:0.1, filterSize:1).wait;
//check the number of slices: it is the number of frames in the transBuf minus the boundary index. //check the number of slices: it is the number of frames in the transBuf minus the boundary index.
c.query; c.query;
}.play;
)
//play slice number 3 //play slice number 3
( (
{ {
@ -164,7 +166,7 @@ c = Buffer.new(s);
// with basic params // with basic params
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufNoveltySlice.process(s,b, indices: c, threshold:0.3); FluidBufNoveltySlice.process(s,b, indices: c, threshold:0.3).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -93,7 +93,7 @@ c = Buffer.new(s);
// with basic params // with basic params
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufOnsetSlice.process(s,b, indices: c, threshold:0.5); FluidBufOnsetSlice.process(s,b, indices: c, threshold:0.5).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -134,7 +134,7 @@ c = Buffer.new(s);
// with basic params // with basic params
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufOnsetSlice.process(s,b, indices: c, threshold:0.00001); FluidBufOnsetSlice.process(s,b, indices: c, threshold:0.00001).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -44,7 +44,7 @@ ARGUMENT:: algorithm
TABLE:: TABLE::
## 0 || Cepstrum: Returns a pitch estimate as the location of the second highest peak in the Cepstrum of the signal (after DC). ## 0 || Cepstrum: Returns a pitch estimate as the location of the second highest peak in the Cepstrum of the signal (after DC).
## 1 || Harmonic Product Spectrum: Implements the Harmonic Product Spectrum algorithm for pitch detection . See e.g. FOOTNOTE:: A. Lerch, "An Introduction to Audio Content Analysis: Applications in Signal Processing and Music Informatics." John Wiley & Sons, 2012.https://onlinelibrary.wiley.com/doi/book/10.1002/9781118393550 :: ## 1 || Harmonic Product Spectrum: Implements the Harmonic Product Spectrum algorithm for pitch detection . See e.g. FOOTNOTE:: A. Lerch, "An Introduction to Audio Content Analysis: Applications in Signal Processing and Music Informatics." John Wiley & Sons, 2012.https://onlinelibrary.wiley.com/doi/book/10.1002/9781118393550 ::
## 2 || YinFFT: Implements the frequency domain version of the YIN algorithm, as described in FOOTNOTE::P. M. Brossier, "Automatic Annotation of Musical Audio for Interactive Applications. QMUL, London, UK, 2007. :: See also https://essentia.upf.edu/documentation/reference/streaming_PitchYinFFT.html ## 2 || YinFFT: Implements the frequency domain version of the YIN algorithm, as described in FOOTNOTE::P. M. Brossier, "Automatic Annotation of Musical Audio for Interactive Applications." QMUL, London, UK, 2007. :: See also https://essentia.upf.edu/documentation/reference/streaming_PitchYinFFT.html
:: ::
ARGUMENT:: minFreq ARGUMENT:: minFreq
@ -87,7 +87,7 @@ b.play; b.plot;
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufPitch.process(s, b, features: c); FluidBufPitch.process(s, b, features: c).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -122,7 +122,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufPitch.process(s, b, features: c, minFreq:200, maxFreq:2000, unit:1); FluidBufPitch.process(s, b, features: c, minFreq:200, maxFreq:2000, unit:1).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -143,7 +143,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufPitch.process(s, b, features: c,action:{c.loadToFloatArray(action: {|x| d = x.reshape((x.size()/2).asInteger, 2)})}); FluidBufPitch.process(s, b, features: c,action:{c.loadToFloatArray(action: {|x| d = x.reshape((x.size()/2).asInteger, 2)})}).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -59,30 +59,49 @@ Nothing, as the destination buffer is declared in the function call.
EXAMPLES:: EXAMPLES::
code:: code::
// make a buffer of know qualities (
b = Buffer.sendCollection(s,1.0.series(1.1,2.0)) Routine{
// make a buffer of known qualities
b = Buffer.sendCollection(s,1.0.series(1.1,2.0));
// and a destination buffer // and a destination buffer
c = Buffer(s) c = Buffer(s);
// play with the scaling // play with the scaling
FluidBufScale.process(s, b, destination: c, inputLow: 0, inputHigh: 1, outputLow: 20, outputHigh:10) FluidBufScale.process(s, b, destination: c, inputLow: 0, inputHigh: 1, outputLow: 20, outputHigh:10).wait;
// retrieve the buffer and enjoy the results. // retrieve the buffer and enjoy the results.
c.getn(0,10,{|x|x.postln;}) c.getn(0,10,{|x|x.postln;})
}.play
)
// also works in multichannel - explore the following buffer // also works in multichannel - explore the following buffer
b = Buffer.sendCollection(s,-10.0.series(-9,10.0).scramble,2)
b.plot.plotMode_(\points)
//process //process
FluidBufScale.process(s, b, destination: c, inputLow: -20, inputHigh: 20, outputLow: 0, outputHigh:1) (
Routine{
b = Buffer.sendCollection(s,-10.0.series(-9,10.0).scramble,2);
c = Buffer(s);
s.sync;
defer{b.plot(bounds:Rect(400,400,400,400)).plotMode_(\points).bounds};
FluidBufScale.process(s, b, destination: c, inputLow: -20, inputHigh: 20, outputLow: 0, outputHigh:1).wait;
//enjoy - same shape, different range //enjoy - same shape, different range
c.plot.plotMode_(\points) defer{c.plot(bounds:Rect(800,400,400,400)).plotMode_(\points)};
}.play;
)
//also works with a subset of the input, resizing the output //also works with a subset of the input, resizing the output
b = Buffer.sendCollection(s,0.0.series(0.1,3.0).reshape(3,10).flop.flat,3) (
b.plot(separately: true).plotMode_(\points) Routine{
b = Buffer.sendCollection(s,0.0.series(0.1,3.0).reshape(3,10).flop.flat,3);
c = Buffer(s);
s.sync;
defer{b.plot(separately: true,bounds:Rect(400,400,400,400)).plotMode_(\points)};
//process //process
FluidBufScale.process(s, b, startFrame: 3,numFrames: 4,startChan: 1,numChans: 1, destination: c, inputLow: 0, inputHigh: 3, outputLow: 0, outputHigh:1) FluidBufScale.process(s, b, startFrame: 3,numFrames: 4,startChan: 1,numChans: 1, destination: c, inputLow: 0, inputHigh: 3, outputLow: 0, outputHigh:1).wait;
//enjoy //enjoy
c.plot(separately: true).plotMode_(\points) c.query;
c.query c.getn(0,4,{|x|x.postln;});
c.getn(0,4,{|x|x.postln;}) defer{c.plot(separately: true,bounds:Rect(800,400,400,400)).plotMode_(\points)};
}.play
)
:: ::

@ -102,7 +102,7 @@ d = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufSines.process(s, b, sines: c, residual:d); FluidBufSines.process(s, b, sines: c, residual:d).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -135,7 +135,7 @@ d = Buffer.new(s); e = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufSines.process(s, b, sines: d, residual:e, windowSize: 2048, hopSize: 256, fftSize: 16384); FluidBufSines.process(s, b, sines: d, residual:e, windowSize: 2048, hopSize: 256, fftSize: 16384).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -83,7 +83,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufSpectralShape.process(s, b, features: c); FluidBufSpectralShape.process(s, b, features: c).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -113,7 +113,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufSpectralShape.process(s, b, features: c); FluidBufSpectralShape.process(s, b, features: c).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -73,7 +73,7 @@ STRONG::A didactic example::
CODE:: CODE::
// make a buffer of known lenght // make a buffer of known length
b = Buffer.alloc(s,101); b = Buffer.alloc(s,101);
// add known values - here, a ramp up // add known values - here, a ramp up
@ -86,7 +86,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufStats.process(s, b, stats:c, numDerivs:1); FluidBufStats.process(s, b, stats:c, numDerivs:1).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -144,11 +144,13 @@ g= Array.new;
Routine({ Routine({
e.doAdjacentPairs({ e.doAdjacentPairs({
arg start,end; arg start,end;
FluidBufStats.processBlocking(s,c,(start/512).asInteger,((end-start)/512).max(2).asInteger,0,1,d,1, action: {d.loadToFloatArray(action: { FluidBufStats.processBlocking(s,c,(start/512).asInteger,((end-start)/512).max(2).asInteger,0,1,d,1,
action: {d.loadToFloatArray(action: {
arg array; arg array;
g = g.add(array[12]); g = g.add(array[12]);
"% % %\n".postf((start/512).asInteger,((end-start)/512).max(2).asInteger, array[12]); "% % %\n".postf((start/512).asInteger,((end-start)/512 ).max(2).asInteger, array[12]);})
})}); }
).wait;
}); });
"Done".postln; "Done".postln;
}).play; }).play;
@ -182,10 +184,11 @@ c = Buffer.new(s);
// run the stats and send back the values // run the stats and send back the values
FluidBufStats.process(s, b, stats:c, numDerivs:1, action:{c.getn(0,c.numFrames * c.numChannels,{|item|d = item; d.postln})}); FluidBufStats.process(s, b, stats:c, numDerivs:1, action:{c.getn(0,c.numFrames * c.numChannels,{|item|d = item; d.postln})});
OSCFunc.allEnabled
//looking at the result is not easy to grasp, since it is interleaved: first number is mean of L, second is mean of R, third is stddev of L, fourth is stddev or R //looking at the result is not easy to grasp, since it is interleaved: first number is mean of L, second is mean of R, third is stddev of L, fourth is stddev or R
//this will make it tidier - the first value of each line is Left, the second is Right //this will make it tidier - the first value of each line is Left, the second is Right
d.reshape(14,2).do({|x,i|["mean\t\t","stddev\t\t","skew\t\t\t", "kurtosis\t", "min\t\t\t", "median\t\t", "max\t\t\t","d-mean\t","d-stddev\t","d-skew\t\t", "d-kurtosis", "d-min\t\t", "d-median\t", "d-max\t\t"].at(i).post;x.round(0.01).postln});"".postln; d.reshape(14,2).do({|x,i|["mean\t\t","stddev\t\t","skew\t\t", "kurtosis\t\t", "min\t\t\t", "median\t\t", "max\t\t\t","d-mean\t","d-stddev\t","d-skew\t\t", "d-kurtosis", "d-min\t\t", "d-median\t", "d-max\t\t"].at(i).post;x.round(0.01).postln});"".postln;
:: ::
STRONG::Outliers and Weights:: STRONG::Outliers and Weights::

@ -56,11 +56,11 @@ CODE::
b=Buffer.alloc(s,1); b=Buffer.alloc(s,1);
// a simple call, where we query the destination buffer upon completion with the action message. // a simple call, where we query the destination buffer upon completion with the action message.
FluidBufThreadDemo.process(s, b, 1000, {|x|x.get(0,{|y|y.postln});}); FluidBufThreadDemo.process(s, b, 1000, action:{|x|x.get(0,{|y|y.postln});});
// as the 'process' returns its parent UGen, we can cancel the process easily // as the 'process' returns its instance, we can cancel the process easily
c = FluidBufThreadDemo.process(s, b, 100000, {|x|x.get(0,{|y|y.postln});}); c = FluidBufThreadDemo.process(s, b, 100000, action: {|x|x.get(0,{|y|y.postln});});
c.cancel //it stops silently for now but check the synth count going down by 1. c.cancel;
// if a simple call to the UGen is used, the progress can be monitored. The usual cmd. will cancel the job by freeing the synth. // if a simple call to the UGen is used, the progress can be monitored. The usual cmd. will cancel the job by freeing the synth.
{c = FluidBufThreadDemo.kr(b,10000, Done.freeSelf); Poll.kr(Impulse.kr(2),c);}.scope; {c = FluidBufThreadDemo.kr(b,10000, Done.freeSelf); Poll.kr(Impulse.kr(2),c);}.scope;

@ -85,7 +85,7 @@ c = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufTransientSlice.process(s,b, indices:c); FluidBufTransientSlice.process(s,b, indices:c).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -111,7 +111,7 @@ c.query;
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufTransientSlice.process(s,b, 0, 220500, 0, 1, c, 200, 2048, 1024, 1, 3, 1, 15, 30, 4410); FluidBufTransientSlice.process(s,b, 0, 220500, 0, 1, c, 200, 2048, 1024, 1, 3, 1, 15, 30, 4410).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -138,7 +138,7 @@ c = Buffer.new(s);
// with basic params // with basic params
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufTransientSlice.process(s,b, indices: c, threshFwd: 1.2); FluidBufTransientSlice.process(s,b, indices: c, threshFwd: 1.2).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -89,7 +89,7 @@ d = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufTransients.process(s,b, transients:c, residual:d); FluidBufTransients.process(s,b, transients:c, residual:d).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
); );
@ -106,7 +106,7 @@ d.play;
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufTransients.process(s,b, 0, 220500, 0, 1, c, d, 200, 2048, 1024, 1, 3, 1, 15, 30); FluidBufTransients.process(s,b, 0, 220500, 0, 1, c, d, 200, 2048, 1024, 1, 3, 1, 15, 30).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )
@ -136,7 +136,7 @@ d = Buffer.new(s); e = Buffer.new(s);
( (
Routine{ Routine{
t = Main.elapsedTime; t = Main.elapsedTime;
FluidBufTransients.process(s, b, transients: d, residual:e, threshFwd:1.2, clumpLength:40); FluidBufTransients.process(s, b, transients: d, residual:e, threshFwd:1.2, clumpLength:40).wait;
(Main.elapsedTime - t).postln; (Main.elapsedTime - t).postln;
}.play }.play
) )

@ -82,7 +82,7 @@ CODE::
s.boot; s.boot;
( (
fork{ fork{
~ds = FluidDataSet.new(s,\simple1d_1); ~ds = FluidDataSet.new(s);
~point = Buffer.alloc(s,1,1); ~point = Buffer.alloc(s,1,1);
s.sync; s.sync;
10.do{|i| 10.do{|i|
@ -90,7 +90,9 @@ fork{
~ds.addPoint(i.asString,~point,{("addPoint"+i).postln}); ~ds.addPoint(i.asString,~point,{("addPoint"+i).postln});
s.sync; s.sync;
}; };
~ds.dump; s.sync; ~ds.free; ~ds.dump;
s.sync;
~ds.free;
}; };
) )
@ -100,39 +102,39 @@ d = Dictionary.new;
d.add(\cols -> 1); d.add(\cols -> 1);
d.add(\data -> Dictionary.newFrom(10.collect{|i|[i.asString, [i.asFloat]]}.flatten)); d.add(\data -> Dictionary.newFrom(10.collect{|i|[i.asString, [i.asFloat]]}.flatten));
fork{ fork{
~ds = FluidDataSet.new(s,\simple1d_2); s.sync; ~ds = FluidDataSet.new(s); s.sync;
~ds.load(d); s.sync; ~ds.load(d); s.sync;
~ds.dump; s.sync; ~ds.free; ~ds.dump; s.sync; ~ds.free;
} }
) )
// Using synth // Using a synth
( (
~ds = FluidDataSet.new(s,\simple1d_3); ~ds = FluidDataSet.new(s);
{ {
var trig = Impulse.kr(20); var trig = Impulse.kr(20);
var count = PulseCount.kr(trig) - 1; var count = PulseCount.kr(trig) - 1;
var buf = LocalBuf(1); var buf = LocalBuf(1);
BufWr.kr(count, buf); BufWr.kr(count, buf);
FluidDataSetWr.kr(\simple1d_3, buf: buf, trig: trig); FluidDataSetWr.kr(~ds.asUGenInput, buf: buf, trig: trig);
FreeSelf.kr(count - 8); FreeSelf.kr(count - 8);
}.play.onFree{~ds.dump{|o| o.postln;~ds.free}} }.play.onFree{~ds.dump{|o| o.postln;~ds.free}}
) )
:: ::
STRONG:: Merging Datasets:: STRONG:: Merging Datasets::
s.dumpOSC
code:: code::
//this is how to add items between 2 datasets. //this is how to add items between 2 datasets.
//create 2 datasets //create 2 datasets
( (
~dsA = FluidDataSet.new(s,\simple1d_4a); ~dsA = FluidDataSet.new(s);
~dsB = FluidDataSet.new(s,\simple1d_4b); ~dsB = FluidDataSet.new(s);
) )
Dictionary.new
//feed them items with same dimensions but different labels //feed them items with same dimensions but different labels
~dsA.load(Dictionary.newFrom([\cols, 1, \data, Dictionary.newFrom([\one,[1],\two,[2]])])); ~dsA.load(Dictionary.newFrom([\cols, 1, \data, Dictionary.newFrom([\one,1,\two,2])]));
~dsB.load(Dictionary.newFrom([\cols, 1, \data, Dictionary.newFrom([\three,[3],\four,[4]])])); ~dsB.load(Dictionary.newFrom([\cols, 1, \data, Dictionary.newFrom([\three,3,\four,4])]));
~dsA.print; ~dsA.print;
~dsB.print; ~dsB.print;
@ -141,8 +143,8 @@ code::
~dsB.print; ~dsB.print;
//change the content of the dataset to shared labels //change the content of the dataset to shared labels
~dsA.load(Dictionary.newFrom([\cols, 1, \data, Dictionary.newFrom([\three,[333],\four,[444]])])); ~dsA.load(Dictionary.newFrom([\cols, 1, \data, Dictionary.newFrom([\three,333,\four,444])]));
~dsB.load(Dictionary.newFrom([\cols, 1, \data, Dictionary.newFrom([\three,[3],\four,[4]])])); ~dsB.load(Dictionary.newFrom([\cols, 1, \data, Dictionary.newFrom([\three,3,\four,4])]));
~dsA.print; ~dsA.print;
~dsB.print; ~dsB.print;

@ -108,10 +108,8 @@ EXAMPLES::
code:: code::
s.reboot; s.reboot;
// Create a DataSet with random data // Create a DataSet with random data
~dataSet= FluidDataSet(s,\help_fluid_dataset_query); ~dataSet= FluidDataSet(s);
( (
~points = 100.collect{|i|5.collect{|j|j+(i/100)}}; ~points = 100.collect{|i|5.collect{|j|j+(i/100)}};
@ -122,8 +120,9 @@ fork{
s.sync; s.sync;
~points.do{|x,i| ~points.do{|x,i|
~tmpbuf.setn(0,x); ~tmpbuf.setn(0,x);
s.sync;
~dataSet.addPoint(i,~tmpbuf); ~dataSet.addPoint(i,~tmpbuf);
s.sync // s.sync
} }
} }
) )
@ -133,11 +132,12 @@ fork{
// Prepare a FluidDataSetQuery object // Prepare a FluidDataSetQuery object
~query = FluidDataSetQuery.new; ~query = FluidDataSetQuery.new;
~out = FluidDataSet(s,\help_fluid_dataset_query_out); ~out = FluidDataSet(s);
// prepare a simple query // prepare a simple query
~query.filter(0,"<",0.04); ~query.filter(0,"<",0.04);
~query.addColumn(2); ~query.addColumn(2);
s.dumpOSC
~query.transform(~dataSet, ~out); ~query.transform(~dataSet, ~out);
// check the result // check the result

@ -48,8 +48,6 @@ A function that will run when the query returns, whose argument is an array of d
EXAMPLES:: EXAMPLES::
code:: code::
// Make a DataSet of random 2D points // Make a DataSet of random 2D points
s.reboot; s.reboot;
( (
@ -117,13 +115,40 @@ fork{
) )
:: ::
subsection:: Server Side Queries subsection:: Queries in a Synth
Input and output is done via buffers, which will need to be preallocated to the correct sizes:
LIST::
##Your input buffer should be sized to the input data dimension (2, in this example)
##Your output buffer should be the number of neighbours * output dimensionality
::
We can't simply return labels (i.e. strings) from a UGen, so the query returns the actual data points from a DataSet instead. By default, this is the FluidDataSet against which the tree was fitted. However, by passing a different dataset to code::kr::'s code::lookupDataSet:: argument instead, you can return different points, so long as the labels in the two datasets match. In this way, the FluidKDTree can be used to perform nearest neighbour mappings in a synth.
For instance, whilst fitting the tree against some n-dimensional descriptor data, our lookup dataset could use the same labels to map descriptor entries back to buffers, or locations in buffers, so that queries can be used to trigger audio.
code:: code::
//populate the lookupDataSet
(
Routine{
var inputBuffer = Buffer.alloc(s,2);
var outputBuffer = Buffer.alloc(s,10);//5 neighbours * 2D data points
s.sync;
{
var trig = Impulse.kr(4); //can go as fast as ControlRate.ir/2
var point = 2.collect{TRand.kr(0,1,trig)};
point.collect{|p,i| BufWr.kr([p],inputBuffer,i)};
~tree.kr(trig,inputBuffer,outputBuffer,5,nil);
Poll.kr(trig, BufRd.kr(1,outputBuffer,Array.iota(5)),5.collect{|i| "Neighbour" + i});
Silent.ar;
}.play;
}.play;
)
//Using a lookup data set instead:
//here we populate with numbers that are in effect the indicies, but it could be anything numerical that will be returned on the server-side and would be usable on that side //here we populate with numbers that are in effect the indicies, but it could be anything numerical that will be returned on the server-side and would be usable on that side
( (
~dsL = FluidDataSet.new(s,\kdtree_help_indices); ~dsL = FluidDataSet.new(s);
fork{ fork{
d = Dictionary.with( d = Dictionary.with(
*[\cols -> 1,\data -> Dictionary.newFrom( *[\cols -> 1,\data -> Dictionary.newFrom(
@ -132,31 +157,23 @@ fork{
~dsL.load(d, {~dsL.print}); ~dsL.load(d, {~dsL.print});
} }
) )
~dsL.asUGenInput
// instantiate a tree with a lookup dataset. Note that this 'association' has to be done at instantiation. s.dumpOSC
~tree = FluidKDTree(s,numNeighbours:5, lookupDataSet:~dsL); // Create the buffers, and make a synth, querying our tree with some random points
~tree.fit(~ds)
//set the buffers and busses needed
( (
~inputPoint = Buffer.alloc(s,2); Routine{
~predictPoint = Buffer.alloc(s,5); var inputBuffer = Buffer.alloc(s,2);
~pitchingBus = Bus.control; var outputBuffer = Buffer.alloc(s,5);//5 neighbours * 1D points
~catchingBus = Bus.control; s.sync;
)
(
~tree.inBus_(~pitchingBus).outBus_(~catchingBus).inBuffer_(~inputPoint).outBuffer_(~predictPoint);
{ {
var trig = Impulse.kr(4); //can go as fast as ControlRate.ir/2 var trig = Impulse.kr(4); //can go as fast as ControlRate.ir/2
var point = 2.collect{TRand.kr(0,1,trig)}; var point = 2.collect{TRand.kr(0,1,trig)};
point.collect{|p,i| BufWr.kr([p],~inputPoint,i)}; point.collect{|p,i| BufWr.kr([p],inputBuffer,i)};
Poll.kr(trig,point); ~tree.kr(trig,inputBuffer,outputBuffer,5,~dsL);
Out.kr(~pitchingBus.index,[trig]); Poll.kr(trig, BufRd.kr(1,outputBuffer,Array.iota(5)),5.collect{|i| "Neighbour" + i});
Poll.kr(In.kr(~catchingBus.index),BufRd.kr(1,~predictPoint,Array.iota(5)));
Silent.ar; Silent.ar;
}.play(~tree.synth,addAction:\addBefore); }.play;
}.play;
) )
:: ::

@ -87,7 +87,7 @@ fork{
// Create a KMeans instance and a LabelSet for the cluster labels in the server // Create a KMeans instance and a LabelSet for the cluster labels in the server
~clusters = FluidLabelSet(s,\kmeans_help_clusters); ~clusters = FluidLabelSet(s);
~kmeans = FluidKMeans(s); ~kmeans = FluidKMeans(s);
// Fit into 4 clusters // Fit into 4 clusters
@ -101,10 +101,15 @@ fork{
) )
// Cols of kmeans should match DataSet, size is the number of clusters // Cols of kmeans should match DataSet, size is the number of clusters
~kmeans.cols; ~kmeans.cols;
~kmeans.size; ~kmeans.size;
~kmeans.dump; ~kmeans.dump;
~clusters.getLabel(0,{|clusterID|
(0.asString+clusterID).postln;
});
// Retrieve labels of clustered points // Retrieve labels of clustered points
( (
~assignments = Array.new(128); ~assignments = Array.new(128);
@ -147,57 +152,38 @@ w.front;
~kmeans.predictPoint(~inbuf,{|x|x.postln;}); ~kmeans.predictPoint(~inbuf,{|x|x.postln;});
:: ::
subsection:: Server Side Queries subsection:: Queries in a Synth
This is the equivalent of predictPoint, but wholly on the server This is the equivalent of predictPoint, but wholly on the server
FluidKMeans is accessed via its own synth, so we need to use
a bus to communicate with it. The inBus receives a trigger to query, using data
from inBuffer; a trigger is then send to outBus with the prediction in outBuffer
code:: code::
(
~ib = Bus.control(s); // input bus
~ob = Bus.control(s); //output bus
~tempPoint = Buffer.alloc(s,1,2);
~inpPoint = Buffer.alloc(s,2);
~outPoint = Buffer.alloc(s,1);
)
//We make two Synths. One, before FluidKMeans, generates a random point and sends outputPoint.getToFloatArray(action:{|a|a.postln})
//a trigger to query. The second, after FluidKMeans, gives us the predicted cluster //triggering upadtes from the outBus
( (
//Set properties on FluidKMeans:
~kmeans.inBus_(~ib).outBus_(~ob).inBuffer_(~inpPoint).outBuffer_(~outPoint);
//pitching
{ {
var trig = Impulse.kr(5); var trig = Impulse.kr(5);
var point = WhiteNoise.kr(1.dup); var point = WhiteNoise.kr(1.dup);
var copied; var inputPoint = LocalBuf(2);
var outputPoint = LocalBuf(1);
Poll.kr(trig, point, [\pointX,\pointY]); Poll.kr(trig, point, [\pointX,\pointY]);
point.collect{ |p,i| BufWr.kr([p],~inpPoint,i)}; point.collect{ |p,i| BufWr.kr([p],inputPoint,i)};
Out.kr(~ib.index,[trig]); ~kmeans.kr(trig,inputPoint,outputPoint);
}.play(~kmeans.synth,addAction:\addBefore); Poll.kr(trig,BufRd.kr(1,outputPoint,0,interpolation:0),\cluster);
//catching }.play;
{
Poll.kr(In.kr(~ob),Latch.kr(BufRd.kr(1,~outPoint,0,interpolation:0),In.kr(~ob)),\cluster);
}.play(~kmeans.synth,addAction:\addAfter);
) )
// to sonify the output, here are random values alternating quadrant, generated more quickly as the cursor moves rightwards // to sonify the output, here are random values alternating quadrant, generated more quickly as the cursor moves rightwards
( (
//Set properties on FluidKMeans:
~kmeans.inBus_(~ib).outBus_(~ob).inBuffer_(~inpPoint).outBuffer_(~outPoint);
//pitching
{ {
var trig = Impulse.kr(MouseX.kr(0,1).exprange(0.5,ControlRate.ir / 2)); var trig = Impulse.kr(MouseX.kr(0,1).exprange(0.5,ControlRate.ir / 2));
var step = Stepper.kr(trig,max:3); var step = Stepper.kr(trig,max:3);
var point = TRand.kr(-0.1, [0.1, 0.1], trig) + [step.mod(2).linlin(0,1,-0.6,0.6),step.div(2).linlin(0,1,-0.6,0.6)] ; var point = TRand.kr(-0.1, [0.1, 0.1], trig) + [step.mod(2).linlin(0,1,-0.6,0.6),step.div(2).linlin(0,1,-0.6,0.6)] ;
point.collect{|p,i| BufWr.kr([p],~inpPoint,i)}; var inputPoint = LocalBuf(2);
Out.kr(~ib.index,[trig]); var outputPoint = LocalBuf(1);
T2A.ar(trig)*0.1; point.collect{|p,i| BufWr.kr([p],inputPoint,i)};
}.play(~kmeans.synth,addAction:\addBefore); ~kmeans.kr(trig,inputPoint,outputPoint);
//catching SinOsc.ar((BufRd.kr(1,outputPoint,0,interpolation:0) + 69).midicps,mul: 0.1);
{ }.play;
SinOsc.ar((Latch.kr(BufRd.kr(1,~outPoint,0,interpolation:0),In.kr(~ob)) + 69).midicps,mul: 0.1);
}.play(~kmeans.synth,addAction:\addAfter);
) )
:: ::

@ -57,12 +57,12 @@ code::
( (
~classifier = FluidKNNClassifier(s); ~classifier = FluidKNNClassifier(s);
~source= FluidDataSet(s,\knnclassify_help_examples); ~source= FluidDataSet(s);
~labels = FluidLabelSet(s,\knnclassify_help_labels); ~labels = FluidLabelSet(s);
~test = FluidDataSet(s,\knnclassify_help_test); ~test = FluidDataSet(s);
~mapping = FluidLabelSet(s,\knnclassify_help_mapping); ~mapping = FluidLabelSet(s);
) )
s.dumpOSC
//Make some clumped 2D points and place into a DataSet //Make some clumped 2D points and place into a DataSet
( (
~examplepoints = [[0.5,0.5],[-0.5,0.5],[0.5,-0.5],[-0.5,-0.5]]; ~examplepoints = [[0.5,0.5],[-0.5,0.5],[0.5,-0.5],[-0.5,-0.5]];
@ -149,52 +149,33 @@ w.front;
subsection::Server Side Queries subsection::Server Side Queries
This is the equivalent of predictPoint, but wholly on the server This is the equivalent of predictPoint, but wholly on the server
FluidKNNClassifier is accessed via its own synth, so we need to use
a bus to communicate with it. The inBus receives a trigger to query, using data
from inBuffer; a trigger is then send to outBus with the prediction in outBuffer
code:: code::
(
~ib = Bus.control(s); // input bus
~ob = Bus.control(s); //output bus
~inpPoint = Buffer.alloc(s,2);
~outPoint = Buffer.alloc(s,1);
)
//We make two Synths. One, before FluidKMeans, generates a random point and sends //Generate a random point and sends a trigger to query, and return the class that point matches
//a trigger to query. The second, after FluidKMeans, gives us the predicted cluster //triggering upadtes from the outBus
( (
//Set properties on FluidKNNClassifier:
~classifier.inBus_(~ib).outBus_(~ob).inBuffer_(~inpPoint).outBuffer_(~outPoint);
//pitching
{ {
var trig = Impulse.kr(5); var trig = Impulse.kr(5);
var point = WhiteNoise.kr(1.dup); var point = WhiteNoise.kr(1.dup);
var inputPoint = LocalBuf(2);
var outputPoint = LocalBuf(1);
Poll.kr(trig, point, [\pointX,\pointY]); Poll.kr(trig, point, [\pointX,\pointY]);
point.collect{ |p,i| BufWr.kr([p],~inpPoint,i)}; point.collect{ |p,i| BufWr.kr([p],inputPoint,i)};
Out.kr(~ib.index,[trig]); ~classifier.kr(trig,inputPoint,outputPoint);
}.play(~classifier.synth,addAction:\addBefore); Poll.kr(trig,BufRd.kr(1,outputPoint,0,interpolation:0),\cluster);
//catching }.play;
{
Poll.kr(In.kr(~ob),Latch.kr(BufRd.kr(1,~outPoint,0,interpolation:0),In.kr(~ob)),\cluster);
}.play(~classifier.synth,addAction:\addAfter);
) )
// to sonify the output, here are random values alternating quadrant. // to sonify the output, here are random values alternating quadrant.
( (
//Set properties on FluidKNNClassifier:
~classifier.inBus_(~ib).outBus_(~ob).inBuffer_(~inpPoint).outBuffer_(~outPoint);
//pitching
{ {
var trig = Impulse.kr(MouseX.kr(0,1).exprange(0.5,ControlRate.ir /2).poll(trig:2, label: "Query Frequency")); var trig = Impulse.kr(MouseX.kr(0,1).exprange(0.5,ControlRate.ir /2).poll(trig:2, label: "Query Frequency"));
var step = Stepper.kr(trig,max:3); var step = Stepper.kr(trig,max:3);
var point = TRand.kr(-0.1, [0.1, 0.1], trig) + [step.mod(2).linlin(0,1,-0.6,0.6),step.div(2).linlin(0,1,-0.6,0.6)] ; var point = TRand.kr(-0.1, [0.1, 0.1], trig) + [step.mod(2).linlin(0,1,-0.6,0.6),step.div(2).linlin(0,1,-0.6,0.6)] ;
point.collect{|p,i| BufWr.kr([p],~inpPoint,i)}; var inputPoint = LocalBuf(2);
Out.kr(~ib.index,[trig]); var outputPoint = LocalBuf(1);
T2A.ar(trig)*0.1; point.collect{|p,i| BufWr.kr([p],inputPoint,i)};
}.play(~classifier.synth,addAction:\addBefore); ~classifier.kr(trig,inputPoint,outputPoint);
//catching SinOsc.ar((BufRd.kr(1,outputPoint,0,interpolation:0) + 69).midicps,mul: 0.1);
{ }.play
SinOsc.ar((Latch.kr(BufRd.kr(1,~outPoint,0,interpolation:0),In.kr(~ob)) + 69).midicps,mul: 0.1);
}.play(~classifier.synth,addAction:\addAfter);
) )
:: ::

@ -53,10 +53,10 @@ code::
//Make a simple mapping between a ramp and a sine cycle, test with an exponentional ramp //Make a simple mapping between a ramp and a sine cycle, test with an exponentional ramp
( (
~source = FluidDataSet(s,\knn_regress_src); ~source = FluidDataSet(s);
~target = FluidDataSet(s,\knn_regress_tgt); ~target = FluidDataSet(s);
~test = FluidDataSet(s,\knn_regress_test); ~test = FluidDataSet(s);
~output = FluidDataSet(s,\knn_regress_out); ~output = FluidDataSet(s);
~tmpbuf = Buffer.alloc(s,1); ~tmpbuf = Buffer.alloc(s,1);
~regressor = FluidKNNRegressor(s); ~regressor = FluidKNNRegressor(s);
) )
@ -106,7 +106,7 @@ d = Dictionary.with(
//We should see a single cycle of a chirp //We should see a single cycle of a chirp
~outputdata.plot; ~outputdata.plot;
s.dumpOSC
// single point transform on arbitrary value // single point transform on arbitrary value
~inbuf = Buffer.loadCollection(s,[0.5]); ~inbuf = Buffer.loadCollection(s,[0.5]);
~regressor.predictPoint(~inbuf,{|x|x.postln;}); ~regressor.predictPoint(~inbuf,{|x|x.postln;});
@ -117,28 +117,15 @@ subsection:: Server Side Queries
code:: code::
//Setup //Setup
( (
~inputPoint = Buffer.alloc(s,1); {
~predictPoint = Buffer.alloc(s,1);
~pitchingBus = Bus.control;
~catchingBus = Bus.control;
)
(
~regressor.inBus_(~pitchingBus).outBus_(~catchingBus).inBuffer_(~inputPoint).outBuffer_(~predictPoint);
~inputSynth = {
var input = Saw.kr(2).linlin(-1,1,0,1); var input = Saw.kr(2).linlin(-1,1,0,1);
var trig = Impulse.kr(ControlRate.ir/10); var trig = Impulse.kr(ControlRate.ir/10);
BufWr.kr(input,~inputPoint,0); var inputPoint = LocalBuf(1);
Out.kr(~pitchingBus.index,[trig]); var outputPoint = LocalBuf(1);
}; BufWr.kr(input,inputPoint,0);
~regressor.kr(trig,inputPoint,outputPoint);
~inputSynth.play(~regressor.synth,addAction:\addBefore); BufRd.kr(1,outputPoint,0);//,"mapped value")
}.scope
~outputSynth = {
Poll.kr(In.kr(~catchingBus.index),BufRd.kr(1,~predictPoint,0),"mapped value")
};
~outputSynth.play(~regressor.synth,addAction:\addAfter);
~outputSynth.scope
) )
:: ::

@ -56,7 +56,7 @@ Post an abbreviated content of the label set in the window by default, but you c
EXAMPLES:: EXAMPLES::
code:: code::
~ls = FluidLabelSet.new(s,\labelset_example); ~ls = FluidLabelSet.new(s);
["one", "two", "three"].collect{|x,i| ~ls.addLabel(i, x);}; ["one", "two", "three"].collect{|x,i| ~ls.addLabel(i, x);};
~ls.print; ~ls.print;

@ -73,7 +73,6 @@ code::
~mds = FluidMDS(s); ~mds = FluidMDS(s);
) )
// Load audio and run an mfcc analysis, which gives us 13 points (we'll throw the 0th away) // Load audio and run an mfcc analysis, which gives us 13 points (we'll throw the 0th away)
( (
~audio = Buffer.read(s,~audiofile); ~audio = Buffer.read(s,~audiofile);
@ -90,14 +89,14 @@ FluidBufMFCC.process(s,~audio, features: ~mfcc_feature);
var chunkLen = (~mfcc_feature.numFrames / 100).asInteger; var chunkLen = (~mfcc_feature.numFrames / 100).asInteger;
var stats = FluidBufStats.kr( var stats = FluidBufStats.kr(
source: ~mfcc_feature, startFrame: count * chunkLen, source: ~mfcc_feature, startFrame: count * chunkLen,
startChan:1, numFrames: chunkLen, stats: ~stats, trig: trig startChan:1, numFrames: chunkLen, stats: ~stats, trig: trig,blocking:1
); );
var rd = BufRd.kr(12, ~stats, DC.kr(0), 0, 1); var rd = BufRd.kr(12, ~stats, DC.kr(0), 0, 1);
var bufWr, dsWr; var bufWr, dsWr;
12.do{|i| 12.do{|i|
bufWr = BufWr.kr(rd[i], buf, DC.kr(i)); bufWr = BufWr.kr(rd[i], buf, DC.kr(i));
}; };
dsWr = FluidDataSetWr.kr(\mds_help_12D, buf: buf, trig: Done.kr(stats)); dsWr = FluidDataSetWr.kr(~raw, buf: buf, trig: Done.kr(stats),blocking:1);
LocalOut.kr(Done.kr(dsWr)); LocalOut.kr(Done.kr(dsWr));
FreeSelf.kr(count - 99); FreeSelf.kr(count - 99);
Poll.kr(trig,count); Poll.kr(trig,count);
@ -109,9 +108,11 @@ FluidBufMFCC.process(s,~audio, features: ~mfcc_feature);
//Then apply the MDS in-place on the standardized data to get 2 dimensions, using a Euclidean distance metric //Then apply the MDS in-place on the standardized data to get 2 dimensions, using a Euclidean distance metric
//Download the DataSet contents into an array for plotting //Download the DataSet contents into an array for plotting
( (
~reducedarray = Array.new(100); ~reducedarray = Array.new(100);
~standardizer.fitTransform(~raw, ~standardized); ~standardizer.fitTransform(~raw, ~standardized);
~mds.fitTransform(~standardized, ~reduced, action:{ ~mds.fitTransform(~standardized, ~reduced, action:{
"HETE".postln;
~reduced.dump{|x| 100.do{|i| ~reduced.dump{|x| 100.do{|i|
~reducedarray.add(x["data"][i.asString]) ~reducedarray.add(x["data"][i.asString])
}}}); }}});
@ -175,5 +176,4 @@ w.drawFunc = {
w.refresh; w.refresh;
w.front; w.front;
) )
:: ::

@ -86,11 +86,11 @@ EXAMPLES::
code:: code::
( (
~classifier = FluidMLPClassifier(s, [6], FluidMLPClassifier.tanh, 1000, 0.1, 0.1, 50, 0); ~classifier=FluidMLPClassifier(s).hidden_([6]).activation_(FluidMLPClassifier.tanh).maxIter_(1000).learnRate_(0.1).momentum_(0.1).batchSize_(50).validation_(0);
~sourcedata= FluidDataSet(s,\mlpclassify_help_examples); ~sourcedata= FluidDataSet(s);
~labels = FluidLabelSet(s,\mlpclassify_help_labels); ~labels = FluidLabelSet(s);
~testdata = FluidDataSet(s,\mlpclassify_help_test); ~testdata = FluidDataSet(s);
~predictedlabels = FluidLabelSet(s,\mlpclassify_help_mapping); ~predictedlabels = FluidLabelSet(s);
) )
//Make some clumped 2D points and place into a DataSet //Make some clumped 2D points and place into a DataSet
( (
@ -104,8 +104,8 @@ code::
~labeldata.put("mlpclass"++i++\_++j,[~categories[i]]); ~labeldata.put("mlpclass"++i++\_++j,[~categories[i]]);
} }
}; };
~sourcedata.load(Dictionary.with(*[\cols->2,\data->~trainingset])); ~sourcedata.load(Dictionary.with(*[\cols->2,\data->~trainingset]),action:{~sourcedata.print});
~labels.load(Dictionary.with(*[\cols->1,\data->~labeldata])); ~labels.load(Dictionary.with(*[\cols->1,\data->~labeldata]),action:{~labels.print});
) )
//Fit the classifier to the example DataSet and labels, and then run prediction on the test data into our mapping label set //Fit the classifier to the example DataSet and labels, and then run prediction on the test data into our mapping label set
@ -119,14 +119,16 @@ code::
~testset.put("mlpclass_test"++i++\_++j, ~centroids[i].collect{|x| x.gauss(0.5/3)}); ~testset.put("mlpclass_test"++i++\_++j, ~centroids[i].collect{|x| x.gauss(0.5/3)});
} }
}; };
~testdata.load(Dictionary.with(*[\cols->2,\data->~testset])); ~testdata.load(Dictionary.with(*[\cols->2,\data->~testset]),action:{~testdata.print});
) )
//Run the test data through the network, into the predicted labelset //Run the test data through the network, into the predicted labelset
~classifier.predict(~testdata,~predictedlabels,action:{"Test complete".postln}); ~classifier.predict(~testdata,~predictedlabels,action:{"Test complete".postln});
OSCFunc.trace(true,true)
OSCFunc.allEnabled
//get labels from server //get labels from server
~predictedlabels.dump(action:{|d| ~labelsdict = d["data"]}); ~predictedlabels.dump(action:{|d|~labelsdict = d["data"]};~labelsdict.postln);
//Visualise: we're hoping to see colours neatly mapped to quandrants... //Visualise: we're hoping to see colours neatly mapped to quandrants...
( (
c = Dictionary(); c = Dictionary();
@ -155,56 +157,34 @@ w.front;
~inbuf = Buffer.loadCollection(s,0.5.dup); ~inbuf = Buffer.loadCollection(s,0.5.dup);
~classifier.predictPoint(~inbuf,{|x|x.postln;}); ~classifier.predictPoint(~inbuf,{|x|x.postln;});
:: ::
subsection::Querying in a Synth
This is the equivalent of code::predictPoint::, but wholly on the server
subsection::Server Side Queries
This is the equivalent of predictPoint, but wholly on the server
FluidMLPClassifier is accessed via its own synth, so we need to use
a bus to communicate with it. The inBus receives a trigger to query, using data
from inBuffer; a trigger is then send to outBus with the prediction in outBuffer
code:: code::
( (
~ib = Bus.control(s); // input bus
~ob = Bus.control(s); //output bus
~inpPoint = Buffer.alloc(s,2);
~outPoint = Buffer.alloc(s,1);
)
//We make two Synths. One, before FluidKMeans, generates a random point and sends
//a trigger to query. The second, after FluidKMeans, gives us the predicted cluster //triggering upadtes from the outBus
(
//Set properties on FluidMLPClassifier:
~classifier.inBus_(~ib).outBus_(~ob).inBuffer_(~inpPoint).outBuffer_(~outPoint);
//pitching
{ {
var trig = Impulse.kr(5); var trig = Impulse.kr(5);
var point = WhiteNoise.kr(1.dup); var point = WhiteNoise.kr(1.dup);
var inputPoint = LocalBuf(2);
var outputPoint = LocalBuf(1);
Poll.kr(trig, point, [\pointX,\pointY]); Poll.kr(trig, point, [\pointX,\pointY]);
point.collect{ |p,i| BufWr.kr([p],~inpPoint,i)}; point.collect{ |p,i| BufWr.kr([p],inputPoint,i)};
Out.kr(~ib.index,[trig]); ~classifier.kr(trig,inputPoint,outputPoint);
}.play(~classifier.synth,addAction:\addBefore); Poll.kr(trig,BufRd.kr(1,outputPoint,0,interpolation:0),\cluster);
//catching Silent.ar;
{ }.play;
Poll.kr(In.kr(~ob),Latch.kr(BufRd.kr(1,~outPoint,0,interpolation:0),In.kr(~ob)),\cluster);
}.play(~classifier.synth,addAction:\addAfter);
) )
// to sonify the output, here are random values alternating quadrant. // to sonify the output, here are random values alternating quadrant.
( (
//Set properties on FluidMLPClassifier:
~classifier.inBus_(~ib).outBus_(~ob).inBuffer_(~inpPoint).outBuffer_(~outPoint);
//pitching
{ {
var trig = Impulse.kr(MouseX.kr(0,1).exprange(0.5,ControlRate.ir /2).poll(trig: 2,label: "Query Frequency")); var trig = Impulse.kr(MouseX.kr(0,1).exprange(0.5,ControlRate.ir /2).poll(trig: 2,label: "Query Frequency"));
var step = Stepper.kr(trig,max:3); var step = Stepper.kr(trig,max:3);
var point = TRand.kr(-0.1, [0.1, 0.1], trig) + [step.mod(2).linlin(0,1,-0.6,0.6),step.div(2).linlin(0,1,-0.6,0.6)] ; var point = TRand.kr(-0.1, [0.1, 0.1], trig) + [step.mod(2).linlin(0,1,-0.6,0.6),step.div(2).linlin(0,1,-0.6,0.6)] ;
point.collect{|p,i| BufWr.kr([p],~inpPoint,i)}; var inputPoint = LocalBuf(2);
Out.kr(~ib.index,[trig]); var outputPoint = LocalBuf(1);
T2A.ar(trig)*0.1; point.collect{|p,i| BufWr.kr([p],inputPoint,i)};
}.play(~classifier.synth,addAction:\addBefore); ~classifier.kr(trig,inputPoint,outputPoint);
//catching SinOsc.ar((BufRd.kr(1,outputPoint,0,interpolation:0) + 69).midicps,mul: 0.1)
{ }.play;
SinOsc.ar((Latch.kr(BufRd.kr(1,~outPoint,0,interpolation:0),In.kr(~ob)) + 69).midicps,mul: 0.1);
}.play(~classifier.synth,addAction:\addAfter);
) )
:: ::

@ -101,12 +101,12 @@ code::
//Make a simple mapping between a ramp and a sine cycle, test with an exponentional ramp //Make a simple mapping between a ramp and a sine cycle, test with an exponentional ramp
( (
~source = FluidDataSet(s,\mlp_regressor_source); ~source = FluidDataSet(s);
~target = FluidDataSet(s,\mlp_regressor_target); ~target = FluidDataSet(s);
~test = FluidDataSet(s,\mlp_regressor_dest); ~test = FluidDataSet(s);
~output = FluidDataSet(s,\mlp_regress_out); ~output = FluidDataSet(s);
~tmpbuf = Buffer.alloc(s,1); ~tmpbuf = Buffer.alloc(s,1);
~regressor = FluidMLPRegressor(s,[2], FluidMLPRegressor.tanh, FluidMLPRegressor.tanh, maxIter: 1000, learnRate: 0.1, momentum: 0.1, batchSize: 1, validation: 0); ~regressor=FluidMLPRegressor(s).hidden_([2]).activation_(FluidMLPRegressor.tanh).outputActivation_(FluidMLPRegressor.tanh).maxIter_(1000).learnRate_(0.1).momentum_(0.1).batchSize_(1).validation_(0);
) )
//Make source, target and test data //Make source, target and test data
@ -156,7 +156,8 @@ d = Dictionary.with(
~output.dump{|x| 128.do{|i| ~output.dump{|x| 128.do{|i|
~outputdata.add(x["data"][i.asString][0]) ~outputdata.add(x["data"][i.asString][0])
}}; }};
}); }
);
) )
//We should see a single cycle of a chirp. If not, fit a little more epochs //We should see a single cycle of a chirp. If not, fit a little more epochs
@ -166,37 +167,23 @@ d = Dictionary.with(
~inbuf = Buffer.loadCollection(s,[0.5]); ~inbuf = Buffer.loadCollection(s,[0.5]);
~outbuf = Buffer.new(s); ~outbuf = Buffer.new(s);
~regressor.predictPoint(~inbuf,~outbuf,{|x|x.postln;x.getn(0,1,{|y|y.postln;};)}); ~regressor.predictPoint(~inbuf,~outbuf,{|x|x.postln;x.getn(0,1,{|y|y.postln;};)});
:: ::
subsection:: Server Side Queries subsection:: Querying in a Synth
code:: This is the equivalent of calling code::predictPoint::, except wholly on the server
//Setup
(
~inputPoint = Buffer.alloc(s,1);
~predictPoint = Buffer.alloc(s,1);
~pitchingBus = Bus.control;
~catchingBus = Bus.control;
)
code::
( (
~regressor.inBus_(~pitchingBus).outBus_(~catchingBus).inBuffer_(~inputPoint).outBuffer_(~predictPoint); {
~inputSynth = {
var input = Saw.kr(2).linlin(-1,1,0,1); var input = Saw.kr(2).linlin(-1,1,0,1);
var trig = Impulse.kr(ControlRate.ir/10); var trig = Impulse.kr(ControlRate.ir/10);
BufWr.kr(input,~inputPoint,0); var inputPoint = LocalBuf(1);
Out.kr(~pitchingBus.index,[trig]); var outputPoint = LocalBuf(1);
}; BufWr.kr(input,inputPoint,0);
~regressor.kr(trig,inputPoint,outputPoint,0,-1);
~inputSynth.play(~regressor.synth,addAction:\addBefore); Poll.kr(trig,BufRd.kr(1,outputPoint,0),"mapped value");
}.scope;
~outputSynth = {
Poll.kr(In.kr(~catchingBus.index),BufRd.kr(1,~predictPoint,0),"mapped value")
};
~outputSynth.play(~regressor.synth,addAction:\addAfter);
~outputSynth.scope
) )

@ -79,8 +79,7 @@ d.play //////(beware !!!! loud!!!)
( (
// separate them in 2 components // separate them in 2 components
Routine { Routine {
FluidBufNMF.process(s, d, bases: e, components:2); FluidBufNMF.process(s, d, bases: e, components:2).wait;
s.sync;
e.query; e.query;
}.play }.play
) )
@ -112,7 +111,7 @@ c = Buffer.new(s);
// train only 2 seconds // train only 2 seconds
( (
Routine { Routine {
FluidBufNMF.process(s,b,0,88200,0,1, c, ~bases, components:10,fftSize:2048); FluidBufNMF.process(s,b,0,88200,0,1, c, ~bases, components:10,fftSize:2048).wait;
c.query; c.query;
}.play; }.play;
) )
@ -120,12 +119,13 @@ Routine {
// wait for the query to print // wait for the query to print
// find the component that has the picking sound checking the median spectral centroid // find the component that has the picking sound checking the median spectral centroid
( (
FluidBufSpectralShape.process(s, c, features: ~spectralshapes, action:{ FluidBufSpectralShape.process(s, c, features: ~spectralshapes, action:{ |shapes|
|shapes|FluidBufStats.process(s,shapes,stats:~stats, action:{ FluidBufStats.process(s,shapes,stats:~stats, action:{|stats|
|stats|stats.getn(0, (stats.numChannels * stats.numFrames) ,{ stats.getn(0, (stats.numChannels * stats.numFrames), {|x|
|x| ~centroids = x.select({ ~centroids = x.select({
|item, index| (index.mod(7) == 0) && (index.div(70) == 5); |item, index| (index.mod(7) == 0) && (index.div(70) == 5);
}) });
~centroids.postln;
}) })
}) })
}); });

@ -92,18 +92,18 @@ FluidBufPitch.process(s,~audio, features: ~pitch_feature);
var chunkLen = (~pitch_feature.numFrames / 10).asInteger; var chunkLen = (~pitch_feature.numFrames / 10).asInteger;
var stats = FluidBufStats.kr( var stats = FluidBufStats.kr(
source: ~pitch_feature, startFrame: count * chunkLen, source: ~pitch_feature, startFrame: count * chunkLen,
numFrames: chunkLen, stats: ~stats, trig: trig numFrames: chunkLen, stats: ~stats, trig: (trig * (count <=9)), blocking:1
); );
var rd = BufRd.kr(2, ~stats, DC.kr(0), 0, 1);// pick only mean pitch and confidence var rd = BufRd.kr(2, ~stats, DC.kr(0), 0, 1);// pick only mean pitch and confidence
var wr1 = BufWr.kr(rd[0], buf, DC.kr(0)); var wr1 = BufWr.kr(rd[0], buf, DC.kr(0));
var wr2 = BufWr.kr(rd[1], buf, DC.kr(1)); var wr2 = BufWr.kr(rd[1], buf, DC.kr(1));
var dsWr = FluidDataSetWr.kr(\norm_help_raw, buf: buf, trig: Done.kr(stats)); var dsWr = FluidDataSetWr.kr(~raw, buf: buf, trig: Done.kr(stats));
LocalOut.kr( Done.kr(dsWr)); LocalOut.kr( Done.kr(dsWr));
Poll.kr(trig,count,\count);
FreeSelf.kr(count - 9); FreeSelf.kr(count - 9);
}.play; }.play;
) )
// Normalize and load to language-side array // Normalize and load to language-side array
( (
~rawarray = Array.new(10); ~rawarray = Array.new(10);
@ -130,42 +130,28 @@ FluidBufPitch.process(s,~audio, features: ~pitch_feature);
~inbuf = Buffer.loadCollection(s,0.5.dup); ~inbuf = Buffer.loadCollection(s,0.5.dup);
~outbuf = Buffer.new(s); ~outbuf = Buffer.new(s);
~normalizer.transformPoint(~inbuf,~outbuf,{|x|x.postln;x.getn(0,2,{|y|y.postln;};)}); ~normalizer.transformPoint(~inbuf,~outbuf,{|x|x.postln;x.getn(0,2,{|y|y.postln;};)});
OSCFunc.trace(false,true)
//Server side queries //Server side queries
//Setup
(
~tempPoint = Buffer.alloc(s,2);
~predictPoint = Buffer.alloc(s,2);
~avgBuf = Buffer.alloc(s,100,2);
~pitchingBus = Bus.control;
~catchingBus = Bus.control;
)
( (
~normalizer.inBus_(~pitchingBus).outBus_(~catchingBus).inBuffer_(~tempPoint).outBuffer_(~predictPoint);
//Pitching (no pun intended): read frames out of buffer and pass to standardize
{ {
var audio = BufRd.ar(1,~audio,LFSaw.ar(BufDur.ir(~audio).reciprocal).range(0, BufFrames.ir(~audio))); var audio = BufRd.ar(1,~audio,LFSaw.ar(BufDur.ir(~audio).reciprocal).range(0, BufFrames.ir(~audio)));
var counter = Stepper.ar(Impulse.ar(ControlRate.ir),max:99); var counter = Stepper.ar(Impulse.ar(ControlRate.ir),max:99);
var trig = HPZ1.ar(counter) < 0; var trig = A2K.kr(HPZ1.ar(counter) < 0);
//average 100 frames: one could use the MovingAverage extension here //average 100 frames: one could use the MovingAverage extension here
var avg; var avg;
BufWr.kr(FluidPitch.kr(audio),~avgBuf,phase:counter); var inputPoint = LocalBuf(2);
avg = Mix.new(BufRd.kr(2, ~avgBuf, phase:100.collect{|x|x})) * 0.01; var outputPoint = LocalBuf(2);
var avgBuf = LocalBuf(100,2);
//running average of pitch features
BufWr.kr(FluidPitch.kr(audio),avgBuf,phase:counter);
avg = Mix.new(BufRd.kr(2, avgBuf, phase:100.collect{|x|x})) * 0.01;
//assemble data point //assemble data point
BufWr.kr(avg[0],~tempPoint,0); BufWr.kr(avg[0],inputPoint,0);
BufWr.kr(avg[1],~tempPoint,1); BufWr.kr(avg[1],inputPoint,1);
Poll.kr(T2K.kr(trig),BufRd.kr(1,~tempPoint,[0,1]),["pitch (raw)", "confidence (raw)"]); ~normalizer.kr(trig,inputPoint,outputPoint);
Out.kr(~pitchingBus.index,[T2K.kr(trig)]); Poll.kr(trig,BufRd.kr(1,inputPoint,[0,1]),["pitch (raw)", "confidence (raw)"]);
}.play(~normalizer.synth,addAction:\addBefore); Poll.kr(trig,BufRd.kr(1,outputPoint,[0,1]),["pitch (normalized)", "confidence (normalized)"])
}.play;
{
Poll.kr(In.kr(~catchingBus.index),BufRd.kr(1,~predictPoint,[0,1]),["pitch (normalized)", "confidence (normalized)"])
}.play(~normalizer.synth,addAction:\addAfter);
) )
:: ::

@ -63,9 +63,9 @@ s.boot;
//Preliminaries: we want some audio, a couple of FluidDataSets, some Buffers, a FluidStandardize and a FluidPCA //Preliminaries: we want some audio, a couple of FluidDataSets, some Buffers, a FluidStandardize and a FluidPCA
( (
~audiofile = File.realpath(FluidBufPitch.class.filenameSymbol).dirname +/+ "../AudioFiles/Tremblay-ASWINE-ScratchySynth-M.wav"; ~audiofile = File.realpath(FluidBufPitch.class.filenameSymbol).dirname +/+ "../AudioFiles/Tremblay-ASWINE-ScratchySynth-M.wav";
~raw = FluidDataSet(s,\pca_help_12D); ~raw = FluidDataSet(s);
~standardized = FluidDataSet(s,\pca_help_12Ds); ~standardized = FluidDataSet(s);
~reduced = FluidDataSet(s,\pca_help_2D); ~reduced = FluidDataSet(s);
~audio = Buffer.read(s,~audiofile); ~audio = Buffer.read(s,~audiofile);
~mfcc_feature = Buffer.new(s); ~mfcc_feature = Buffer.new(s);
~stats = Buffer.alloc(s, 7, 12); ~stats = Buffer.alloc(s, 7, 12);
@ -78,7 +78,7 @@ s.boot;
// Load audio and run an mfcc analysis, which gives us 13 points (we'll throw the 0th away) // Load audio and run an mfcc analysis, which gives us 13 points (we'll throw the 0th away)
( (
~audio = Buffer.read(s,~audiofile); ~audio = Buffer.read(s,~audiofile);
FluidBufMFCC.process(s,~audio, features: ~mfcc_feature); FluidBufMFCC.process(s,~audio, features: ~mfcc_feature,action:{"Done MFCCs".postln});
) )
// Divide the time series in 100, and take the mean of each segment and add this as a point to // Divide the time series in 100, and take the mean of each segment and add this as a point to
@ -91,14 +91,15 @@ FluidBufMFCC.process(s,~audio, features: ~mfcc_feature);
var chunkLen = (~mfcc_feature.numFrames / 100).asInteger; var chunkLen = (~mfcc_feature.numFrames / 100).asInteger;
var stats = FluidBufStats.kr( var stats = FluidBufStats.kr(
source: ~mfcc_feature, startFrame: count * chunkLen, source: ~mfcc_feature, startFrame: count * chunkLen,
startChan:1, numFrames: chunkLen, stats: ~stats, trig: trig startChan:1, numFrames: chunkLen, stats: ~stats,
trig: trig * (count < 100), blocking: 1
); );
var rd = BufRd.kr(12, ~stats, DC.kr(0), 0, 1); var rd = BufRd.kr(12, ~stats, DC.kr(0), 0, 1);
var bufWr, dsWr; var bufWr, dsWr;
12.do{|i| 12.do{|i|
bufWr = BufWr.kr(rd[i], buf, DC.kr(i)); bufWr = BufWr.kr(rd[i], buf, DC.kr(i));
}; };
dsWr = FluidDataSetWr.kr(\pca_help_12D, buf: buf, trig: Done.kr(stats)); dsWr = FluidDataSetWr.kr(~raw, buf: buf, trig: Done.kr(stats));
LocalOut.kr( Done.kr(dsWr)); LocalOut.kr( Done.kr(dsWr));
FreeSelf.kr(count - 99); FreeSelf.kr(count - 99);
Poll.kr(trig,count); Poll.kr(trig,count);
@ -150,29 +151,19 @@ subsection:: Server Side Queries
Let's map our learned PCA dimensions to the controls of a processor Let's map our learned PCA dimensions to the controls of a processor
code:: code::
(
~inputPoint = Buffer.alloc(s,12);
~predictPoint = Buffer.alloc(s,2);
~pitchingBus = Bus.control;
~catchingBus = Bus.control;
)
( (
~pca.inBus_(~pitchingBus).outBus_(~catchingBus).inBuffer_(~inputPoint).outBuffer_(~predictPoint);
{ {
var mapped; var mapped;
var audio = BufRd.ar(1,~audio,LFSaw.ar(BufDur.ir(~audio).reciprocal).range(0, BufFrames.ir(~audio))); var audio = BufRd.ar(1,~audio,LFSaw.ar(BufDur.ir(~audio).reciprocal).range(0, BufFrames.ir(~audio)));
var mfcc = FluidMFCC.kr(audio)[1..12]; var mfcc = FluidMFCC.kr(audio)[1..12];
var smoothed = LagUD.kr(mfcc,1*ControlDur.ir,500*ControlDur.ir); var smoothed = LagUD.kr(mfcc,1*ControlDur.ir,500*ControlDur.ir);
var trig = Impulse.kr(ControlRate.ir / 2); var trig = Impulse.kr(ControlRate.ir / 2);
smoothed.collect{|coeff,i| BufWr.kr([coeff],~inputPoint,i)}; var inputPoint = LocalBuf(12);
Out.kr(~pitchingBus,[trig]); var outputPoint = LocalBuf(2);
mapped = Latch.kr(BufRd.kr(1,~predictPoint, phase:[0,1]).linlin(-3,3,0,3),In.kr(~catchingBus)).scope; smoothed.collect{|coeff,i| BufWr.kr([coeff],inputPoint,i)};
~pca.kr(trig, inputPoint, outputPoint, 2);
mapped = BufRd.kr(1,outputPoint, phase:[0,1]).linlin(-3,3,0,3);
CombC.ar(audio,3,mapped[0],mapped[1]*3) CombC.ar(audio,3,mapped[0],mapped[1]*3)
}.play(~pca.synth,addAction:\addBefore); }.play;
) )
:: ::

@ -59,8 +59,8 @@ s.boot;
//Preliminaries: we want some audio, a couple of FluidDataSets, some Buffers and a FluidStandardize //Preliminaries: we want some audio, a couple of FluidDataSets, some Buffers and a FluidStandardize
( (
~audiofile = File.realpath(FluidBufPitch.class.filenameSymbol).dirname +/+ "../AudioFiles/Tremblay-ASWINE-ScratchySynth-M.wav"; ~audiofile = File.realpath(FluidBufPitch.class.filenameSymbol).dirname +/+ "../AudioFiles/Tremblay-ASWINE-ScratchySynth-M.wav";
~raw = FluidDataSet(s,\stand_help_raw); ~raw = FluidDataSet(s);
~stand = FluidDataSet(s,\stand_help_standd); ~stand = FluidDataSet(s);
~audio = Buffer.read(s,~audiofile); ~audio = Buffer.read(s,~audiofile);
~pitch_feature = Buffer.new(s); ~pitch_feature = Buffer.new(s);
~stats = Buffer.alloc(s, 7, 2); ~stats = Buffer.alloc(s, 7, 2);
@ -71,10 +71,10 @@ s.boot;
// Load audio and run a pitch analysis, which gives us pitch and pitch confidence (so a 2D datum) // Load audio and run a pitch analysis, which gives us pitch and pitch confidence (so a 2D datum)
( (
~audio = Buffer.read(s,~audiofile); ~audio = Buffer.read(s,~audiofile);
FluidBufPitch.process(s,~audio, features: ~pitch_feature); FluidBufPitch.process(s,~audio, features: ~pitch_feature,action:{"Analysed Pitch".postln});
) )
// Divide the time series in to 10, and take the mean of each segment and add this as a point to // Divide the time series in to 10, and take the mean of each s"egment and add this as a point to
// the 'raw' FluidDataSet // the 'raw' FluidDataSet
( (
{ {
@ -84,18 +84,19 @@ FluidBufPitch.process(s,~audio, features: ~pitch_feature);
var chunkLen = (~pitch_feature.numFrames / 10).asInteger; var chunkLen = (~pitch_feature.numFrames / 10).asInteger;
var stats = FluidBufStats.kr( var stats = FluidBufStats.kr(
source: ~pitch_feature, startFrame: count * chunkLen, source: ~pitch_feature, startFrame: count * chunkLen,
numFrames: chunkLen, stats: ~stats, trig: trig numFrames: chunkLen, stats: ~stats,
trig: trig * (count < 10), blocking: 1
); );
var rd = BufRd.kr(2, ~stats, DC.kr(0), 0, 1);// pick only mean pitch and confidence var rd = BufRd.kr(2, ~stats, DC.kr(0), 0, 1);// pick only mean pitch and confidence
var wr1 = BufWr.kr(rd[0], buf, DC.kr(0)); var wr1 = BufWr.kr(rd[0], buf, DC.kr(0));
var wr2 = BufWr.kr(rd[1], buf, DC.kr(1)); var wr2 = BufWr.kr(rd[1], buf, DC.kr(1));
var dsWr = FluidDataSetWr.kr(\stand_help_raw, buf: buf, trig: Done.kr(stats)); var dsWr = FluidDataSetWr.kr(~raw, buf: buf, trig: Done.kr(stats));
LocalOut.kr( Done.kr(dsWr)); LocalOut.kr( Done.kr(dsWr));
FreeSelf.kr(count - 9); FreeSelf.kr(count - 9);
Poll.kr(trig,count, \count);
}.play; }.play;
) )
// Standardize and load to language-side array // Standardize and load to language-side array
( (
~rawarray = Array.new(10); ~rawarray = Array.new(10);
@ -123,41 +124,27 @@ FluidBufPitch.process(s,~audio, features: ~pitch_feature);
:: ::
subsection::Server Side Querying subsection::Server Side Querying
Because FluidStandardize runs in its own link::Classes/Synth:: on the server, communication is done via control-rate link::Classes/Bus:: objects for triggering and link::Classes/Buffer:: objects for passing and retreiving data.
code:: code::
//Setup
(
~tempPoint = Buffer.alloc(s,2);
~predictPoint = Buffer.alloc(s,2);
~avgBuf = Buffer.alloc(s,100,2);
~pitchingBus = Bus.control;
~catchingBus = Bus.control;
)
( (
~standardizer.inBus_(~pitchingBus).outBus_(~catchingBus).inBuffer_(~tempPoint).outBuffer_(~predictPoint); // read frames out of buffer and pass to standardize
//Pitching (no pun intended): read frames out of buffer and pass to standardize
{ {
var audio = BufRd.ar(1,~audio,LFSaw.ar(BufDur.ir(~audio).reciprocal).range(0, BufFrames.ir(~audio))); var audio = BufRd.ar(1,~audio,LFSaw.ar(BufDur.ir(~audio).reciprocal).range(0, BufFrames.ir(~audio)));
var counter = Stepper.ar(Impulse.ar(ControlRate.ir),max:99); var counter = Stepper.ar(Impulse.ar(ControlRate.ir),max:99);
var trig = HPZ1.ar(counter) < 0; var trig = A2K.kr(HPZ1.ar(counter) < 0);
//average 10 frames: one could use the MovingAverage extension here //average 10 frames: one could use the MovingAverage extension here
var avg; var avg;
BufWr.kr(FluidPitch.kr(audio),~avgBuf,phase:counter); var inputPoint= LocalBuf(2);
avg = Mix.new(BufRd.kr(2, ~avgBuf, phase:100.collect{|x|x})) * 0.01; var outputPoint = LocalBuf(2);
var avgBuf = LocalBuf(100,2);
//average of pitch features
BufWr.kr(FluidPitch.kr(audio),avgBuf,phase:counter);
avg = Mix.new(BufRd.kr(2, avgBuf, phase:100.collect{|x|x})) * 0.01;
//assemble data point //assemble data point
BufWr.kr(avg[0],~tempPoint,0); BufWr.kr(avg[0],inputPoint,0);
BufWr.kr(avg[1],~tempPoint,1); BufWr.kr(avg[1],inputPoint,1);
Poll.kr(T2K.kr(trig),BufRd.kr(1,~tempPoint,[0,1]),["pitch (raw)", "confidence (raw)"]); Poll.kr(trig,BufRd.kr(1,inputPoint,[0,1]),["pitch (raw)", "confidence (raw)"]);
Out.kr(~pitchingBus.index,[T2K.kr(trig)]); ~standardizer.kr(trig,inputPoint,outputPoint);
}.play(~standardizer.synth,addAction:\addBefore); Poll.kr(trig,BufRd.kr(1,outputPoint,[0,1]),["pitch (standardized)", "confidence (standardized)"]);
}.play;
//catching
{
Poll.kr(In.kr(~catchingBus.index),BufRd.kr(1,~predictPoint,[0,1]),["pitch (standardized)", "confidence (standardized)"])
}.play(~standardizer.synth,addAction:\addAfter);
) )
:: ::

@ -47,13 +47,13 @@ code::
//Preliminaries: we want some points, a couple of FluidDataSets, a FluidStandardize and a FluidUMAP //Preliminaries: we want some points, a couple of FluidDataSets, a FluidStandardize and a FluidUMAP
( (
~raw = FluidDataSet(s,\umap_help_3D); ~raw = FluidDataSet(s);
~standardized = FluidDataSet(s,\umap_help_3Ds); ~standardized = FluidDataSet(s);
~reduced = FluidDataSet(s,\umap_help_2D); ~reduced = FluidDataSet(s);
~normalized = FluidDataSet(s,\umap_help_2Dn); ~normalized = FluidDataSet(s);
~standardizer = FluidStandardize(s); ~standardizer = FluidStandardize(s);
~normalizer = FluidNormalize(s); ~normalizer = FluidNormalize(s);
~umap = FluidUMAP(s, numDimensions: 2, numNeighbours: 5, minDist: 0.2,iterations: 50, learnRate: 0.2,batchSize: 50); ~umap = FluidUMAP(s).numDimensions_(2).numNeighbours_(5).minDist_(0.2).iterations_(50). learnRate_(0.2).batchSize_(50);
) )
@ -65,11 +65,11 @@ code::
~raw.print; ~raw.print;
//First standardize our DataSet, then apply the UMAP to get 2 dimensions, then normalise these 2 for drawing in the full window size //First standardize our DataSet, then apply the UMAP to get 2 dimensions, then normalise these 2 for drawing in the full window size
(
~standardizer.fitTransform(~raw,~standardized) ~standardizer.fitTransform(~raw,~standardized,action:{"Standardized".postln});
~umap.fitTransform(~standardized,~reduced) ~umap.fitTransform(~standardized,~reduced,action:{"Finished UMAP".postln});
~normalizer.fitTransform(~reduced,~normalized) ~normalizer.fitTransform(~reduced,~normalized,action:{"Normalized Output".postln});
)
//we recover the reduced dataset //we recover the reduced dataset
~normalized.dump{|x| ~normalizedDict = x["data"]}; ~normalized.dump{|x| ~normalizedDict = x["data"]};

Loading…
Cancel
Save