diff --git a/doc/NMFMorph.rst b/doc/NMFMorph.rst index 648bd1c..4b7679d 100644 --- a/doc/NMFMorph.rst +++ b/doc/NMFMorph.rst @@ -1,22 +1,21 @@ :digest: Morph between sounds -:species: transformer +:species: transformer[0] :sc-categories: FluidCorpusManipulation :sc-related: Classes/FluidAudioTransport, Classes/FluidBufNMFCross -:see-also: +:see-also: BufNMF, NMFCross, AudioTransport, BufAudioTransport :description: - Perform cross-synthesis using Nonnegative Matrix Factorization (NMF) and Optimal Transport (OT). NMF analyses of ``source`` and ``target`` sounds decompose their material in to a selectable number of components, which are in turn represented by their *bases* (spectrum) and *activations* (temporal pattern of each component). - - ``FluidNMFMorph`` provides the ability to interpolate between ``source`` and ``target`` spectra using a technique called Optimal Transport, that provides richer results than a simple linear interpolation between spectral shapes. The resulting sound is built up using a buffer of temporal activations, then resynthesised using a phase estimate. - + Perform cross-synthesis using Nonnegative Matrix Factorization (NMF) and Optimal Transport (OT). +:discussion: + The algorithm uses NMF analyses of the ``source`` and ``target`` sounds. It decomposes their material in to a selectable number of components, which are in turn represented by their *bases* (spectrum) and *activations* (temporal pattern of each component). ``NMFMorph`` provides the ability to interpolate between ``source`` and ``target`` bases using a technique called Optimal Transport, that provides richer results than a simple linear interpolation between spectral shapes. The resulting sound is built up using a buffer of temporal activations, then resynthesised using a phase estimate. :control source: - A |buffer| with the spectral bases for the source sound. + A |buffer| with the spectral bases for the source sound (must be the same number of spectral bases as ``target``). :control target: - A |buffer| with the spectral bases for the target sound. + A |buffer| with the spectral bases for the target sound (must be the same number of spectral bases as ``source``). :control activations: @@ -45,4 +44,3 @@ :control maxFFTSize: The maximum FFT size to allocate memory for - diff --git a/example-code/sc/NMFMorph.scd b/example-code/sc/NMFMorph.scd index 27cb273..371f6f5 100644 --- a/example-code/sc/NMFMorph.scd +++ b/example-code/sc/NMFMorph.scd @@ -1,5 +1,5 @@ -code::FluidNMFMorph:: relies on preexisting NMF analyses to generate variations between sounds. We can produce these using link::Classes/FluidBufNMF:: +code::FluidNMFMorph:: relies on preexisting NMF analyses to generate variations between sounds. Produce these using link::Classes/FluidBufNMF:: code:: //read some audio @@ -7,38 +7,29 @@ code:: ~src1 = Buffer.readChannel(s,FluidFilesPath("Nicol-LoopE-M.wav"),channels:[0]); //some drums ~src2 = Buffer.readChannel(s,FluidFilesPath("Tremblay-SA-UprightPianoPedalWide.wav"),channels:[0]);//some piano -~src1Bases = Buffer.new; -~src2Bases = Buffer.new; -~src1Activations = Buffer.new; -~src2Activations = Buffer.new; +~src1Bases = Buffer(s); +~src2Bases = Buffer(s); +~src1Activations = Buffer(s); +~src2Activations = Buffer(s); ) -//nmf analyses + +//nmf analyses -- must have the same number of components (wait for this to complete!) ( -FluidBufNMF.process(s,~src1,bases:~src1Bases,activations:~src1Activations,components:5, action:{"Analysed Source 1".postln}); -FluidBufNMF.process(s,~src2,bases:~src2Bases,activations:~src2Activations, components:5, action:{"Analysed Source 2".postln}); +FluidBufNMF.processBlocking(s,~src1,bases:~src1Bases,activations:~src1Activations,components:5, action:{"Analysed Source 1".postln}); +FluidBufNMF.processBlocking(s,~src2,bases:~src2Bases,activations:~src2Activations, components:5, action:{"Analysed Source 2".postln}); ) ( -~morph = { |source, target, activations, interp, autoassign| - FluidNMFMorph.ar(source,target,activations,autoassign,interp) * 80 -}; +~synth = { |source, target, activations, autoassign| + FluidNMFMorph.ar(source,target,activations,autoassign,MouseX.kr).dup * 80 +}.play(s,args:[\source,~src1Bases,\target,~src2Bases,\activations,~src1Activations,\autoassign,1]); ) -~synth = ~morph.play(s,args:[\source,~src1Bases,\target,~src2Bases,\activations,~src2Activations,\interp,0.5,\autoassign,1]); - -//Play with different interpolation values -~synth.set(\interp,0.0); -~synth.set(\interp,1.0); -:: -warning::The following parameters current require one to change the 'autoassign' control to update the process:: -code:: -//Change the actvations -~synth.set(\activations, ~src1Activations, \autoassign,0); -~synth.set(\autoassign,1); +// Change the actvations +// when changing the activations, one needs to change the 'autoassign' control to update the process ~synth.set(\activations, ~src2Activations, \autoassign,0); -~synth.set(\autoassign,1); -//Swap source and target -~synth.set(\source,~src2Bases,\target,~src1Bases, \autoassign,0); +// change autoassign back to 1 to hear the difference ~synth.set(\autoassign,1); + :: diff --git a/flucoma/doc/templates/schelp_transformer[0].schelp b/flucoma/doc/templates/schelp_transformer[0].schelp new file mode 100644 index 0000000..d49bfb9 --- /dev/null +++ b/flucoma/doc/templates/schelp_transformer[0].schelp @@ -0,0 +1,8 @@ +{% extends "schelp_base.schelp" %} +{% block classmethods %} +CLASSMETHODS:: + +METHOD:: ar + +{% include "schelp_controls.schelp" %} +{% endblock %}