0

I'm trying to make an effect where I freeze the sound using a phase vocoder. I do this by storing the spectral frame (magnitude and phase) and also the phase difference between the previous frame and the current one. To play back the frozen frame, I just plug the spectral frame into the inverse function of the phase vocoder repeatedly, each time incrementing (and wrapping) the phase with my phase difference values.

Here is some pseudocode (for brevity) of what I'm doing at the moment, where frameA and frameB are magnitude/phase representations of the phase vocoder's fft representation.

void analyze(inputSignal) {
    // convert time domain "inputSignal" to frequency domain
    frameA = vocoder.forward(inputSignal); 

    // calculate the inter-frame phase delta
    phaseDeltaA = frameA.phase - lastPhases;
    lastPhases = frameA.phase; 
}

void playback(outputSignal) {
    frameA.phase += phaseDeltaA;
    outputSignal = vocoder.reverse(frameA);
}

It works nicely. But what I want to do is combine this frozen spectral frame with other "frozen" frames (accumulating them).

I've tried adding the frames together, also tried adding the phase differences together, but it just makes nasty noises.

void analyze(inputSignal) {

    ...

    // naively sum the magnitudes and phases of both frames
    combinedFrame.magnitude = frameA.magnitude + frameB.magnitude;
    combinedFrame.phase = frameA.phase + frameB.phase;

    // sum the phase deltas
    combinedPhaseDelta = phaseDeltaA + phaseDeltaB;

}
void playback(outputSignal) {
    combinedFrame.phase += combinedPhaseDelta;
    outputSignal = vocoder.reverse(combinedFrame);
}
mazbox
  • 613
  • 1
  • 5
  • 7

1 Answers1

0

Adding delta phases together will change the frequencies, thus ruining any harmonic relationships required to make the synthesis sound "good".

Another possibile solution would be to combine not frames, but complete synthetic sound tracks. e.g. make sure each phase vocoder synthesized sound track sounds good by itself, and then use a mixer to composite the result.

hotpaw2
  • 70,107
  • 14
  • 90
  • 153
  • I agree that summing the sounds after moving them back into the time domain would work (I actually have this working, in order to hear what it should sound like), but I want to do it in the frequency domain so I don't have to do more IFFT's than I need to. I thought because the FFT is linear you should be able to add the two signals somehow. Just can't get my head around it. – mazbox Aug 08 '17 at 22:49