0

I have a BufferSource, which I create thusly:

const proxyUrl = location.origin == 'file://' ? 'https://cors-anywhere.herokuapp.com/' : '';
const request = new XMLHttpRequest();
request.open('GET', proxyUrl + 'http://heliosophiclabs.com/~mad/projects/mad-music/non.mp3', true);
// request.open('GET', 'non.mp3', true);
request.responseType = 'arraybuffer';
request.onload = () => {
    audioCtx.decodeAudioData(request.response, buffer => {
        buff = buffer;
    }, err => {
        console.error(err);
    });
}
request.send();

Yes, the CORS workaround is pathetic, but this is the way I found to be able to work locally without needing to run a HTTP server. Anyway...

I would like to shift the pitch of this buffer. I've tried various different forms of this:

const source = audioCtx.createBufferSource();
source.buffer = buff;

const analyser = audioCtx.createAnalyser();
analyser.connect(audioCtx.destination);
analyser.minDecibels = -140;
analyser.maxDecibels = 0;

analyser.smoothingTimeConstant = 0.8;
analyser.fftSize = 2048;
const dataArray = new Float32Array(analyser.frequencyBinCount);
source.connect(analyser);
analyser.connect(audioCtx.destination);
source.start(0);
analyser.getFloatFrequencyData(dataArray);
console.log('dataArray', dataArray);

All to no avail. dataArray is always filled with -Infinity values, no matter what I try.

My idea is to get this frequency domain data and then to move all the frequencies up/down by some amount and create a new Oscillator node out of these, like this:

    const wave = audioCtx.createPeriodicWave(real, waveCompnents);
    oscillator.setPeriodicWave(wave);

Anyway. If anyone has a better idea of how to shift pitch, I'd love to hear it. Sadly, detune and playbackRate both seem to do basically the same thing (why are there two ways of doing the same thing?), namely just to speed up or slow down the playback, so that's not it.

Sebastian Simon
  • 18,263
  • 7
  • 55
  • 75
Marc
  • 1,812
  • 4
  • 23
  • 36

1 Answers1

1

First, there's a small issue with the code: you connect the analyser to the destination twice. You don't actually need to connect it at all.

Second, I think the reason you're getting all -infinity values is because you call getFloatFrequencyData right after you start the source. There's a good chance that no samples have been played so the analyser only has buffers of all zeros.

You need to call getFloatFrequencyData after a bit of time to see non-zero values.

Third, I don't think this will work at all, even for shifting the pitch of an oscillator. getFloatFrequencyData only returns the magnitude information. You will need the phase information for the harmonics to get everything shifted correctly. Currently there's no way to get the phase information.

Fourth, if you have an AudioBuffer with the data you need, consider using the playbackRate to change the pitch. Not sure if this will produce the shift you want.

Raymond Toy
  • 5,490
  • 10
  • 13
  • I did finally get the Analyser to work by using a Uint8Array and getByteFrequencyData instead of Float32Array and getFloatFrequencyData. But yeah, you're right about lacking the phase information, so that sucks. As for using playbackRate, no. Like I said: Sadly, detune and playbackRate both seem to do basically the same thing (why are there two ways of doing the same thing?), namely just to speed up or slow down the playback, so that's not it. – Marc Aug 05 '20 at 06:57
  • That doesn't make sense to me. getByteFrequencyData is basically getFloatFrequencyData with the float values mapped to the range 0-255. If one works, the other should too. But I'm glad you got that to work. Too bad it doesn't provide you with what you need. People have requested a pitch shifter in WebAudio. – Raymond Toy Aug 05 '20 at 14:43
  • As for the pitch shifting, which is the point of this exercise, I found this: https://github.com/ChenMachluf/pitch-shift-node . Now... I just need to figure out how to use a npm package in a strictly client-side (not NodeJS) app. – Marc Aug 06 '20 at 16:44