How to mix PCM audio sources (Java)?

Question

Here's what I'm working with right now:

for (int i = 0, numSamples = soundBytes.length / 2; i < numSamples; i += 2)
{
    // Get the samples.
    int sample1 = ((soundBytes[i] & 0xFF) << 8) | (soundBytes[i + 1] & 0xFF);   // Automatically converts to unsigned int 0...65535                                 
    int sample2 = ((outputBytes[i] & 0xFF) << 8) | (outputBytes[i + 1] & 0xFF); // Automatically converts to unsigned int 0...65535

    // Normalize for simplicity.
    float normalizedSample1 = sample1 / 65535.0f;
    float normalizedSample2 = sample2 / 65535.0f;

    float normalizedMixedSample = 0.0f;

    // Apply the algorithm.
    if (normalizedSample1 < 0.5f && normalizedSample2 < 0.5f)
        normalizedMixedSample = 2.0f * normalizedSample1 * normalizedSample2;
    else
        normalizedMixedSample = 2.0f * (normalizedSample1 + normalizedSample2) - (2.0f * normalizedSample1 * normalizedSample2) - 1.0f;

    int mixedSample = (int)(normalizedMixedSample * 65535);

    // Replace the sample in soundBytes array with this mixed sample.
    soundBytes[i] = (byte)((mixedSample >> 8) & 0xFF);
    soundBytes[i + 1] = (byte)(mixedSample & 0xFF);
}

From as far as I can tell, it's an accurate representation of the algorithm defined on this page: http://www.vttoth.com/CMS/index.php/technical-notes/68

However, just mixing a sound with silence (all 0's) results in a sound that very obviously doesn't sound right, maybe it's best to describe it as higher-pitched and louder.

Would appreciate help in determining if I'm implementing the algorithm correctly, or if I simply need to go about it a different way (different algorithm/method)?

Answer 1

In the linked article the author assumes A and B to represent entire streams of audio. More specifically X means the maximum abs value of all of the samples in stream X - where X is either A or B. So what his algorithm does is scans the entirety of both streams to compute the max abs sample of each and then scales things so that the output theoretically peaks at 1.0. You'll need to make multiple passes over the data in order to implement this algorithm and if your data is streaming in then it simply will not work.

Here is an example of how I think the algorithm to work. It assumes that the samples have already been converted to floating point to side step the issue of your conversion code being wrong. I'll explain what is wrong with it later:

 double[] samplesA = ConvertToDoubles(samples1);
 double[] samplesB = ConvertToDoubles(samples2);
 double A = ComputeMax(samplesA);
 double B = ComputeMax(samplesB);

 // Z always equals 1 which is an un-useful bit of information.
 double Z = A+B-A*B;

 // really need to find a value x such that xA+xB=1, which I think is:
 double x = 1 / (Math.sqrt(A) * Math.sqrt(B));

 // Now mix and scale the samples
 double[] samples = MixAndScale(samplesA, samplesB, x);

Mixing and scaling:

 double[] MixAndScale(double[] samplesA, double[] samplesB, double scalingFactor)
 {
     double[] result = new double[samplesA.length];
     for (int i = 0; i < samplesA.length; i++)
         result[i] = scalingFactor * (samplesA[i] + samplesB[i]);
 }

Computing the max peak:

double ComputeMaxPeak(double[] samples)
{
    double max = 0;
    for (int i = 0; i < samples.length; i++)
    {
        double x = Math.abs(samples[i]);
        if (x > max)
            max = x;
    }
    return max;
}

And conversion. Notice how I'm using short so that the sign bit is properly maintained:

double[] ConvertToDouble(byte[] bytes)
{
    double[] samples = new double[bytes.length/2];
    for (int i = 0; i < samples.length; i++)
    {
        short tmp = ((short)bytes[i*2])<<8 + ((short)(bytes[i*2+1]);
        samples[i] = tmp / 32767.0;
    }
    return samples;
}