Convert .ogg to correct .wav in C#

Question

I need to convert .ogg file to correct .wav 16khz 1 channel or just .mp3. But only code that requires 3 different streams works, otherwise I get an "Invalid wave file RIFF header." error. Can it be done better?

• Important note, I need to do this in memory without saving intermediate files

// converting to wave file
// 1
using (MemoryStream pcmStream = new()) 
{
    OpusDecoder decoder = new OpusDecoder(16000, 1);
    OpusOggReadStream oggIn = new(decoder, DestinationStream);
    while (oggIn.HasNextPacket) 
    {
        short[] packet = oggIn.DecodeNextPacket();
        if (packet != null) 
        {
            foreach (short v in packet) 
            {
                var bytes = BitConverter.GetBytes(v);
                pcmStream.Write(bytes, 0, bytes.Length);
            }
        }
    }

    pcmStream.Position = 0;
    // 2
    var wavStream = new RawSourceWaveStream(pcmStream, new WaveFormat(16000, 1));
    // 3
    using var memoryStream = new MemoryStream();
    // doesn't work :(
    // https://stackoverflow.com/a/24435749/21547031
    //WriteWavHeader(memoryStream, false, 1, 16, 16000, memoryStream.Capacity);
    WaveFileWriter.WriteWavFileToStream(memoryStream, wavStream);
    memoryStream.Position = 0;
}