Sample rate & format conversion with LibAV API (libavresample)

Question

I'm writing a code which will merge multiple audios (with different formats) and create a single audio. When i set the encoder sample_rate and sample_fmt same with the input videos i have no problem merging the audios. However as obvious, all of the input audio formats are not same with the output format, so i have to do format conversion. I tried to use "avresample" for this purpose but could not manage to encode the output frames when sample_rate and sample_fmt are different for input&output.

It might be done manually by hand (by sample dropping, interpolation etc.), but since libav provides a conversion api i think this can (and may be should for tidyness) be done automatically.

Here is how i set encoder and resampling context params:

AVCodecContext* avAudioEncoder = outputAudioStream->codec;
AVCodec * audioEncoder = avcodec_find_encoder(AV_CODEC_ID_MP3);

avcodec_get_context_defaults3(avAudioEncoder, audioEncoder);

avAudioEncoder->sample_fmt = AV_SAMPLE_FMT_S16P;
avAudioEncoder->sample_rate = 48000;
avAudioEncoder->channels = 2;
avAudioEncoder->time_base.num = 1;
avAudioEncoder->time_base.den = 48000;
avAudioEncoder->strict_std_compliance = FF_COMPLIANCE_EXPERIMENTAL;

if (outputAVFormat->oformat->flags & AVFMT_GLOBALHEADER)
{
    avAudioEncoder->flags |= CODEC_FLAG_GLOBAL_HEADER;
}

avcodec_open2(avAudioEncoder, audioEncoder, nullptr);

std::shared_ptr<AVAudioResampleContext> avAudioResampleContext(avresample_alloc_context(), [](AVAudioResampleContext * avARC){avresample_close(avARC), avresample_free(&avARC); });

av_opt_set_int(avAudioResampleContext.get(), "in_channel_layout", 2, 0);
av_opt_set_int(avAudioResampleContext.get(), "in_sample_rate", 44100, 0);
av_opt_set_int(avAudioResampleContext.get(), "in_sample_fmt", AV_SAMPLE_FMT_S16P, 0);
av_opt_set_int(avAudioResampleContext.get(), "out_channel_layout", avAudioEncoder->channels, 0);
av_opt_set_int(avAudioResampleContext.get(), "out_sample_rate", avAudioEncoder->sample_rate, 0);
av_opt_set_int(avAudioResampleContext.get(), "out_sample_fmt", avAudioEncoder->sample_fmt, 0);

And here is how i read & encode the frames

...
int result = avcodec_decode_audio4(avAudioDecoder.get(), audioFrame.get(), &isFrameAvailable, &decodingPacket);
...
if (isFrameAvailable)
{
    decodingPacket.size -= result;
    decodingPacket.data += result;

    encodeAudioFrame->format = outputAudioStream->codec->sample_fmt;
    encodeAudioFrame->channel_layout = outputAudioStream->codec->channel_layout;

    auto available = avresample_available(avAudioResampleContext.get());
    auto delay = avresample_get_delay(avAudioResampleContext.get());

    encodeAudioFrame->nb_samples =  available + av_rescale_rnd( delay + audioFrame->nb_samples, avAudioEncoder->sample_rate, audioStream->codec->sample_rate, AV_ROUND_ZERO);
    int linesize;
    av_samples_alloc(encodeAudioFrame->data, &linesize, avAudioEncoder->channels, encodeAudioFrame->nb_samples, avAudioEncoder->sample_fmt, 1);
    encodeAudioFrame->linesize[0] = linesize;

    avresample_convert(avAudioResampleContext.get(), nullptr, encodeAudioFrame->linesize[0], encodeAudioFrame->nb_samples, &audioFrame->data[0], audioFrame->linesize[0], audioFrame->nb_samples*outputAudioStream->codec->channels); 


    std::shared_ptr<AVPacket> outPacket(new AVPacket, [](AVPacket* p){ av_free_packet(p); delete p; });
    av_init_packet(outPacket.get());
    outPacket->data = nullptr;
    outPacket->size = 0;

    while (avresample_available(avAudioResampleContext.get()) >= encodeAudioFrame->nb_samples)
    {
        avresample_read(avAudioResampleContext.get(), &encodeAudioFrame->data[0], encodeAudioFrame->nb_samples*outputAudioStream->codec->channels);


        encodeAudioFrame->pts = av_rescale_q(++encodedAudioPts, outputAudioStream->codec->time_base, outputAudioStream->time_base);

        encodeAudioFrame->pts *= avAudioEncoder->frame_size;
        ... 
        auto ret = avcodec_encode_audio2(avAudioEncoder, outPacketPtr, encodeAudioFramePtr, &got_output);
        ...
    }

It seems i can't use avresample properly, but i could not figure out how to solve this problem. Any help will be appreciated.

Sample rate & format conversion with LibAV API (libavresample)

0 Answers0