4

I have a job where I have to take continuous screenshots and capture sound from a desktop, then publish them as a live video stream. I use Wowza Media Server 3.0.3 for stream publishing. I also use Xuggler to generate image frames and put them with sound buffers into packets. I have the following problem:

I start my program, and the publishing of image frames and sound packets are in progress. Wowza console informs me, that packets are published. When I open a media player (in this case VLC), the video part of the stream works like a charm (I see the imageframes captured from my desktop continuously), but the audio part is very poor. I mean, when I start to play the livestream, the VLC buffers an approximately 3 second long sound part recorded from my desktop, and plays it back at a higher speed. After a longer break it buffers again and play the next part. In my code I continuously send sound iBuffers encoded in MP3 and publish them into packets, so I cannot understand why the sound is not playing continuously as the image frames.

Can anyone got the answer or any experience in my problem?

I've made a copy from my code, where I just stream desktop sound, not with the image frames. This is the snippet, where I get the sound, and send it to encode and publish:

while (true)
    {
        byte buffer[] = new byte[line.available()];
        int count = line.read(buffer, 0, buffer.length);
        IBuffer iBuf = IBuffer.make(null, buffer, 0, count);

        //Itt írjuk a stream-be az audioframe-et
        _AudioWriter.encodeFrameToStream(iBuf, buffer, firstTimeStamp);
        try {
            Thread.sleep(100);
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

This is the part, where I get the iBuffer and encode it to mp3. After I publish it as a packet:

public void encodeFrameToStream(IBuffer ibuffer, byte[] buffer, long firstTimeStamp) {
    long now = System.currentTimeMillis();
    long timeStamp = (now - firstTimeStamp);

    IAudioSamples outChunk = IAudioSamples.make(ibuffer, 1, IAudioSamples.Format.FMT_S16);
    if (outChunk == null)
    {
        return;
    }
    long numSample = buffer.length / outChunk.getSampleSize();
    outChunk.setComplete(true, numSample, 44100, 1, Format.FMT_S16, timeStamp);

    //System.out.println(outChunk + " =========== " + outChunk.getPts());
    IPacket packet2 = IPacket.make();
    packet2.setStreamIndex(0);
    getCoder2().encodeAudio(packet2, outChunk, 0);
    outChunk.delete();

    if (packet2.isComplete()) {
        //System.out.println("completed");
        getContainer().writePacket(packet2);
        //System.out.println("Size: "+packet2.getSize());
    }
}
Mike G
  • 4,232
  • 9
  • 40
  • 66
Bandi
  • 41
  • 1

1 Answers1

0

We'll have to debug a bit more to get to know all factors involved.

  • Usually when a audio stream is played back with a different pitch, it means the sample rates of the input and output do not match. You are currently manually setting the sample format to FMT_S16 and the sample rate to 44.100 Hz. This will work fine, as long as the input is already formatted this way.

    You might want to make sure the packets have the correct number of channels, sample format and sample rate by using an IAudioResampler in between input and output. Use IMediaWriter and IStreamCoder functions getChannels() and getSampleRate() as input for the IAudioResampler.

  • I'm not familiar with Wowza Media Server, but it seems it performs some sort of transcoding itself. I couldn't tell from your code, but it seems you are directly streaming to Wowza instead of using using file container. You could try outputting to a file and see if you can play it afterwards. That way you can check if the audio/video-data is properly encoded.

    If it is, the problem probably lies with Wowza. Check and see if it poses any particular restrictions on codec, sample format, sample size, channels, sample rate and bitrate.

    If the output file doesn't play, try just writing the audio stream and leaving out any video. If that does play, then the problem lies with the forming of packets out of audio and video data.

  • Lastly, could you try and output the timestamp of each video frame and audio sample when you write them? This way you can make sure that all data packets are neatly aligned by chronology. If at some point in a video file these packets are in the wrong order, the file cannot be properly streamed and played back.

For example, this is a malformed video file:

0ms video frame  1
0ms audio sample 1
10ms video sample 2
8ms audio frame 2`
Luke
  • 409
  • 3
  • 12