BigEndian, LittleEndian Confusion In Xuggler

Question

Previously I posed a question about converting a byte[] to short[] and a new problem I encountered is converting/ not converting the data from byte[] to BigEndian.
Here is what is going on:

I am using TargetDataLine to read data into a byte[10000].

The AudioFormat object has BigEndian set to true, arbitrarily.

This byte[] needs to be converted to short[] so that it can be encoded using Xuggler

I don't know whether the AudioFormat BigEndian should be set to true or false.
I have tried both the cases and I get an Exception in both the cases.
To convert byte[] to short[], I do this:

fromMic.read(tempBufferByte, 0, tempBufferByte.length);
for(int i=0;i<tempBufferShort.length;i++){
tempBufferShort[i] = (short) tempBufferByte[i];
}  

where:
fromMic is TargetDataLine
tempBufferbyte is byte[10000]
tempBufferShort is short[10000]
I get the Exception:

   java.lang.RuntimeException: failed to write packet: com.xuggle.xuggler.IPacket@90098448[complete:true;dts:12;pts:12;size:72;key:true;flags:1;stream index:1;duration:1;position:-1;time base:9/125;]

---

Miscellaneous information that may be needed:

How I set the stream for adding audio in Xuggler:

writer.addAudioStream(0,1,fmt.getChannels(),(int)fmt.getSampleRate());

How I perform the encoding

writer.encodeAudio(1,tempBufferShort,timeStamp,TimeUnit.NANOSECONDS);

Java Doc on AudioFormat

...In addition to the encoding, the audio format includes other properties that further specify the exact arrangement of the data. These include the number of channels, sample rate, sample size, byte order, frame rate, and frame size...

and

For 16-bit samples (or any other sample size larger than a byte), byte order is important; the bytes in each sample are arranged in either the "little-endian" or "big-endian" style.

Questions:

Do I need to keep the BigEndian as true in javax.sound.sampled.AudioFormat object?

What is causing the error? Is it the format?

---

I guess I get BigEndian data preformatted by the AudioFormat object.

Answer 1

If your data is indeed big endian, you can directly convert it to a (big endian) short array like this:

ByteBuffer buf = ByteBuffer.wrap(originalByteArray);
short[] shortArray = buf.asShortBuffer().array();

The resulting short array will have all the original byte array directly, and correctly, mapped, given that your data is big endian. So, an original array, such as:

// bytes
[00], [ae], [00], [7f]

will be converted to:

// shorts
[00ae], [007f]

Answer 2

You need to convert two bytes into one short, so this line is wrong:

tempBufferShort[i] = (short) tempBufferByte[i];

You need something along the lines of

tempBufferShort[i] = (short) 
   (tempBufferByte[i*2] & 0xFF)*256 + (tempBufferByte[i*2+1] & 0xFF);

This would be in line with a big-endian byte array.

Answer 3

What others here have said about the byte-to-short conversion is correct, but it cannot cause the problem you see, it would just cause the output audio to be mostly noise. You can call writeAudio with a buffer of all zeros (or anything, really) so, everything else being equal, the values in the buffer don't matter to whether the call succeeds (they do matter to what you hear in the output, of course :)

1. Does the exception happen at the beginning of the stream (first audio chunk)? Can you write an audio-only stream successfully?

2. Set the audio codec when you call addAudioStream. Try ICodec.ID.CODEC_ID_MP3 or ICodec.ID.CODEC_ID_AAC.

3. Check that fmt.getChannels() and fmt.getSampleRate() are correct. Not all possible values are supported by any particular codec. (2 ch, 44100 Hz should be supported by just about anything).

4. Are you writing your audio and video such that the timestamps are strictly non-decreasing?

5. Do you have enough audio samples for the duration your timestamps are indicating? Does tempBufferShort.length == ((timeStamp - lastTimeStamp) / 1e+9) * sampleRate * channels ? (This may be only approximately equal, but it should be very close, slight rounding errors probably ok).