1

What are the standard ways of reading and writing audio files on Android / Kotlin?

I am very confused. I've found plenty of posts that discuss this at some level, but they're all either giving a third party answer (someone's own implementation like https://medium.com/@rizveeredwan/working-with-wav-files-in-android-52e9500297e or https://stackoverflow.com/a/43569709/4959635 or https://gist.github.com/kmark/d8b1b01fb0d2febf5770) or using some Java class, of which I don't know how it's related to the Android SDK (https://stackoverflow.com/a/26598862/4959635, https://gist.github.com/niusounds/3e49013a8e942cdba3fbfe1c336b61fc, https://github.com/google/oboe/issues/548#issuecomment-502758633).

I cannot find a standard way from the Android documentation. Some answer said to use https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/read-bytes.html for reading, but I'm quite sure this doesn't parse the file header.

So what's the standard way of processing audio files on Android / Kotlin?

I'm already using dr_wav just fine on desktop, so I am actually thinking of just using that through NDK and maybe creating a wrapper to it.

mavavilj
  • 129
  • 13
  • There are a few system components that deal with various use cases: encoding, decoding, recording, playback. Some are more low level than others. In order to get a recommendation, consider stating what exactly are you planning to do with your audio files. – dev.bmax Nov 08 '22 at 12:21
  • @dev.bmax I'm just asking for the standard functions. It would be surprising if a media platform does not have standard audio I/O functions. – mavavilj Nov 08 '22 at 13:46

2 Answers2

2

Your use case is not clear from the question.

Assuming that you need to process raw audio data (PCM samples) - the standard way is to read the (compressed) input file using the MediaExtractor and decode the packets using the MediaCodec. Note that the documentation includes some example code.

The MediaCodec outputs ByteBuffers containing raw PCM samples. The binary format is described here.

dev.bmax
  • 8,998
  • 3
  • 30
  • 41
  • Yes this is good advice. Found some example code: https://gist.github.com/a-m-s/1991ab18fbcb0fcc2cf9 – mavavilj Nov 08 '22 at 13:37
  • Additionally, there are tools in the Android NDK for audio processing: https://developer.android.com/ndk/guides/audio. Consider them if the performance of your Java / Kotlin code is not sufficient. – dev.bmax Nov 08 '22 at 14:23
  • However, this answer suggests that MediaExtractor could not extract audio from .wav https://stackoverflow.com/a/38037383/4959635 – mavavilj Nov 09 '22 at 06:17
  • The MediaExtractor doc (https://developer.android.com/reference/android/media/MediaExtractor) does not specify whether the file headers are processed, but I assume that's the case. – mavavilj Nov 09 '22 at 07:48
  • The samples are also for Java, which makes it slightly non-trivial to map to current Kotlin/Android. I'm also having and seeing some troubles with the MediaExtractor class, so I wonder if it's a mature solution for Kotlin apps. – mavavilj Nov 09 '22 at 11:24
  • I can confirm that `MediaExtractor` and `MediaCodec` CAN handle most uncompresses `WAV` audio files. See the full list of supported formats: https://developer.android.com/guide/topics/media/media-formats – dev.bmax Nov 09 '22 at 11:54
  • Looking at the answer that you mention above it seems that the problem is caused by incorrect configuration of the components which is a very common issue. – dev.bmax Nov 09 '22 at 11:56
  • Here's a quick rule of thumb: if the system media player on the device can play your file it's most likely that MediaExtractor+MediaCodec are also able to handle it. Internally, the same low level platform tools are used in both cases. – dev.bmax Nov 09 '22 at 12:04
  • I'm however struggling to find working example code in Kotlin. – mavavilj Nov 10 '22 at 07:30
  • Did you know that when you paste Java code into Android Studio it suggests to automatically convert it to Kotlin? – dev.bmax Nov 10 '22 at 11:20
  • Yes, but I'm quite certain it will not work without breaking something except for some very simple Java code. – mavavilj Nov 10 '22 at 11:49
  • Usually, there is no problem converting Java code to Kotlin. Kotlin is compiled to Java bytecode and runs inside the same virtual machine anyway. Kotlin code can call Java code and vice versa. – dev.bmax Nov 10 '22 at 13:38
  • Also, I was confused about whether the MediaCodec must also be used for PCM data. It seems as if it's meant to be used as well, but I was confused why they exist as separate classes, rather than as an integrated class. – mavavilj Nov 13 '22 at 15:20
  • You can think about `MediaExtractor` and `MediaCodec` as abstract classes. Different implementations can handle different types of containers, streams (audio, video) and formats. Most of the time media files are compressed. Uncompressed media occupy a lot of storage space. Also, working with one particular format in the app is not a good strategy. – dev.bmax Nov 13 '22 at 16:47
  • I also cannot get MediaExtractor by the example code to produce the correct values, whereas https://medium.com/@rizveeredwan/working-with-wav-files-in-android-52e9500297e does this without MediaExtractor. – mavavilj Nov 14 '22 at 16:04
  • `MediaExtractor` is the short path. One can choose the long path and write a lot of code to open the file, detect and handle the format (sample rate, channel count, sample size, channel mask, byte order), do the buffering, etc. It's up to you. I answered the question about the standard way of doing it. If something doesn't work out, one can ask a specific question on SO showing the broken code... – dev.bmax Nov 15 '22 at 09:52
  • Also, you suggest the PCM samples also require the use of MediaCodec. However, my example .wav already reports `{pcm-encoding=2, mime=audio/raw, channel-count=1, channel-mask=0, track-id=1, max-input-size=32768, durationUs=30000000, sample-rate=44100}` using getTrackFormat(). Doesn't this suggest that readSampleData() will already return PCM samples? – mavavilj Nov 24 '22 at 09:06
  • 1
    If you are not required to support compressed audio, and you only ever deal with raw audio files, then you should be able to consume buffers filled with samples using `readSampleData()`. Passing them though `MediaCodec` would not be necessary. But note that the size of `ByteBuffers` that you allocate must be >= `max-input-size`. – dev.bmax Nov 24 '22 at 09:51
1

Well, there is no strict standard.

In production, you usually choose stable third party library or your company's reusable internal solution for this kind of tasks. You still can implement it yourself, but it will cost you time, since most likely the implementation will consist of hundreds of lines of code and you probably will just create another variation of existing solution which is present on the internet.

Steyrix
  • 2,796
  • 1
  • 9
  • 23
  • I think there should be a standard way that works. Since audio I/O is a pretty standard thing for apps. – mavavilj Nov 08 '22 at 12:12
  • If you mean some solution that is present in Android SDK and parses audio file, I don't know if such thing exists. Could you tell what type of application you are developing? – Steyrix Nov 08 '22 at 12:15
  • An audio analysis app. – mavavilj Nov 08 '22 at 12:19
  • I think the simplest and pretty standard way will be to parse raw bytes via stream. It parses the file header too, since the header is the part of the file. – Steyrix Nov 08 '22 at 12:23
  • I tried reading using File(filePath).readBytes(), but I think it messes something up, because my analysis code doesn't work, even when it does work with the same file when reading into the same analysis code using dr_wav in C. So what streams you mean? – mavavilj Nov 08 '22 at 12:24
  • Why can't you use Java classes to perform this? Since you are using Kotlin, it should be fully compatible. https://docs.oracle.com/javase/8/docs/technotes/guides/sound/programmer_guide/chapter7.html#a114527 – Steyrix Nov 08 '22 at 12:27
  • I don't think there's such compatible library in Java language features: https://developer.android.com/studio/write/java8-support.html#kts – mavavilj Nov 08 '22 at 12:29