Unity 2019 - How can I save mixer audio output to an AudioClip or load a newly saved .wav to an audioclip at runtime?

Question

I'm making a game with musical instruments and a recording device in-game. I want the user to be able to play the instruments and record them to an audioclip - and to a wav file - which they can then play back as accompaniment to themselves playing another instrument, the output again recorded, allowing them to make tracks with combined instruments. So far I have everything working except I can't figure out how to get the audioclip - I have .wav files being saved to the asset folder but I don't understand the code enough to figure out where to set the audioclip's data to the data being written.

I'm using code from Olkor here to save the recording as wav, which is working great: https://unitylist.com/p/za/Output-Audio-Recorder

My two options to save an audioclip from this seem to be a) save to an audioclip at the same time as saving to disk (I can't figure this out) or save it to disk and then load it into the game as an audioclip - using Unity Web Request which I have tried to do but I get an error either cannot access the .audioclip property of an aborted DownloadHandlerAudioClip or if I invoke the function with a delay to load the file, there is an error decoding the audio. Either way I have a saved .wav audio file in my assets folder but no audioclip.

InvalidOperationException: Cannot access the .audioClip property of an aborted DownloadHandlerAudioClip
UnityEngine.Networking.DownloadHandlerAudioClip.GetContent 
(UnityEngine.Networking.UnityWebRequest www) (at 






C:/buildslave/unity/build/Modules/UnityWebRequestAudio/Public/DownloadHandler 
   Audio.bindings.cs:49)
   OutputAudioRecorder+<GetAudioClip>d__27.MoveNext () (at 
   Assets/Scripts/OutputAudioRecorder.cs:202)
   UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator 
   enumerator, System.IntPtr returnValueAddress) (at C:/buildslave/unity/build/Runtime/Export/Scripting/Coroutines.cs:17)

The script below that I'm using is attached to the audiolistener and so it's picking up all the output of audio from the game. I have looked at doing something with audioListener GetOutputData and passing it to audioClip SetData but I haven't been able to figure it out and I admit that this is beyond my current ability - I'd really appreciate some insight into how to approach this problem in any of the ways possible.

using System;
        using System.IO;
        using UnityEngine;
        using UnityEngine.Networking;
        using System.Collections;


            public class OutputAudioRecorder : MonoBehaviour
            {
                public const string DEFAULT_FILENAME = "record";
                public const string FILE_EXTENSION = ".wav";

                public bool IsRecording { get { return recOutput; } }

                private int bufferSize;
                private int numBuffers;
                private int outputRate;
                private int headerSize = 44; //default for uncompressed wav
                private String fileName;
                private bool recOutput = false;
                private AudioClip newClip;
                private FileStream fileStream;
                private AudioClip[] audioClips;
                private AudioSource[] audioSources;
                public int currentSlot;
                float[] tempDataSource;

                void Awake()
                {
                    outputRate = AudioSettings.outputSampleRate;

                }

                void Start()
                {
                    AudioSettings.GetDSPBufferSize(out bufferSize, out numBuffers);
                    audioSources = new AudioSource[3]; 
                    audioSources[0] = GameObject.FindWithTag("RecSlot1").GetComponent<AudioSource>();
                    audioSources[1] = GameObject.FindWithTag("RecSlot2").GetComponent<AudioSource>();
                    audioSources[2] = GameObject.FindWithTag("RecSlot3").GetComponent<AudioSource>();
                }

                public void StartRecording(string recordFileName)
                {
                    fileName = Path.GetFileNameWithoutExtension(recordFileName) + FILE_EXTENSION;


                    if (!recOutput)
                    {
                        StartWriting(fileName);
                        recOutput = true;
                    }
                    else
                    {
                        Debug.LogError("Recording is in progress already");
                    }
                }

                public void StopRecording()
                {
                    recOutput = false;
                    WriteHeader();
                    UpdateClip();

                }


                private void StartWriting(String name)
                {
                    fileStream = new FileStream(Application.dataPath + "/" + name, FileMode.Create);

                    var emptyByte = new byte();
                    for (int i = 0; i < headerSize; i++) //preparing the header
                    {
                        fileStream.WriteByte(emptyByte);
                    }
                }

                private void OnAudioFilterRead(float[] data, int channels)
                {
                    if (recOutput)
                    {
                        ConvertAndWrite(data); //audio data is interlaced
                    }
                }

                private void ConvertAndWrite(float[] dataSource)
                {

                    var intData = new Int16[dataSource.Length];
                    //converting in 2 steps : float[] to Int16[], //then Int16[] to Byte[]

                    var bytesData = new Byte[dataSource.Length * 2];
                    //bytesData array is twice the size of
                    //dataSource array because a float converted in Int16 is 2 bytes.

                    var rescaleFactor = 32767; //to convert float to Int16

                    for (var i = 0; i < dataSource.Length; i++)
                    {
                        intData[i] = (Int16)(dataSource[i] * rescaleFactor);
                        var byteArr = new Byte[2];
                        byteArr = BitConverter.GetBytes(intData[i]);
                        byteArr.CopyTo(bytesData, i * 2);
                    }

                    fileStream.Write(bytesData, 0, bytesData.Length);

                    tempDataSource = new float[dataSource.Length];
                    tempDataSource = dataSource;


                }

                private void WriteHeader()
                {

                    fileStream.Seek(0, SeekOrigin.Begin);

                    var riff = System.Text.Encoding.UTF8.GetBytes("RIFF");
                    fileStream.Write(riff, 0, 4);

                    var chunkSize = BitConverter.GetBytes(fileStream.Length - 8);
                    fileStream.Write(chunkSize, 0, 4);

                    var wave = System.Text.Encoding.UTF8.GetBytes("WAVE");
                    fileStream.Write(wave, 0, 4);

                    var fmt = System.Text.Encoding.UTF8.GetBytes("fmt ");
                    fileStream.Write(fmt, 0, 4);

                    var subChunk1 = BitConverter.GetBytes(16);
                    fileStream.Write(subChunk1, 0, 4);

                    UInt16 two = 2;
                    UInt16 one = 1;

                    var audioFormat = BitConverter.GetBytes(one);
                    fileStream.Write(audioFormat, 0, 2);

                    var numChannels = BitConverter.GetBytes(two);
                    fileStream.Write(numChannels, 0, 2);

                    var sampleRate = BitConverter.GetBytes(outputRate);
                    fileStream.Write(sampleRate, 0, 4);

                    var byteRate = BitConverter.GetBytes(outputRate * 4);

                    fileStream.Write(byteRate, 0, 4);

                    UInt16 four = 4;
                    var blockAlign = BitConverter.GetBytes(four);
                    fileStream.Write(blockAlign, 0, 2);

                    UInt16 sixteen = 16;
                    var bitsPerSample = BitConverter.GetBytes(sixteen);
                    fileStream.Write(bitsPerSample, 0, 2);

                    var dataString = System.Text.Encoding.UTF8.GetBytes("data");
                    fileStream.Write(dataString, 0, 4);

                    var subChunk2 = BitConverter.GetBytes(fileStream.Length - headerSize);
                    fileStream.Write(subChunk2, 0, 4);

                    fileStream.Close();

                }

                void UpdateClip()
                {
                    StartCoroutine(GetAudioClip());

                }

                IEnumerator GetAudioClip()
                {
                    using (UnityWebRequest www = UnityWebRequestMultimedia.GetAudioClip("file://" + Application.dataPath + "myRecord1.wav", AudioType.WAV))
                    {
                        yield return www.Send();

                        if (www.isNetworkError)
                        {
                            Debug.Log(www.error);
                        }
                        else
                        {
                            AudioClip newClip = DownloadHandlerAudioClip.GetContent(www);

                            Debug.Log(newClip.name + "name    " + newClip.length);
                            keyboardScript.audioClip = newClip;
                        }
                    }
                }

score 0 · Answer 1 · answered Aug 23 '19 at 21:18

0

Try to load as audio clip using this:

        private IENumerator LoadAudio(string url) {

        WWW www;           
        www = new WWW("file:///" + url);  

        yield return www;

        if (www != null && www.isDone)
        {
            AudioClip audioClip;
            audioClip = www.GetAudioClip(true, false, AudioType.WAV);
        }
        }

That works for me at least.

answered Aug 23 '19 at 21:18

Nicklas C.

138
1
8

1

Thanks for answering! I just tried that and Unity says it's obsolete - using WWW and instead suggests I use what I was using that didn't work for me - (UnityWebRequest www = UnityWebRequestMultimedia.GetAudioClip("file://" + Application.dataPath + "myRecord1.wav", AudioType.WAV)) I am running your example anyway but it doesn't do anything. No error but no audioclip being created either. – LampshadeArmchair Aug 23 '19 at 21:54
No problem! Have you checked that the url is valid? And when are you running the script? You cant use it before the wav file is finished writing. – Nicklas C. Aug 23 '19 at 22:01
1

I've tried the URL three ways: with file://, file:/// and just referencing the path it saved to plus the filename. I was calling the LoadAudio function after the stop recording function in the script that records the wav - I tried invoking the function with a really long delay to make sure it's already written but I can see the file in the folder written and playable long before the function that calls LoadAudio runs - and still it's not outputting an audioclip. – LampshadeArmchair Aug 23 '19 at 23:49

LampshadeArmchair · Answer 2 · 2019-08-25T19:28:49.873

Thought this was solution but still not working to load at runtime. (It just refers to the old copy of the clip until I reload the game. Didn't notice this at first.)

While I wasn't able to get the web request to work for me or get WWW to work either, I did get the effect I wanted by creating a blank clip in the assets folder where the .wav saves, then dragging it in the properties window as a reference, setting the audio clip to streaming and then using the code above to write the wav, it automatically overwrites the clip with the data I recorded. The audio has a bit of distortion on it (crackling) which I'll look into next.

This solution is I'm aware probably incredibly obvious (i guess I had to give unity something to refer to but didn't realise it) and doesn't solve the problem of loading files at runtime if they aren't predefined like mine are, but it fixed my problem at least.

Unity 2019 - How can I save mixer audio output to an AudioClip or load a newly saved .wav to an audioclip at runtime?

2 Answers2