Recording In-Game Audio in Unity

Recently I began doing a second pass on my synthesizers in the Google Play store. I think the core of each of those synths is pretty solid, but they are still missing some key features. For example, if you want to record a performance, you must record the output of the headphone jack.

So I just finished writing a class that renders a Unity audio stream to a wave file, and I wanted to share it here.

The class is called AudioRenderer. It’s a MonoBehaviour that uses the OnAudioFilterRead method to write chunks of data to a stream. When the performance ends, the Save method is used to save to a canonical wav file.

The full AudioRenderer class is pasted here.

using UnityEngine;
using System;
using System.IO;

public class AudioRenderer : MonoBehaviour
{
    #region Fields, Properties, and Inner Classes
    // constants for the wave file header
    private const int HEADER_SIZE = 44;
    private const short BITS_PER_SAMPLE = 16;
    private const int SAMPLE_RATE = 44100;

    // the number of audio channels in the output file
    private int channels = 2;

    // the audio stream instance
    private MemoryStream outputStream;
    private BinaryWriter outputWriter;

    // should this object be rendering to the output stream?
    public bool Rendering = false;

    /// The status of a render
    public enum Status
    {
        UNKNOWN,
        SUCCESS,
        FAIL,
        ASYNC
    }

    /// The result of a render.
    public class Result
    {
        public Status State;
        public string Message;

        public Result(Status newState = Status.UNKNOWN, string newMessage = "")
        {
            this.State = newState;
            this.Message = newMessage;
        }
    }
    #endregion

    public AudioRenderer()
    {
        this.Clear();
    }

    // reset the renderer
    public void Clear()
    {
        this.outputStream = new MemoryStream();
        this.outputWriter = new BinaryWriter(outputStream);
    }

    /// Write a chunk of data to the output stream.
    public void Write(float[] audioData)
    {
        // Convert numeric audio data to bytes
        for (int i = 0; i < audioData.Length; i++)
        {
            // write the short to the stream
            this.outputWriter.Write((short)(audioData[i] * (float)Int16.MaxValue));
        }
    }

    // write the incoming audio to the output string
    void OnAudioFilterRead(float[] data, int channels)
    {
        if( this.Rendering )
        {
            // store the number of channels we are rendering
            this.channels = channels;

            // store the data stream
            this.Write(data);
        }
            
    }

    #region File I/O
    public AudioRenderer.Result Save(string filename)
    {
        Result result = new AudioRenderer.Result();

        if (outputStream.Length > 0)
        {
            // add a header to the file so we can send it to the SoundPlayer
            this.AddHeader();

            // if a filename was passed in
            if (filename.Length > 0)
            {
                // Save to a file. Print a warning if overwriting a file.
                if (File.Exists(filename))
                    Debug.LogWarning("Overwriting " + filename + "...");

                // reset the stream pointer to the beginning of the stream
                outputStream.Position = 0;

                // write the stream to a file
                FileStream fs = File.OpenWrite(filename);

                this.outputStream.WriteTo(fs);

                fs.Close();

                // for debugging only
                Debug.Log("Finished saving to " + filename + ".");
            }

            result.State = Status.SUCCESS;
        }
        else
        {
            Debug.LogWarning("There is no audio data to save!");

            result.State = Status.FAIL;
            result.Message = "There is no audio data to save!";
        }

        return result;
    }

    /// This generates a simple header for a canonical wave file, 
    /// which is the simplest practical audio file format. It
    /// writes the header and the audio file to a new stream, then
    /// moves the reference to that stream.
    /// 
    /// See this page for details on canonical wave files: 
    /// http://www.lightlink.com/tjweber/StripWav/Canon.html
    private void AddHeader()
    {
        // reset the output stream
        outputStream.Position = 0;

        // calculate the number of samples in the data chunk
        long numberOfSamples = outputStream.Length / (BITS_PER_SAMPLE / 8);

        // create a new MemoryStream that will have both the audio data AND the header
        MemoryStream newOutputStream = new MemoryStream();
        BinaryWriter writer = new BinaryWriter(newOutputStream);

        writer.Write(0x46464952); // "RIFF" in ASCII

        // write the number of bytes in the entire file
        writer.Write((int)(HEADER_SIZE + (numberOfSamples * BITS_PER_SAMPLE * channels / 8)) - 8);

        writer.Write(0x45564157); // "WAVE" in ASCII
        writer.Write(0x20746d66); // "fmt " in ASCII
        writer.Write(16);

        // write the format tag. 1 = PCM
        writer.Write((short)1);

        // write the number of channels.
        writer.Write((short)channels);

        // write the sample rate. 44100 in this case. The number of audio samples per second
        writer.Write(SAMPLE_RATE);

        writer.Write(SAMPLE_RATE * channels * (BITS_PER_SAMPLE / 8));
        writer.Write((short)(channels * (BITS_PER_SAMPLE / 8)));

        // 16 bits per sample
        writer.Write(BITS_PER_SAMPLE);

        // "data" in ASCII. Start the data chunk.
        writer.Write(0x61746164);

        // write the number of bytes in the data portion
        writer.Write((int)(numberOfSamples * BITS_PER_SAMPLE * channels / 8));

        // copy over the actual audio data
        this.outputStream.WriteTo(newOutputStream);

        // move the reference to the new stream
        this.outputStream = newOutputStream;
    }
    #endregion
}

As written it will only work on 16bit/44kHz audio streams, but it should be easily adaptable.

Granular Synthesis for Android Phones

Granular is a granular synthesizer for Android devices. Play it by dragging your fingers around the waveform for the source audio file. You can upload your own audio files, or just play with the sounds that are distributed with the app.



The horizontal position on the waveform controls the location from which grains will be pulled. The vertical position controls the grain size. The leftmost slider controls the amount of frequency modulation applied to the grains. The middle slider controls the time interval between grains. The rightmost slider controls randomness.

Download Granular from the Google Play Store and start making grainy soundscapes on your phone.

The Simplest Unity Bundling Example I Could Make

Asset bundles in Unity can be tricky. I’ve spent a lot of time at work developing a pretty slick bundling system. The system pulls the latest from our git repo, then uses manifests to pull in new procedural assets, then generates all the bundles for each level, then uploads them into versioned folders on Amazon S3.

Most big projects will need something like this.

But it’s probably best to start with something simpler. The system I’ve built for work is probably 500 – 1000 lines of code with everything included. This weekend I wanted to find the least amount of code with which I could make a working bundles system.

This system has no extra features. It doesn’t support simulation mode. It doesn’t support procedural assets. It doesn’t support multiple simultaneous downloads. It doesn’t support viewing download progress. But it does work.

Here’s how to get Unity asset bundles working in under 200 lines of code.

Download the AssetBundleManager from the Asset Store

You don’t need the examples. You’re really only using this for the UI features added to the Editor, but some of the included classes in that package are also a good starting point.

Assign assets to bundles

When you select an asset, you should see a drop down all the way at the very bottom of the inspector. Use that dropdown to create a new asset bundle, then add your assets to it.

Create editor script for clearing bundle cache

When you are testing bundles, you will need an extra menu item to delete your local cache. Make an Editor folder somewhere and drop this script in it.

using UnityEngine;
using UnityEditor;

public class BundleOptions
{
    /// 
    /// Delete all cached bundles. This is necessary for testing.
    /// 
    [MenuItem("Assets/AssetBundles/Clear Bundle Cache")]
    static void LogSelectedTransformName()
    {
        if( Caching.CleanCache() )
        {
            Debug.Log("All asset bundles deleted from cache!");
        }
        else
        {
            Debug.LogWarning("Unable to delete cached bundles! Are any bundles in use?");
        }
    }
}

You will want to click Assets > AssetBundles > Clear Bundle Cache between each generation of bundles to make sure that you are using the latest version in editor.

Create a BundleManager class

Your BundleManager class will download bundles and allow the program to access asset bundles. If you expand this example, then this will be the center of your work.

using UnityEngine;
using System.Collections;
using System.Collections.Generic;

public class BundleManager : MonoBehaviour
{
    // the web server where your bundles are stored - you can test this locally using file://c:/path-to-bundles/etc
#if UNITY_EDITOR
    private static string baseBundleUrl = "file://c:/example/Unity/MyGame/AssetBundles";
#else
    private static string baseBundleUrl = "http://www.example.com/MyGame/AssetBundles";
#endif

    // this dictionary will store references to each downloaded asset bundle
    private static Dictionary bundles;

    /// 
    /// Get a reference to an AssetBundle instance that has already been downloaded.
    /// 
    /// This should be used when you want to load an asset from a bundle.
    /// 
    /// the name of the bundle to get
    /// a reference to an AssetBundle or null if that bundle has not yet been downloaded or cached
    public static AssetBundle GetBundle(string name)
    {
        if( bundles != null && bundles.ContainsKey(name) )
        {
            return bundles[name];
        }
        return null;
    }

    /// 
    /// Very simple method for downloading a single bundle.
    /// Extensions to this method may include a progress monitor and a callback for when the download is complete.
    /// 
    /// the name of the bundle you want to download
    /// an IEnumerator to be run as a Coroutine
    public static IEnumerator DownloadBundle(string bundleName, System.Action downloadCallback = null)
    {
        Debug.Log("Attempting to load bundle " + bundleName + "...");

        WWW www = WWW.LoadFromCacheOrDownload(getFullBundleUrl(bundleName), 1);

        yield return www;

        // if there was a download error, then log it
        if( !string.IsNullOrEmpty(www.error) )
        {
            Debug.LogError("Error while downloading bundle " + bundleName + ": " + www.error);
        }
        else // if there was no download error
        {
            // try to get the downloaded bundle
            AssetBundle newBundle = www.assetBundle;

            // if the bundle is null, then log an error
            if( newBundle == null )
            {
                Debug.LogError("Unable to save bundle " + bundleName + ": Bundle is null!");
            }
            else // if a valid bundle was downloaded
            {
                if (bundles == null)
                    bundles = new Dictionary();

                // store a reference to that bundle in the dictionary
                bundles[bundleName] = newBundle;

                Debug.Log("Successfully loaded " + bundleName + ".");
            }
        }

        // if there is a downloadCallback, then call it
        if( downloadCallback != null )
        {
            downloadCallback();
        }
    }

    /// 
    /// Return a string representation of the platform that will go into the bundle path.
    /// 
    private static string getBundlePlatform()
    {
        if (Application.platform == RuntimePlatform.Android)
        {
            return "Android";
        }
        else if( Application.platform == RuntimePlatform.WindowsEditor )
        {
            return "Windows";
        }

        // maybe this is some strange version of Android? Need this for Kindle.
        return "Android";

        // do not support other platforms
        //throw new System.Exception("This platform is not supported!");
    }

    private static string getFullBundleUrl(string bundleName)
    {
        return baseBundleUrl + "/" + getBundlePlatform() + "/" + bundleName;
    }

    // Use this to initialize the bundles dictionary
    void Start ()
    {
        if (bundles == null)
            bundles = new Dictionary();
    }

}

Modify that script by replacing baseBundleUrl with the place where your bundles are stored locally for development and remotely for production.

You will also need to hook this script into the program somehow. You can attach it to a GameObject if you like, but that isn’t strictly necessary. All you really need to do is start a coroutine to run the DownloadBundle method, then call GetBundle to get the download. Then just call the LoadAsset method on the asset you want to instantiate.

Generate bundles

When you click Assets > AssetBundles > Build AssetBundles, it will generate bundles for whatever platform is selected. You will need to generate bundles for your development machine and for your target platform. Make sure to modify the getBundlePlatform method to support your platform.

And…. ?

And that’s it. Asset bundles are not really that complex when you boil them down. For my music apps, I just want them to store a few extra sound files that I don’t want to distribute with the executable. So a system like this works fine. I just start the download when the app starts.

Of course, for larger projects you will need many more features. Versioning bundles is very important for the development and build processes. Also you will want to show a progress bar on the screen. And you will want to load entire levels into bundles, which is a little more tricky. But hopefully this will get you started on the right track.

Make Explosive Soundscapes with Circular Sound

I’ve just finished work on a new musical instrument for Android devices. It’s called Circular Sound, and it’s aimed at people who like noise.



Circular Sound is similar to my other recent mobile instruments in that it combines a sampler with custom digital signal processing, and a unique interface. Sounds can be loaded from a configurable directory on the device, or you can play around with the default sounds, which are from freesound.org. Then they are loaded into spheres that are arranged in a circle on the left half of the screen. The left half of the screen is the source audio mixer, while the right half is used to control effects. The effects include waveshaping, granulation, delay, and modulation.

The goal of Circular Sound is to give a simple access point into generating various types of noise that is related in some way to the source sounds provided by the user.

Download it for free on Google Play and shoot me a comment to let me know if you make something cool with it!