Recording In-Game Audio in Unity

Recently I began doing a second pass on my synthesizers in the Google Play store. I think the core of each of those synths is pretty solid, but they are still missing some key features. For example, if you want to record a performance, you must record the output of the headphone jack.

So I just finished writing a class that renders a Unity audio stream to a wave file, and I wanted to share it here.

The class is called AudioRenderer. It’s a MonoBehaviour that uses the OnAudioFilterRead method to write chunks of data to a stream. When the performance ends, the Save method is used to save to a canonical wav file.

The full AudioRenderer class is pasted here.

using UnityEngine;
using System;
using System.IO;

public class AudioRenderer : MonoBehaviour
{
    #region Fields, Properties, and Inner Classes
    // constants for the wave file header
    private const int HEADER_SIZE = 44;
    private const short BITS_PER_SAMPLE = 16;
    private const int SAMPLE_RATE = 44100;

    // the number of audio channels in the output file
    private int channels = 2;

    // the audio stream instance
    private MemoryStream outputStream;
    private BinaryWriter outputWriter;

    // should this object be rendering to the output stream?
    public bool Rendering = false;

    /// The status of a render
    public enum Status
    {
        UNKNOWN,
        SUCCESS,
        FAIL,
        ASYNC
    }

    /// The result of a render.
    public class Result
    {
        public Status State;
        public string Message;

        public Result(Status newState = Status.UNKNOWN, string newMessage = "")
        {
            this.State = newState;
            this.Message = newMessage;
        }
    }
    #endregion

    public AudioRenderer()
    {
        this.Clear();
    }

    // reset the renderer
    public void Clear()
    {
        this.outputStream = new MemoryStream();
        this.outputWriter = new BinaryWriter(outputStream);
    }

    /// Write a chunk of data to the output stream.
    public void Write(float[] audioData)
    {
        // Convert numeric audio data to bytes
        for (int i = 0; i < audioData.Length; i++)
        {
            // write the short to the stream
            this.outputWriter.Write((short)(audioData[i] * (float)Int16.MaxValue));
        }
    }

    // write the incoming audio to the output string
    void OnAudioFilterRead(float[] data, int channels)
    {
        if( this.Rendering )
        {
            // store the number of channels we are rendering
            this.channels = channels;

            // store the data stream
            this.Write(data);
        }
            
    }

    #region File I/O
    public AudioRenderer.Result Save(string filename)
    {
        Result result = new AudioRenderer.Result();

        if (outputStream.Length > 0)
        {
            // add a header to the file so we can send it to the SoundPlayer
            this.AddHeader();

            // if a filename was passed in
            if (filename.Length > 0)
            {
                // Save to a file. Print a warning if overwriting a file.
                if (File.Exists(filename))
                    Debug.LogWarning("Overwriting " + filename + "...");

                // reset the stream pointer to the beginning of the stream
                outputStream.Position = 0;

                // write the stream to a file
                FileStream fs = File.OpenWrite(filename);

                this.outputStream.WriteTo(fs);

                fs.Close();

                // for debugging only
                Debug.Log("Finished saving to " + filename + ".");
            }

            result.State = Status.SUCCESS;
        }
        else
        {
            Debug.LogWarning("There is no audio data to save!");

            result.State = Status.FAIL;
            result.Message = "There is no audio data to save!";
        }

        return result;
    }

    /// This generates a simple header for a canonical wave file, 
    /// which is the simplest practical audio file format. It
    /// writes the header and the audio file to a new stream, then
    /// moves the reference to that stream.
    /// 
    /// See this page for details on canonical wave files: 
    /// http://www.lightlink.com/tjweber/StripWav/Canon.html
    private void AddHeader()
    {
        // reset the output stream
        outputStream.Position = 0;

        // calculate the number of samples in the data chunk
        long numberOfSamples = outputStream.Length / (BITS_PER_SAMPLE / 8);

        // create a new MemoryStream that will have both the audio data AND the header
        MemoryStream newOutputStream = new MemoryStream();
        BinaryWriter writer = new BinaryWriter(newOutputStream);

        writer.Write(0x46464952); // "RIFF" in ASCII

        // write the number of bytes in the entire file
        writer.Write((int)(HEADER_SIZE + (numberOfSamples * BITS_PER_SAMPLE * channels / 8)) - 8);

        writer.Write(0x45564157); // "WAVE" in ASCII
        writer.Write(0x20746d66); // "fmt " in ASCII
        writer.Write(16);

        // write the format tag. 1 = PCM
        writer.Write((short)1);

        // write the number of channels.
        writer.Write((short)channels);

        // write the sample rate. 44100 in this case. The number of audio samples per second
        writer.Write(SAMPLE_RATE);

        writer.Write(SAMPLE_RATE * channels * (BITS_PER_SAMPLE / 8));
        writer.Write((short)(channels * (BITS_PER_SAMPLE / 8)));

        // 16 bits per sample
        writer.Write(BITS_PER_SAMPLE);

        // "data" in ASCII. Start the data chunk.
        writer.Write(0x61746164);

        // write the number of bytes in the data portion
        writer.Write((int)(numberOfSamples * BITS_PER_SAMPLE * channels / 8));

        // copy over the actual audio data
        this.outputStream.WriteTo(newOutputStream);

        // move the reference to the new stream
        this.outputStream = newOutputStream;
    }
    #endregion
}

As written it will only work on 16bit/44kHz audio streams, but it should be easily adaptable.

Erratum, an Album Made Entirely with Custom Noise Apps

Erratum is an album that has been in gestation for over a year, and even as I release it into the wild I am refining my ideas about it, and apps, and the place of apps in music-making.

Erratum noise music album cover

Every track on the album was made using freely available sound mangling apps of my own creation. This intersects with my current philosophies about music and music-making in a few ways.

First, by making all the apps publicly available, I’m basically open-sourcing the album. Okay, the apps aren’t open source (yet), but other musicians can now very easily make very similar music. I think this is a good thing. I hope people find my apps useful. But this is a significant change from my thinking of just a few years ago, which was dominated by a slightly-more-insular academic perspective. The academic perspective says something like “I put in a lot of working making the software, so why should I let just anyone use it, or copy my algorithms.” This is an attitude displayed often by the old-guard type of guys I learned from, and in my previous art albums like Disconnected, I took the same stance. With the continuing dominance of social media over good-old-fashioned-blogs, I’m starting to think that sharing is more important than building up my own ivory tower though, and I tried to do that with this album.

Second, this album is full of short pieces. I’m starting to come around to the idea reflected in Cage’s Sonatas and Interludes, which is that if you’re going to write weird music, it’s better to write many short pieces or movements than to write something monolithic. So each of the pieces on this album are short and unique. The album is held together only by the thread of the mobile apps used to make them.

Finally, this album reflects the increasing pleasure I get from listing to music that is very close to noise. Some listeners might call some of this music noise. One of the apps I used to create this album, Radio Synthesizer simply adds radio-like noise to an audio file in greater or lesser proportions. When I had my first child I remember putting her to sleep with white noise, and for awhile, white noise was 100% effective at putting her to sleep. I think that made me more appreciative of all the different ways that noise can be generated. This album reflects a lot of different ways of getting to and from a noise-like state.

Stream Erratum from evanxmerz.bandcamp.com or check out the apps I used to make it on Google Play.

Granular Synthesis for Android Phones

Granular is a granular synthesizer for Android devices. Play it by dragging your fingers around the waveform for the source audio file. You can upload your own audio files, or just play with the sounds that are distributed with the app.



The horizontal position on the waveform controls the location from which grains will be pulled. The vertical position controls the grain size. The leftmost slider controls the amount of frequency modulation applied to the grains. The middle slider controls the time interval between grains. The rightmost slider controls randomness.

Download Granular from the Google Play Store and start making grainy soundscapes on your phone.

Make Explosive Soundscapes with Circular Sound

I’ve just finished work on a new musical instrument for Android devices. It’s called Circular Sound, and it’s aimed at people who like noise.



Circular Sound is similar to my other recent mobile instruments in that it combines a sampler with custom digital signal processing, and a unique interface. Sounds can be loaded from a configurable directory on the device, or you can play around with the default sounds, which are from freesound.org. Then they are loaded into spheres that are arranged in a circle on the left half of the screen. The left half of the screen is the source audio mixer, while the right half is used to control effects. The effects include waveshaping, granulation, delay, and modulation.

The goal of Circular Sound is to give a simple access point into generating various types of noise that is related in some way to the source sounds provided by the user.

Download it for free on Google Play and shoot me a comment to let me know if you make something cool with it!