Procedural Sound Design

The concept of procedural sound design aims to algorithmically approach sound within games through the use of dynamic sound systems, working in tandem with the ludic functions of game audio as to assist a player in achieving mastery within a game; and as to further immerse the player through the creation of game audio systems that persuasively replicate a sense of realism. (Stevens, 2016)

Though the philosophy of procedural sound design and procedural audio have comparability, they are not to be confused, and their methods are what separates them from one another, with procedural sound design making use of pre-recorded audio files. For example Foley sound, and procedural audio originating by means of synthesis to create sound and music in real time. In “Designing Sound” Farnell, 2010, proposes that the genesis of game audio began via the utilisation of procedural technology, however, has not always made use of pre-recorded assets in the way that procedural sound design does to perceive realism, as procedural sound design works as to make the best use of preexisting audio, and it could therefore be argued that procedural sound design in essence is an evolutionary advancement of procedural audio in relation to game audio in its own right. However, procedural sound design takes a step backwards when it comes to future proofing its systems. In a game like Sim Cell, Composer Jordan Paul utilizes procedural audio to create any and all music and sound effects in the game. He states that this method is future proof, as the fidelity of the sounds created match the power of the engine it resides in (Paul, 2014). Unfortunately for procedural sound design, its reliance on pre-existing audio samples cements the audio fidelity in whatever quality it is uploaded in.

Why Games?

Procedural sound design lends itself to the medium of games more naturally than other forms of multimedia. Film audio for example is an exclusively linear experience, games are non-linear, and serve to be interactive, the distinction between passive and active interaction legitimises procedural sound design for game audio. In comparison to most linear media such as film, the user experience tends to be longer, requiring multiple sessions in order to progress (Sweet, 2014). Therefore it is important for a sound designer to utilise procedural sound design in games, to create convincing, seamless audio.

Why use Procedural Sound Design?

Commonly, though dependant on game genre, a player will have to apply homogenous actions repeatedly within game, as most games require a certain level of skill to be achieved, thus the player will subsequently be hearing repeated audio alongside these actions, which is not sonically comparable to a real-world setting (Stevens, 2016) and therefore takes away from the authenticity of player experience.

Farnell, 2010, states that “a recorded sound always plays precisely the same way, whereas procedural sound may be interactive with continuous real-time parameters being applied” and while repetition can be an important aspect of certain factors of game audio (Stevens, 2016), a key component of procedural sound design is the idea of utilising non-repetitive design when manipulating audio assets in order to construct varied results, assisting the interactive nature of gameplay and further supporting the immersion of the player in doing so. An example of non-repetitive design in games can be examined through the use of audio recordings which can be categorised by sample, such as that of isolated footstep sounds. When used with blending techniques such as pitch shift and modulation, these serve to communicate how a player perceives their environment (Stevens and Raybould, 2015). The creation of several versions of a sound would prove strenuous and inefficient, paired with additional limitations regarding file size within a project. (Stevens, 2016) and therefore, employing a procedural approach to sound design accommodates the needs of both player and developer, saving time in editing and enhancing player experience. Literature regarding procedural sound design suggests that repetition is something to be avoided in the realm of game audio concerning interactivity, allowing for reactive sounds rather than passive, which can be implemented through use of game variables/parameters (Stevens, 2016).

How?

There are multiple ways to create procedural sound systems. In the case of engines like Unity or Unreal, they have in engine tools which can create these audio systems. An example of such tools would be the sound cues function in Unreal 4 (fig. 1). Here we can see a basic system for bird noises in the mysterious forest example (Epic Games, n.d). Here we can see the components of a sound are randomly paired and delayed such that the sounds composite randomly but are still in order. Middlewares like Fmod and Wwise also have these functionalities. Furthermore, Musical programming languages like Max/MSP or Pure data can also utilize procedural sound design. In fig. 2 we can see the game Sim Cells audio system is entirely procedural. However, Jordan Paul (Composer/Sound Designer for Sim Cell) utilizes procedural audio in order to create the sounds for Sim Cell. We can retool his system to make a procedural audio quite easily: instead of getting sound from synthesized patches we can use Pure Data or Max’s sample or playlist objects to pull audio from samples. In the same way we can modulate pitch, delay, reverb, and other effects in Unreal’s sound cue tool, we can use similar objects in Max and Pure Data to achieve these same functions. Using Middleware or Musical programming languages allows us to create these sound systems regardless of the engine we choose to make our game in (Paul, 2014).

(fig. 1 a screenshot from Unreal’s tutorial on procedural sound design by Dr. Richard Stevens)

(fig. 2 video example “The Generative Music and Procedural Sound Design of Sim Cell” via YouTube)

Procedural sound design is an effective method to adapt sound to the non-linear nature of games. While it does have its pitfalls in futureproofing, as long as games engines such as Unity and Unreal exist, it remains relevant.

References:

Stevens, R.S., and Raybould, D.R. (2016) Game Audio Implementation A Practical Guide Using The Unreal Engine. Paperback edn. New York and London: Focal Press.

Stevens, R.S. (2016) A Sound Effect, Why Procedural Game Sound Design is so useful – demonstrated in the Unreal Engine. [Blog] 18/01/2016. Available at https://www.asoundeffect.com/procedural-game-sound-design/ [Accessed 09/10/2021].

Farnell, A. (2010) Designing sound. Cambridge, Mass.: MIT Press, pp.315 – 319.

Stevens, R. and Raybould, D. (2015) The reality paradox: Authenticity, fidelity and the real in Battlefield 4. The Soundtrack, 8(1), pp.58-60.

Sweet, M. (2014) Writing interactive music for video games. Crawfirdsville, Indiana: Pearson Education, pp.16-17.

Epic Games (n.d) Fig 1: Ambient and Procedural Sound Design [Online video]. Available from: <https://learn.unrealengine.com/course/3547811?r=False&ts=637696367685925768&gt; [Accessed 11 October 2021].

School of Video Game Audio. (2013) The Generative Music and Procedural Sound Design of Sim Cell [Online video]. November 2013. Available from: < https://www.youtube.com/watch?v=0xr4aL1C24E > [Accessed 12 October 2021].

Paul, L, J. (2014). The Procedural Sound Design of Sim Cell. Audio Engineering Society, 137(9209).