Sounds, in nature, are unpredictable and non-repetitive. While one sound may sound almost exactly the same as another, it will always have some characteristics separating it from other sounds. One issue sound designers face when creating a soundscape for a virtual game environment is achieving this same sense of randomness as is found in nature. Although a virtual game world does not need to be a realistic alternative to our own universe, we go into it with the expectation of being immersed into a believable universe. An important part of this, is creating a believable soundscape that will help us be further immersed into the world. However, with limited amount of memory, there are limits to how much sound can be implemented in a video game, which in turn can lead to sounds becoming repetitive. The human brain can quickly pick up on repeating patterns, and a very repetitive sound design will then become tiring to listen to, which could eventually lead to breaking the immersion – the game world no longer being believable to the player. In order to work around this issue, methods to create a non-repetitive sound design with a limited amount of sound samples need to be implemented. One approach to non-repetitive sound design in video games is the use of granular synthesis. In granular synthesis, a pre-recorded sound sample is segmented into short sound events, or grains, that can then be modified and played back in virtually limitless combinations (Paul, 2011). In this text, we will look closer at what granular synthesis is, and how this method can be used in order to create a non-repetitive sound design in video games.
Historically, granular synthesis has its roots in electro-acoustic music. Two names of significance are Curtis Roads, the first person to implement a non-real-time granular synthesis in 1974, and Barry Turax, the first person to do a real-time implementation in 1986. The initial idea, however, is even older than that, and stems from Dennis Gabor. In 1947 he proposed that any sound could be created by combining very short grains of sounds (Paul, 2008).
Using granular synthesis offers a variety of ways to modify the sound in order to make it non-repetitive. These includes the possibility to modify playback rate, amplitude range, spatialisation, grain duration, grain density, envelope and DPS effects. Normally, a grain will have a duration of between 1 and 100 ms. Paul (2011), however, suggests that in games it might be preferable to use longer grains for certain sounds, like ambience, as this might help preserve some core characteristics of the sound. He goes on to talk about three different methods of sound granulation, which might be used in games: Manual granulation, where the sound is broken into component parts, and then implemented into the game in real-time. This could be a useful technique in creating an ambience, where some sounds, or grains, could play more or less continuously, while other sounds might be triggered at random. Here it would also be possible to have some of the sounds adapt to gameplay, to create an interactive experience. The other two techniques are both automated, where one uses short grains, and the other long grains. The latter bears some resemblance to concatenative sound synthesis (Schwarz, 2009). This method uses a large database of sounds, where a sound sample is segmented into units. A unit selection algorithm then finds a sequence of units to best match the initial sound. .
The fact that granular synthesis offers so many possibilities for modification, gives the opportunity to create a vast and diverse non-repetetive soundtrack without having to use too much memory. This method could then be very effective if used in gaming devices with little memory, like handheld consoles and mobile phones (Paul, 2011). The use of pre-recorded sound samples also gives this method an edge in comparison with other methods that uses synthesised sounds. This is especially true for the games that aim for a more ‘realistic’ virtual environment. Paul (2011) recognises that some of the difficulties with the method is “discovering how to modulate parameters from the physics engine to arrive at the desired aesthetic result”, and “discovering intelligent ways to turn the raw data from the physics engine into numbers that controls the granulation respecting which follows the aesthetic goals of the sound designer”.
Granular synthesis is an interesting approach to non-repetitive sound design in video games. It offers numerous ways in which sounds can be modified in real-time to create a dynamic, non-repetitive soundscape in virtual game enviroments. Granular synthesis is a fruitful solution to working around the limited memory available for sound, and it would especially be a good method to use for sound implementation is handheld consoles and mobile phones. It remains to be seen how this method will develop in the future.
Parker, Jim (2013) “Creating Sound Effects and Sound Textures from Examples” [online]. GDC Vault. Available from <http://www.gdcvault.com/play/1017739/Creating-Sound-Effects-and-Sound> accessed 12/10/15.
Paul, Leonard (2015) “Advanced Topics in Video Game Audio” [Online]. Video Game Audio. Available from <http://videogameaudio.com/> accessed 12/10/15.
Paul, Leonard (2011) “Granulation of Sound in Video Games” [online], AES 41st International Conference: Audio for Games Available from: <http://www.aes.org/e-lib/browse.cfm?elib=15760> accessed 11/10/15.
Paul, Leonard (2008) “An Introduction to Granular Synthesis in Video Games” in Karen Collins (ed.) From Pac-Man to Pop Music: Interactive Audio in Games and New Media. Aldershot: Ashgate. 135–149.
Picard, Cécile, Nicolas Tsingos and Francois Faure (2009) “Retargetting Example Sounds to Physic-Driven Animations” [online]. AES 35th International Conference, Audio in Games. Available from: <http://www.aes.org/e-lib/browse.cfm?elib=15179> accessed 11/10/15.
Schwarz, D (2006) “Concatenative Sound Synthesis: The Early Years” [online]. Journal of New Music research, 35(1). 3–22. Available from: <http://articles.ircam.fr/textes/Schwarz06b/index.pdf> accessed 11/10/15