I have decided to move my blog over here, and from now on any updates will happen on that page rather than here from now on.
The adress is: http://kristinburdal.weebly.com/
I have been looking at a few games that I might want to make demos for, and would like to write about one that particularly caught my attention: The Rabbit and the Owl by developer Formal Sheep. The game was published on Steam Greenlight on the 11th of March, and crossed into the top 100 within 48 hours.
The game is set to be released in 2017, and the developers describe it as “[…] a single-player puzzle-platformer which introduces an interdependent 2×2-dimensional landscape. These two characters exist in a world in which a window for one is simultaneously a barrier for the other. You must help the Rabbit and the Owl navigate through the positive and negative spaces of mind-bending puzzles to discover the secrets of the strange land that they are a part of”.
The game already has two composers, David Huff and Patrick Neff, and judging from the announment trailer (below), they will compose some great and well fitting music for this game.
Posts about some of the other games I have looked at will follow in the next few days.
This is an update to let any potential readers know that this blog will from now on be used as a log for the work I do on a module at university, Negotiated skills practice. My aim for the module is to develop my portofolio as a video game composer by creating three pieces of music, or demos, for three different games. I plan to further develop one of these into an interactive demo by using either Wwise, FMOD or ELIAS Studio.
Until next time.
In this blog post we will look at generative music, what it is and different ways in which it has been used in interactive media like video games and musical toys. We will have a closer look at three different musical toys that uses different approaches to interactive generative music: Patatap, Plink and Tonfall Sequencer.
The term generative music can often time be a little confusing as different people have different definitions as to what it is. In broad terms, generative music is an algorithmic approach to the generation of music, which does not really clarify anything. Wooller et. al. (2005) exemplifies the confusion around the term generative music, and they look at some of the different meanings this term has been given:
Linguistic/Structural: Music created using analytic theoretical constructs that are explicit enough to generate music (Loy and Abbott 1985; Cope 1991); inspired by generative grammars in language and music, where generative instead refers to mathematical recursion (Chomsky 1956; Lerdahl and Jackendoff 1983).
Interactive/Behavioral: Music resulting from a process with no discernable musical inputs, i.e., not transformational (Rowe 1991; Lippe 1997, p 34; Winkler 1998).
Creative/Procedural: Music resulting from processes set in motion by the composer, such as “In C” by Terry Riley and “Its gonna rain” by Steve Reich (Eno 1996).
Biological/Emergent: “Non-repeatable music” (Biles 2002a) or non-deterministic music, such as wind chimes (Dorin 2001), as a sub-set of “Generative Art”.
(Woller et.al., 2005, p. 1)
Wooller et.al. further argues that the function of algorithmic music can be seen on a continuum that ranges from analytical, through transformational to generative. These terms are applied to describe the “effect of the process upon the data applied to it” (2005, p.8). They claim that an algorithm is generative when “the resulting data representation has more general musical predisposition than the input, and the actual size of the data is increased” (Wooller et.al, 2005, p. 9). This means an algorithm that outputs more information (music) than the initial information given (input).
Brian Eno, sometimes refered to as the person who coined the term ‘generative music’, defines it as a specific set of rules that will generate unpredictable music in real-time (Eno, 1996). Eno describes the creation of music on a continuum from what he calls classical music to generative music. In classical music an “entity” is specified in advanced, then it is built (Eno, 1996). For example, when writing down a score, different aspects such as pitch value, dynamic value, orchestration and playing technique can be specified, then musicians will follow these specifications in creating the music. This is opposed to generative music, where the specified rules generates the music. That is, there is no set score to follow, but rather a set of rules that defines what can possibly happen within the music. This means one can never be entirely certain of what will happen, unless the rules are very specific – in which case you might say it is not truly generative.
Applying the term classical for one end of the continuum may lead people to misunderstand what it actually means. The term “classical music” will likely give connotations either to the term as an umbrella term for art music, as a term describing late 18th and early 19th century classical music or as a term describing classical forms (i.e. sonata form). This is unfortunate, as it does not seem Eno only means classical music,but rather music in where the outcome is more or less pre-determined by the set of instructions given. By such, terms like pre-determined or fixed could be better suited, but these might also give the wrong connotations: Even though the outcome of generative music is not pre-determined, the rules that governs it are; even though the instructions (i.e. the score) for a performance are fixed, you can never truly know the exact outcome unless it is from a recording or completely computer based music. In all, it might not be particularly fruitful to see music as more or less generative or the opposite, but if we are to do so, the best way of describing the opposite of generative might just non-generative music.
Generative, or procedural, music was commonly used in early arcade games. Because there was not enough memory to store and playback pre-recorded sounds, both the sound effects and music was usually synthesised in real-time (Hamilton, 2015). Some other early examples of generative music used in games would include Ballblazer (LucasFilm Games, 1984), which uses an algorithm that the composer, Peter Langston, refers to as a riffology algorithm (Collins, 2009). The algorithm has a choice of 32 eight-note riffs, or melody fragments, to choose from, and it will take decisions like which riff to play next, whether or not to omit any notes from the riff, and further, if that should be done by prolonging another note or putting in a rest. Otocky (ASCII Corp, 1987), with music by Toshio Iwai, is another example of generative music in early games (Collins, 2009). Iwai later moved on to working with other games, or musical toys, that uses generative music, like SimTunes (EA, 1996) and later Electroplankton (Nintendo, 2005). Collins (2009) argues that these cannot really be seen as games as they have no set objectives, rewards or in-game narratives, therefore it is more fitting to call them musical toys. In SimTunes the player paints a picture, where each colour represents a musical tone. After that, the player places Bugz, which represents instruments, on the picture, and as they move over the different colours, music is played. In Electroplankton, the player can interact with different plankton on the screen, which output different music. Many different musical toys have been developed, and in the following we will look at three that uses different approaches to interactive generative music.
Patatap is a musical toy that is available on Android and iOS as well as being an interactive website. The creator, Jono Brandel, describes it as a “portable animation and sound kit”. In the website version, the user can press different keys on the keyboard which output sounds accompanied by visuals shapes on the screen. Keys A to Z outputs sounds and shapes, while the spacebar changes the visual layout and the sounds. As there are quite a few keys to choose from, each containing individual sounds, it takes some time to properly learn how it works in order to create music that does not sound like a selection of random sounds. It might work better when using a touch screen, but as this blog post is focused around websites rather than applications this has not been tested. However, it seems like the system has been quite successful, as it has been used as a part of different installations and performances (Brandel, 2014).
Plink is described as “a super intuitive multiplayer experience” on the website. It is a musical toy in which different players from around the world can create music together. The design is very intuitive: There are 15 horizontal lines running down the screen, and clicking the mouse within any of those lines outputs a pitch. In the sidebar the user can select different colours, which each correspond to different sounds. The musical content is based around the pentatonic scale. This means that it is a fairly limited set of rules that governs the output, and the musical possibilities are by such limited. However, this also means that it is easier to create music that makes sense musically, no matter if the user has a musical background or not. It also makes it easier for multiple users to create something that sounds ‘good’ together. This would become more difficult if the musical content was less limited, by for example having the diatonic scale, as every tone of the diatonic scale is not in consonance with every other tone. The pentatonic scale also has a very organic feel to it, and lends it self well to the creation of simple melodies: Just by dragging the mouse up and down in Plink, something musical and melodic will be created.
Tonfall Sequencer is described as “an experimental particle based audio sequencer, created in Flash using Tonfall; the new open source AS3 audio engine” (Windle, 2010). Windle (2010) goes on to explain that Tonfall is designed to get people started with audio programming in Flash. On the website, the user is presented with a screen containing various neurons and receptors. Connecting them will produce a variety of musical pitches. At the bottom of the screen, there are three sliders that lets the user control the number of neurons and receptors on the screen, as well as their proximity, that is, how far from each other the neurons and receptors have to be in order to connect. There can be a total of 5 neurons and 24 receptors. Besides the sliders, there are also two buttons that can be clicked: ‘Wander’ and ‘spectrum’. The ‘wander’ option allows the user to decide whether the neurons and receptors should move or remain static in one place; the ‘spectrum’ option shows a simple spectrum analysis of the sounds created. In the Flash player the used may drag the neurons and receptors around to make them connect with each other, or she may just turn on ‘wander’ and let the music create itself. Sounds in different pitch ranges will be generated depending on the type of neuron the receptors connect to. Windle (2010) claims that the receptors reacts to connection with the neurons by playing a randomly assigned octave, “depending on which neuron causes it to fire”. This is not entirely true: There are three different pitch ranges, so two pairs of neurons have the same pitch range, while one has the third range: Two are in a lower pitch range, two are in a middle pitch range and one is in a high pitch range. Which neurons have the same sound, is not colour coordinated, but the pitch value is related to the size of the receptors. Each time you restart the sequencer, the colour of the neurons and receptors change, so in that way you could say it is random, but the sound outputs will always be based in the same tonality. There are three different pentatonic scales that make up the musical material: The lowest pitched neurons have the pentatonic scale starting from A (A-B-D-E-F#); the middle pitched neurons have the pentatonic scale starting from E (E-F#-A-B-C#), and the highest pitched neuron has the scale starting from B (B-C#-E-F#-G#). When these three pentatonic scales are combined, all the notes of an A major scale is present. This means that it would probably be more accurate to say that the colours of the neurons and receptors are assigned at random, rather than saying the sound is random.
Brandel, J. (2014) Patatap [Online]. Available from: <http://www.patatap.com/> [Accessed 30/11/15]
Brandel, J. (2014) “Patatap” [Online]. Jonobr1. Available from: <http://works.jonobr1.com/Patatap> [Accessed 01/12/15]
Collins, K. (2009) “An introduction to Procedural Music in Video Games”. Contemporary Music Review, Vol. 28, No.1. 5–15.
Dinahmoe Labs (n.d.) Plink [Online]. Dinahmoe Labs. Available from: <http://dinahmoelabs.com/plink> [Accessed 31/11/15]
Eno, B. (1996) “Generative music” [Online]. Motion Magazine. Available from <http://www.inmotionmagazine.com/eno1.html> [Accessed 01/12/15]
Hamilton., R. (2015) “Designing Next-Gen Academic Curricula for Game-Centric Procedural Audio and Music” [Online]. AES 56th International Conference: Audio for Games. London. Available from: <http://www.aes.org/tmpFiles/elib/20151125/17595.pdf> [Accessed: 10/11/15].
Klazien (2014) “Top 5: Interactive Generative Music Sites” [Online]. Submarine Channel. Available from: <http://www.submarinechannel.com/top5s/top-5-of-our-favourite-things-interactive-generative-music-sites-2/> [Accessed 30/11/15].
Windle, J. (2010) Tonfall Sequencer [Online]. Soulwire Art & Technology. Available from: <http://blog.soulwire.co.uk/wp-content/uploads/2010/10/tonfall-sequencer.swf> [Accessed 30/11/15]
Windle, J. (2010) “AS3 Particle Node Sequencer” [Online]. Soulwire Art & Technology. Available from: <http://blog.soulwire.co.uk/laboratory/flash/as3-tonfall-particle-node-sequencer> [Accessed 01/12/15]
Wooller, R., Brown, A. R., Miranda, E., Berry, R. & Diederich, J. (2005) “A framework for comparison of processes in algorithmic music systems” [Online]. Generative arts practice Sydney: Creativity and Cognition Studios Press. 109–124. Available from: <http://cmr.soc.plymouth.ac.uk/publications/gap05.pdf> [Accessed 12/11/15].
Repetition can be, and often is, as big of a problem with video game music as it is with video game sound in general. One of the main reasons for this, just like with the sound design, being that video games are an interactive media, which makes it difficult to compose music that will adapt to every possible outcome of gameplay. One way of scoring a game is to loop music tracks in certain areas. This is a technique used in a great variety of games, all through video game history; You have the looping music in Super Mario Bros. (Nintendo, 1985) and you have looping music in Dark Souls (From Software, 2011). It should be mentioned, though, that some composers have gone to greater lengths in making the loops feel less repetitive, which is the case of the music for Super Mario Bros. Here Koji Kondo has incorporated out-of-order repetitions, which means that one section will not repeat after the same section twice (Schartman, 2015). This can create the illusion of the music not being as repetitive. Even though looping music is interactive in the sense that the player holds some control over when the loop is being played and not (Stevens and Raybould, 2011), it can still become and feel repetitive because the player might spend a lot of time in a given area. Some games give a solution to this by not having the music loop endlessly; Instead the music will fade out after a set amount of time (Collins, 2009). This, however, might make the player feel like the game is waiting for them to finish, which can be hurtful to the gaming experience and potentially lead to a break in immersion.
Games released on the Xbox 360 (Microsoft, 2005) must give the player the possibility of turning the music of, and even substitute the music with their own choice of music (Collins, 2009). It is also becoming increasingly normal that games released on any platform offer the opportunity to adjust the volume of both music and sound. In one way, this lets the player adjust the sound to their own preferences. This might be regarded as a positive feature, as players will value different parts of the sound mix: Some players might feel that the sound design is of the most importance, while others will feel that the music is more important. As the player gains more control over the sound, it also becomes more interactive. It might also make the sound feel less repetitive, as the player can choose to change the sound mix with every playthrough. What some – the composers and sound designers especially – might consider the downside to this, is that this possibility also enables the player to turn the sound and music completely off. Some consider the possibility of turning the audio off as a “testament to the power of audio: that audio is such an important part of the experience that if a player doesn’t like it, it is better to turn it off than to ruin the game” (Vachon, 2009, pp. 1–2). In some instances, it might be good to have the possibility to turn the music off; When you are playing on your mobile device on the bus and you forgot your headphones being one of them. Another instance, pointed out by Stephen Totilo (2011) is that turning the music down, gives the player the opportunity to multi-task, i.e. by listening to a podcast instead of listening to the game. In the case of the former, it is in some way understandable that the music and sound in general will have to be turned down – many of us will have experienced how annoying it can be when someone is playing music out loud on the bus. Turning it down thus becomes a sign of respect for the people around you. In the case of the latter, however, one question begs to be asked: “Why is it that we play video games to begin with?” People will have different answers to this question, and to some it may indeed be to “have something to do while I listen to my podcasts”. It might also be possible that some players feel more immersed by either playing their own music, playing no music all together, or it might even be possible that the type of music played is not as important as one might think. A study on the concept of cognitive immersion, conducted by Rod Munday (2007) suggests that although the presence of music is important in immersing a player in a game, the choice of music is not as important. Munday states that music helps to create immersion into video games, by “occupying the area of the brain dedicated to dealing with non-linguistic sounds”, which then “prevents this area from hunting around for stimuli outside the game” (p. 57). In perceiving audio, the brain is capable of hearing any number of sounds, while still being able to focus on one sound and by that exclude less relevant sounds (Munday, 2007). In the context of video games, music is only but one out of many audio sources. Thus, Munday implies that music, even though it is of importance to the experience of the game, will be one of the sounds the brain does not focus on – making the type of music playing irrelevant.
If music is superfluous, or if the type of music has no real impact on the degree of immersion the player feels, why is it that the music scores for video games are so highly regarded by many players? Every year, video game music is being played in concert halls around the world, in shows such as Video Games Live; there is an online radio channel dedicated to the music of Final Fantasy. If the music really has no impact on the player’s experience of a game, would we then remember these scores, and seek them out outside of the game? One argument could be that even though the music is memorable, we would still feel the same level of immersion without the music, and that the brain works in twofold; We acknowledge the music for what it is – good music – while a separate process is at work in creating our sense of immersion. In order to fully understand in which way the type of music may or may not impact the player’s experience of the game, it is also necessary to take into consideration the different levels and kinds there are of immersion. While it may be possible to feel immersed in a game with any kind of music playing, the player might not get the experience of being fully immersed. As playing a game is a highly subjective experience, it would be difficult to accurately measure the impact music does or does not have. The answer might also be as easy as that: It varies from player to player.
In some games, turning the music of might be the result of the player feeling there is too much music. This seems to be one of the reasons that lead Totilo (2011) to turn the music off in Read Dead Redemption (Rockstar, 2010), as he explains that he would not turn off the music in those instances where the music is of utmost importance – for example the long ride home at the end of the game. The lack of music can be just as effective in creating a sense of presence, and even more so, the lack of music could have the possibility of making those instances with music feel more powerful. To quote Truman Fisher: “the pause is as important as the note”. A good example of this can be seen in Dark Souls (From Software, 2011), where, in the open world, there is only music in certain areas – most of which are safe areas in the game world. The music becomes extra effective here in that the player will quickly learn to associate lack of music with danger and the presence of music with safety (Burdal, 2014).
As music in games is becoming more and more interactive, in the sense that it is a part of the game’s core mechanics rather than it just being there for enhancing the mood or working as a response system for the player, we might come to see that the doubt in whether or not music is important will vanish. One game that uses the music in such a way is Journey (Thatgamecompany, 2012). In Journey, both the music and sound design work in such a way that the game would most likely feel very empty and shallow without it. As this blog is mainly focused around video game music, the sound design will not be discussed in depth, although the music and sound design often blends together in the game. This is evident in the song talk that the character does as the player pushes a button. This sound is a mixture of re-pitched and processed bird sounds, made by Steve Johnson, the sound designer, musical elements made by Austin Wintory, the composer, and the vocals of a singer. The musical elements are further varied to fit in tonality and style with the music of the different levels (Johnson, 2012). As the player hits the circle button, the character will make a chiming/chirping sound (song talk). The length and quality of this sound varies, depending on the length of which the player holds the button. Johnson (2012) explains this function as such “the singing in the game is of four types; a light quick button press for a ‘coo’, a hard quick press for a ‘chirp’, a reasonably-held press for a ‘call’, and a long-held press for a large ‘shout’.” When the player has a companion – is playing with another player – the two different characters will also make different sounds, so they are easily distinguishable. Although this sound technically belongs in the sound design realm of the game, the musical elements added by Wintory, could make these sounds seem more like musical stingers, responses to the player upon interaction with the controller. Similar musical stingers also accompanies the player when picking up fabric for the scarf. Many other points can be made about the music, just by looking at the first level alone: The general ambience music of the first level is mostly made up of droning sounds or organ points. However, as the player approaches objects in the game world, the music will change and a variation of the Journey theme will play.
Here the music is first at the droning state, and as the player approaches the stone with the rune symbol and the pieces of fabric, and then absorbs the rune, the theme starts playing. As the player floats away from this area, the music goes back to the previous state. As you can still hear the drones play underneath the theme, it would seem that a parallel, or layered approach has been used as a technique here. This musical event informs the player of the fact that picking up fabrics are important, and also that they should look for more runes. This part also shows the close entanglement between the sound design and the music, as the sounds the fabric and rune makes, just as easily could be regarded as a part of the score as of the sound design. These instances keep happening as the player goes through the first level, and in all cases it seems like a parallel/layered technique has been used.
Going through the game in detail, would show that there are so many other things to point out, that really strengthens the argument for why music is of great importance in video games. In the case of Journey, it would simply be impossible to imagine the game without music – as the music moves and evolves with every action done by the player. It could be argued that the music is indeed a dynamic part of the gameplay itself. Even though not all games are Journey, and even though the music of most video games does not aim to do what the music of Journey does, this example should show the importance music can have on the experience of a game, and we might discover that this is the case for more games than we think. To take from personal experience with a completely different game: In retrospect of playing The Elder Scrolls V: Skyrim (Bethesda, 2011), I’ve realised that I most likely would not regard that game as highly had it not been for the music. I’ve even contemplated on whether or not I would even like it – that, of course, is just the opinion of one against many, and cannot be used as a base of anything. While the importance of music will depend on the type of game, players should at the very least give the music a chance to see how it might impact the experience before turning it of.
Borda, Melissa (2013) “Journey: A Critical Analysis” [Online]. Arcade : The Best from Game Design at VFS. Available from: <http://community.vfs.com/arcade/2013/07/journey-a-critical-analysis/> [Accessed 13/11/15]
Broomhall, John (2013) “Heard About: Journey” [Online]. Develop. Available from <http://www.develop-online.net/analysis/heard-about-journey/0117627> [Accessed 13/11/15]
Collins, Karen (2009) “An Introduction to Procedural Music in Video Games”. Contemporary Music Review. Vol. 28, No. 1. 5–15.
Final Fantasy Radio [Online]. Available from <http://finalfantasyradio.co/> [Accessed 15/11/15].
GadgetGirlKylie “JOURNEY PS4 1080P 60FPS No Commentary FULL Complete Walkthrough” [Online]. YouTube. Available from <https://www.youtube.com/watch?v=mJVs4Y71Dsc> [Accessed 15/11/15].
Johnson, Steve (2012) “The Sound Design of Journey” [Online]. Gamasutra: The Art & Business of Making Games. <http://www.gamasutra.com/view/feature/179039/the_sound_design_of_
journey.php?print=1> [Accessed 13/11/15]
Leigh, Alexander (2012) “Journey’s composer finds a game’s soul through his music” [Online]. Gamasutra: The Art and Business of Making Games. Available from: <http://www.gamasutra.com/view/news/183051/Journeys_composer_finds_a_games_soul_through_his_music.php> [Accessed 13/11/15]
Munday, Rod (2007) “Music in Video Games”. In J. Sexton (ed.) Music, Sound and Multimedia: From the Live to the Virtual. Edinburgh: Edinburgh University Press. Pp. 51–67.
Schartman, Andrew (2015) Koji Kondo’s Super Mario Bros. Soundtrack. New York; London: Bloomsbury Publishing Plc.
Stevens, Richard and Dave Raybould (2011) The Game Audio Tutorial. A Practical Guide to Sound and Music for Interactive Games. Boston: Focal Press.
Totilo, Stephen (2011) “The Year I Gained Courage to Ignore Video Game Music” [Online]. Kotaku. Available from <http://kotaku.com/5730637/the-year-i-gained-the-courage-to-ignore-video-game-music> [Accessed 11/11/15]
Vachon, Jean-Frederic (2009) “Avoiding Tedium – Fighting Repetition in Game Audio” [Online] 35th International Conference: Audio for Games. Available from <http://www.aes.org/e-lib/browse.cfm?elib=15158> [Accessed 16/10/15]
Video Games Live [Online]. Available from <http://www.videogameslive.com/index.php?s=home> [Accessed 15/11/15]
Procedural techniques and procedural audio has been used in games almost since the beginning (Fournel 2010), as it was a necessity in the arcade games with limited memory. The trend seemed to disappear when home consoles got increasingly more powerful, but lately, procedural audio has regained some attention – especially in academia. Andy Farnell (2007) defines procedural audio as such “ Procedural audio is non-linear, often synthetic sound, created in real-time according to a set of programmatic rules and live input”.
There are two main approaches to procedural audio (Fournel, 2010): Bottom-up and top-down. In the bottom-up approach, the sound designer examines how the sounds are physically produced, and then write a system to recreate them. This is also known as telelogical modelling. In the top-down approach, the sound designer analysed examples of the sounds she wants to create, then find the adequate synthesis system to emulate them. The top-down approach is also known as ontogenetic modelling. In an interview with N. Varun (2014), Fournel argues that the bottom-up approach is overcomplicated and requires too much knowledge of audio synthesis and audio production mechanisms to be approachable to most sound designers. He then talks in favour of the top-down approach, which he believes could “help creating more convincing sound effects”. He argues that this approach would also be a lot easier to accomplish for most sound designers, and not just for what he calls “a new breed of ‘technical’ sound designers”. Farnell, on the other hand, talks more in favour of the bottom-up approach. According to Farnell (2010), this method allows the sound designer to create sound objects that can be modified in real-time to mimic the unpredictable nature of real world sounds. This approach, however, is still not fully viable, and some critiques would include that the sounds do not resemble real-life sounds, yet. However, both approaches are interesting in that they might be a solution to the ever excisting problem of memory made available for audio in games, and there are some examples of procedural audio in use which are promising.
Sim Cell is an example of procedural audio used in game. Designed by Leonard Paul, the sound and music was created using the visual programming language Pure Data (Paul, 2015). When talking about the generative music for the game, Paul (2015) argues that this system “when combined with synthesis […] allows for the creation of rich and highly adaptive compositions utilizing a small amount of storage space”. Another example is the upcoming 2016 title No Man’s Sky (Hello Games), where both the visual content and the audio will be created using procedural methods. The audio director, Paul Weir, explain that they have created a plug-in which models vocal tracts, and within that the feature of the creature making the sound can be specified. Further these parameters are mapped into the game, and this results in the sound that creature is making.
Barbosa, Alessandro (2015) “No Man’s Sky’s procedural audio is pure musical wizardry” [online]. LazyGamer. Available from: <http://www.lazygamer.net/genre/sim-genre/no-mans-skys-procedural-audio-is-pure-musical-wizardry/> [Accessed 25/10/15]
Farnell, Andy (2007) An introduction to procedural audio and its application in computer games [online]. Obiwannabe. Available from: <http://www.obiwannabe.co.uk/html/papers/proc-audio/proc-audio.html> [Accessed 25/10/15].
Farnell, Andy (2010) Designing Sound. Cambridge, Mass: MIT Press
Fournel, Nicolas (2010) Procedural Audio for Video Games: Are we there yet? [Online]. GDCVault. Available from: <http://gdcvault.com/play/1012645/Procedural-Audio-for-Video-Games> [Accessed 25/10/10]
Nair, Varun (2012) “Procedural Audio: An Interview with Nicolas Fournel” [Online]. Designing Sound. Art and technique of sound design. Available from: <http://designingsound.org/2012/06/procedural-audio-an-interview-with-nicolas-fournel/> [Accessed 26/10/15].
Paul, Leonard (2015) “The Generative Music of Sim Cell” [Online]. AES 56th International Conference: Audio for Games. Available from: http://www.aes.org/e-lib/browse.cfm?elib=17590 [Accessed 30/10/15].
Vachon, Jean-Frederic (2009) “Avoiding Tedium – Fighting Repetition in Game Audio” [Online] 35th International Conference: Audio for Games. Available from: <http://www.aes.org/e-lib/browse.cfm?elib=15158> [Accessed 16/10/15].
Sounds, in nature, are unpredictable and non-repetitive. While one sound may sound almost exactly the same as another, it will always have some characteristics separating it from other sounds. One issue sound designers face when creating a soundscape for a virtual game environment is achieving this same sense of randomness as is found in nature. Although a virtual game world does not need to be a realistic alternative to our own universe, we go into it with the expectation of being immersed into a believable universe. An important part of this, is creating a believable soundscape that will help us be further immersed into the world. However, with limited amount of memory, there are limits to how much sound can be implemented in a video game, which in turn can lead to sounds becoming repetitive. The human brain can quickly pick up on repeating patterns, and a very repetitive sound design will then become tiring to listen to, which could eventually lead to breaking the immersion – the game world no longer being believable to the player. In order to work around this issue, methods to create a non-repetitive sound design with a limited amount of sound samples need to be implemented. One approach to non-repetitive sound design in video games is the use of granular synthesis. In granular synthesis, a pre-recorded sound sample is segmented into short sound events, or grains, that can then be modified and played back in virtually limitless combinations (Paul, 2011). In this text, we will look closer at what granular synthesis is, and how this method can be used in order to create a non-repetitive sound design in video games.
Historically, granular synthesis has its roots in electro-acoustic music. Two names of significance are Curtis Roads, the first person to implement a non-real-time granular synthesis in 1974, and Barry Turax, the first person to do a real-time implementation in 1986. The initial idea, however, is even older than that, and stems from Dennis Gabor. In 1947 he proposed that any sound could be created by combining very short grains of sounds (Paul, 2008).
Using granular synthesis offers a variety of ways to modify the sound in order to make it non-repetitive. These includes the possibility to modify playback rate, amplitude range, spatialisation, grain duration, grain density, envelope and DPS effects. Normally, a grain will have a duration of between 1 and 100 ms. Paul (2011), however, suggests that in games it might be preferable to use longer grains for certain sounds, like ambience, as this might help preserve some core characteristics of the sound. He goes on to talk about three different methods of sound granulation, which might be used in games: Manual granulation, where the sound is broken into component parts, and then implemented into the game in real-time. This could be a useful technique in creating an ambience, where some sounds, or grains, could play more or less continuously, while other sounds might be triggered at random. Here it would also be possible to have some of the sounds adapt to gameplay, to create an interactive experience. The other two techniques are both automated, where one uses short grains, and the other long grains. The latter bears some resemblance to concatenative sound synthesis (Schwarz, 2009). This method uses a large database of sounds, where a sound sample is segmented into units. A unit selection algorithm then finds a sequence of units to best match the initial sound. .
The fact that granular synthesis offers so many possibilities for modification, gives the opportunity to create a vast and diverse non-repetetive soundtrack without having to use too much memory. This method could then be very effective if used in gaming devices with little memory, like handheld consoles and mobile phones (Paul, 2011). The use of pre-recorded sound samples also gives this method an edge in comparison with other methods that uses synthesised sounds. This is especially true for the games that aim for a more ‘realistic’ virtual environment. Paul (2011) recognises that some of the difficulties with the method is “discovering how to modulate parameters from the physics engine to arrive at the desired aesthetic result”, and “discovering intelligent ways to turn the raw data from the physics engine into numbers that controls the granulation respecting which follows the aesthetic goals of the sound designer”.
Granular synthesis is an interesting approach to non-repetitive sound design in video games. It offers numerous ways in which sounds can be modified in real-time to create a dynamic, non-repetitive soundscape in virtual game enviroments. Granular synthesis is a fruitful solution to working around the limited memory available for sound, and it would especially be a good method to use for sound implementation is handheld consoles and mobile phones. It remains to be seen how this method will develop in the future.
Parker, Jim (2013) “Creating Sound Effects and Sound Textures from Examples” [online]. GDC Vault. Available from <http://www.gdcvault.com/play/1017739/Creating-Sound-Effects-and-Sound> accessed 12/10/15.
Paul, Leonard (2015) “Advanced Topics in Video Game Audio” [Online]. Video Game Audio. Available from <http://videogameaudio.com/> accessed 12/10/15.
Paul, Leonard (2011) “Granulation of Sound in Video Games” [online], AES 41st International Conference: Audio for Games Available from: <http://www.aes.org/e-lib/browse.cfm?elib=15760> accessed 11/10/15.
Paul, Leonard (2008) “An Introduction to Granular Synthesis in Video Games” in Karen Collins (ed.) From Pac-Man to Pop Music: Interactive Audio in Games and New Media. Aldershot: Ashgate. 135–149.
Picard, Cécile, Nicolas Tsingos and Francois Faure (2009) “Retargetting Example Sounds to Physic-Driven Animations” [online]. AES 35th International Conference, Audio in Games. Available from: <http://www.aes.org/e-lib/browse.cfm?elib=15179> accessed 11/10/15.
Schwarz, D (2006) “Concatenative Sound Synthesis: The Early Years” [online]. Journal of New Music research, 35(1). 3–22. Available from: <http://articles.ircam.fr/textes/Schwarz06b/index.pdf> accessed 11/10/15