I recently performed at Culturelab in Newcastle as part of the RTV workshops. It was a good opportunity to test out some of my pieces to an audience and also discuss the methods I used to construct them. (some photos of the event here) I’ve since recorded the pieces and put them on youtube and there are some screenshots on Flickr. Ive had a few people ask for some more technical details on how I designed the system. So here is a brief rundown.

The performance system is almost totally automated. This doesn’t mean it is predefined, both the audio and visual components are generative, ie they are variable executions of a pre-trained algorithmic system. For the audio I use Pure Data (PD). This is a visual programming DSP (Digital Signal Processing) system that I have used for years (its an open source alternative to MAX/MSP). I wont go into too much detail on how I program generative audio in, but have posted on this blog before about PD with some downloadable examples etc.

FROM PD

The sub patch in PD that handles sending to osc is shown above. Each ‘song’ consists of a generative audio composition in PD, this sequences and processes an abstract electronic composition in real time, triggering samples, applying effects and playing with time signatures and so on. In most cases the parameters that control the compositional arcs in the audio generation are then sent to Unity via a network messaging service called OSC. (in the above image all the [r ch] & [s osc] boxes are routing these parameters from inside the core of the pd patch to the osc sending part)This allows me to run the two programs in parallel and send specific variables across tcp/ip so Unity can react to changes in the composition. It means that the two aspects could even run on remote machines.

TO UNITY

In unity a series of scripts listen to incoming UDP data on the relevant port (6666 in this case) and extracts OSC packets. A message handling script (C_Receiver, shown above left) then processes these and passes the contained values to the various visual processes controlled by other scripts in the scene.

In the image above-right you can see the inspector of a script called C_Object (I know, terrible naming convention). This script is on an object that uses a simple parametric surface shader to produce a specific mesh surface (in this case a screw formula), there are a number of parameters that define the way the surface is constructed within the vertex shader. Variables such as inner radius, wind direction, scale etc. are exposed through the shader material. These values are then updated by the C_Object script, based on incoming OSC messages. The C_Object script can smooth the incoming OSC values to prevent sudden jumps, it can detect large fluctuations (in order to trigger one-shot events) and it can scale the values over time and perform other interesting modulation functions.

SHADERS

Finally here’s an example of the screw vertex shader function.

v2f vert (appdata_base v)
{
v2f o;

return o;
}

This is based on one of Paul Bourkes formulas, Bourkes pages are an amazing resource for anyone interested in mathematical models of 3d forms. In the current set I also use a version of his supershape code and his spherical harmonics work. Credit also has to go to desaxismundi who converted some of these functions for use in vvvv (another dsp environment I have used). More specifically the surfaces shaders take a simple grid mesh and use a series of trigonometric functions to warp it into mathematical 3d forms. (you can see lots on Bourkes pages). I converted the formulas to work with Unity shaders and then I use these to produce the warping 3d forms you see in the performance videos. This is just one of the shaders, I used a different one for each ‘song’, the others include, vertex noise distortion, superformula, simple scaling & fft analysis and procedural noise. In the tracks i have multiple forms using these shaders, each responding to different track parameters and also effected by their own internal modulators (long term lfos etc)

In addition to the forms being effected/generated in realtime I wrote a series of post process shaders that also respond to incoming OSC signals (bleaching out the screen, causing double vision, pixelation etc). The two work in parallel, the postfx relating to overall track information whereas the forms relate to smaller changes. I was quite pleased with the overall effect, and although much of the technique is classic demoscene work I really enjoy revisiting this sort of coding every now and again.

Grab it here, its pretty cheap and will help support us making the rest of the stuff we are working on at big robot.

Its been great reading through the feedback and reviews we’ve had from different versions. One of the most common misconceptions is that player think there is some sort of precomposed soundtrack that is being slowly unlocked or un-muted as you play. Instead the audio is completely dictated by the position of the atoms you detonate on the sequencer grid. Its completely generative and different each time (although within the same set of sounds per level). This means that sometimes the track you make will build a perfect 4/4 groove and on other plays will be sparse, disjointed or even dischordant. But thats part of the game system, a procedural generation of audio, produced in sync with the visual development of the playing field. Anyway heres the slightly hyperbolic trailer ..

And heres the Sir You Are Being Hunted Teaser Trailer

The player is deposited on the edge of a procedurally generated island. The island consists of overgrown pathways ruined spires and broken ramparts. A central tower looms above the island beckoning the player towards it.

The world is generated procedurally each time the game is run, after the initial cycle is completed the player can access the generation parameters to play with. Although its not directly connected to Sir You are Being Hunted it shares some of the world generation techniques and might give some people a taster of the sort of tech we are messing with at Big Robot (Although ‘Sir…’ represents a much more detailed and advanced generation engine).

The audio is a generative mix of material by Jonny Pilcher. The text extracts are from a 1st century BC didactic poem “On The Nature of Things” by the Roman poet and philosopher Lucretius. The project was made in Unity. You can grab the executables below. You can find more information about the development process in some previous posts here , here and here. It’s reasonably bug free, and runs on a range of machines, if you have any questions or problems you can contact me on twitter @tomnullpointer.

In-Ruins-PC.zip Windows 105mb
In-Ruins-Mac.zip Mac OSX 105mb

Some Screenshots:

The second event was a talk I gave at GaME12 on the same bill as David Braben and Peter Molyneux, which was pretty cool. No video for that one, but I’ve uploaded the slides as a pdf here. Its a similar sort of vibe, but with a bit more of a philosophical slant, less code and more aesthetics.

I really should have posted something about this sooner, especially since I’ve been writing posts about it over on Big Robot for a while now. Sir, You Are Being Hunted, is the next game we are producing at Big Robot, it follows our Channel4 project Fallen City and the conversion of my abstract experimental audiovisual game AvSeq which is due to arrive on steam in teh next month or so.

Sir, You Are Being Hunted is a ‘tweedpunk’ Stalker-like exploration and combat game, set in a procedurally generated English Country landscape. It came about during our experiments with the infinite Lodestone engine. The technology were were developing for Lodestone was fantastic, but we wanted to experiment with how combat and other fps/rpg elements might work in a procedural world. The conceptual framework that James and Jim thought up was the dreamlike setting of Sir, a very British sort of sci-fi drawing from influences in 70s films and classic Dr Who episodes.

The main bulk of work so far has been in the creation of the world engine and its production of both the general aesthetic and the implementation of key game objective via the generation process. Recently we have had Dan Puzey come on board to help out with the AI work (Dans code makes me look like im still working in Sinclair BASIC). Things are progressing steadily, but we still have a long way to go and a lot of balancing to do.

Working on Sir, is one of the reasons this blog has become quieter over the last few months, I’ve written quite an in depth article on some of the tech here and I will be writing more on the Big Robot site as the project progresses. This doesn’t mean I will be in an isolation tank coding for the next year, I recently gave a talk on procedurality and infinity at Imperial in London As part of GAME12. I’ve also been nosing around at Rezzed and at the Valve Greenlight launch. Ill try and post a pdf of the Imperial talk soon and Im on the Big Robot Forums if anyone wants to ask about how Sir is progressing. We’ve already had a lot of interest in the project and we are all really excited to see how it will evolve.

Anyway, moving on. Ive been really into structural and formal artwork for years, particularly early computer art and minimalist works by artists such as Sol LeWitt, and in an attempt to decorate my rather spartan walls I decided to generate a few systematic prints using code and pixels. I used unity and jsut made a few simple iterative scritps that write into a texture and then save the resulting texture as a png/gif.

The first peice is a recreation of a computer art work based on all the possible moore neighbourhood combinations.

The next is all the linear steps of a 0-255 sequence in binary

Then Ive tried a few variations on the nine-square quilting pattern, using every combination of 4 tile types.
The following tests are at different seperations for each ninetile group.

These images were originally produced pixel accurate as really small gifs, I had to scale them up carefully (without any bilinear filtering etc) so they could get large enough to print but without losing the sharp definitions of the individual pixel forms. On the whole they came out pretty well, I might post an image of the final framed versions when i get a decent shot.

Procedural / Generative / Modular – grow yourself a game (Unity)

When programmer art sucks and real artists do everything with ten million polys, how can a lone coder get anything done! In this talk
Tom will show how a procedural & generative methods can help drive content production and game design in engines like Unity. Tom will show a number of his own projects and prototypes where modular and generative design have helped overcome the problems of content production. He will also probably go hideously off topic and end up talking about Dark Souls, Honey and Deleuze.

Tom Betts, aka Nullpointer, is an artist, academic, coder and gamer. He’s been a lecturer, designer, published musician, professional
artist, warlock…. but right about now he is working on his Phd about the sublime in digital games and he’s also head coder at Big Robot making a game for Channel4. Hes exhibited digital artworks and performed at international venues such as Sonar, ZKM, Lovebytes, FACT and has done professional design/coding for Tate, the V&A and the Southbank Centre. He likes coding indie projects, drinking coffee, watching films where nothing much happens and is trying to grow a beard.

In many ways Dark Souls reminds me of another of my favourite games, Monster Hunter, which Ive written about previously. In both these games, death can come quickly if you aren’t paying attention, and the mastery of your chosen skillset is essential if you want to progress. Dark souls provides a reasonably wide range of different offensive options, from tanking approaches to ranged weapons and magic. All of these choices expect the player to learn the skills and potential combos of their chosen path.

As with Monster Hunter, different weapons have different movesets and can also provide opportunities to parry, counter, stagger and inflict status effects. Picking your weapon,armor and style is part of what gives players in both games a real sense of ownership and agency. In parallel to Monster Hunter your chosen skill path is important, but often less vital than the ability to read and exploit the behaviour and attack patterns of the enemies. Even when highly levelled, it is still easy to die if you become careless, or try for that ‘one extra hit’. The resulting sense of danger (specifically in areas like blightown or the tomb of giants) generates a tense but rewarding experience. Dark Souls will kill you if it can, capitalising on any mistakes you make. But when you beat it the feeling of achievement is much greater than that in many easier mainstream games.

Another aspect of Dark Souls that makes it a refreshing change to the formats of many other RPGs is the hazy, dreamlike quality of its narrative. The lore of the world is never imposed through wordy cutscenes or in world texts, rather it is implied through the level designs themselves which are punctuated by lost and wandering NPCs. In a similar narrative style to the Silent Hill series these NPCs deliver fragmented and surreal backstories, of both their own journeys and the lore of the world. Just as in Silent Hill, the inhabitants of Dark Souls mirror the alienating and unknowable nature of their world, hinting at guidance but never quite trustworthy. Like Silent Hill, they reappear at different stages of the game, sometimes transformed or suddenly malevolent. Their personal arcane missions mirror the strange futility of your own quest and their abstract awe and fear of the world supports the general atmosphere of hidden purposes and uncertain goals.

Dark Souls never prioritises “the story” over “the game”, instead it treats the story as the world, as something that pre-exists and is there to be ‘read’ through exploration and sparse NPC driven exposition. The world is also a silent one, populated by mute enemies who forgo the battle shrieks of many games in favour of the unsettling clink of bones or chainmail behind you before a lethal stab (The quiet emptiness of Dark Souls castles and catacombs reminds me of Vagrant Story and its lonely streets). The rules of the game world even allow player actions that could seriously hamper narrative progression. Unlike Bethesdas work you can kill the NPCs, they dont just fall conveniently unconscious. After accidentally hitting a blacksmith, the affronted guy would attack me on sight whenever i ventured near his forge. You can pay for absolution to remove such sins, but the NPC who provides this is equally cryptic and obscure.

The ending cutscene (linking the fire), is possibly the shortest one in the whole game, finishing your playthrough with little resolution to the narrative and dumping you without ceremony back at the beginning of the game. This reflects Dark Souls attitude that the world and the process of playing within it is the story. Like STALKER, there is a powerful sense of autonomy in the game world, and like GSCs scarred wastelands there is also a real sense of loss in leaving this world. Because unlike most other linear script-driven cutscene games, the story isn’t there just for you “the player”, it feels like it is there despite of you, and will continue to be there long after you have turned the power off and walked away.