I’ve always been interested in the relationship between gameplay and musical performance. Theres a remarkable structural similarity between certain game systems/mechanics and compositional ones. There is also a risk/reward/challenge aspect that is core to both practices. Anyway, for a short talk I took part in for the Leeds Evolution Festival I wrote a quick augmented chess/draughts app.
The software (Openframeworks) I wrote keeps track of a matrix of grid points that are mapped to the grid cells of a chess board. The application then samples the color values of each point (via my hacked eyetoy cam) and reports back their position and rgb attribute. In this example 2 ‘watcher’ values can be set by simply clicking the image to define a target color. These ‘watcher’ triggers are called whenever the pixel sample point is within a specific deviation of the target value. The pixel sample points move in a traditional sequencer sweep across the board.
Of course 8×8 is perfect for a basic score/drum machine sequencer (Connect 4 is my other challenge, but it has a more awkward time signature). The height of watcher A is mapped to a different sample in a standard drum machine sample bank. The height of watcher B is mapped to the pitch of a short bass-synth sound. Obviously in the video I am just composing with the peices to demonstrate the system. But you can of course play a proper game of draughts and the software sonifies the play-field in a way that you can hear who is winning & losing. There are versions of the setup that use less rhythmic soundsets and create a more choral/textural result. It could even issue specific audio warnings when certain gameplay situations arrive or segue into different compositional style based on win/loss ratios etc. Theres a lot of potential for this system and I’m going to explore it further, perhaps as far as producing a less DIY setup so It can be used in a more robust way.
I originally wrote the app in MAX/MSP/Jitter and then ported to PD/GEM. But although both these systems are great for audio they don’t perform as well with video (not without convoluted patching). Openframeworks seemed the way to go (again!).
Drum machine draughts from Tom Betts on Vimeo.