blockNect is an multi-purpose interactive interface, realized using Microsoft’s Kinect camera and computer vision, developed along with Thibault Lelievre and Philo van Kemenade as the group Computer Vision assignment for Parag Mital as part of Workshops in Creative Coding 2. Colored blocks are placed on an arbitrary sized chess board, where the block’s can be manipulated by the user. Each color block corresponds to a separate entity, while stacked blocks merge together to form single blocks. This affords the user three separate degrees of freedom; x, y-coordinates and height. This was realized as an eight-step sequencer where different colored blocks would relate to different instrumentation (such as drums, keyboard, bass), and the height to volume.
Using OSC, openFrameworks communicates bi-directionally to Max/MSP. Each beat of an adjustable metronome in Max/MSP requests the current board’s arrangement. The current beat sequenced for each instrument is then analyzed, and corresponding MIDI messages are outputted to a Sampler. However, an interface for sample playback is nothing new, and thus to better showcase the possibilities offered by blockNect, a Soundscape sound synthesis engine was also created. This engine consisted of three simple instruments; an additive synthesizer, a wavetable synthesizer, and a subtractive synthesizer. Following from the homogenous mapping explored for the sequencer, for the Soundscape engine, the y-direction corresponds to pitch, the x-direction to unique properties, and the height to volume.
Figure 1: Snapshot of the program showing register blocks in Max/MSP.
Figure 2: Elegant UI for blockNect.