Concept: To build a matrix sequencer which data input is generated by a 3d model / or several 3d models. Find out good ways to map these parameters to virtual instruments in a musical matter.

Software and platform: Mac OSX and Max/MSP 5

The program

The instruments: First i started by making 3 simple instruments which would be easy to use in a 1-bar matrix sequencer. The basics of a drumkit, a kick drum, snare drum and a hihat. The kick drum is 2 envelope generators. One for controlling a decaying amplitude and one for controlling a decaying pitch. The snare drum is built on the kick drum concept, but also with a noise generator on top. The hihat is a amplitude envelope generator and a noise generator with a highpass-filter which has a bit of resonance and a fast increasing frequency. Some of these parameters will be controllable from the matrix sequencer. All instruments are in one sound channel only.

Suggestions for improvement: The program could be expanded to contain more instruments and could also be expanded to include external vst instruments.

The sequencer: The sequencer itself is a 16x16 matrix from a jitter object with 6 planes. That means 6 parameters pr X-Y position. I made it so the X-axis represent the time when events should be triggered and the Y-axis represents the different instruments being played. The matrix gets its data from the graphic objects elsewhere in the patch. Everytime the sequencer patcher recieves a number from 0 to 15 representing the X position it dumps out data for all the 16 Y positions with values for the 6 different planes in the matrix. The timing of this is coming from a metronome and counter connected to the global transport system in MAX 5. This data is then routed from a object called "ruter" to the different instruments from a selected channel (Y-position). The instruments triggers a note when the value in the first plane is above a selected threshold value.

Suggestions for improvement:  The sequencer could be a larger customizable matrix, for example 32x32 or 64x32 and so on. The matrix could also contain more planes in order to store more parameters. Now the values stored in the matrix limits to the values of red, blue and green in the rendered 3d model, but several different 3d models could be rendered on different planes in the matrix. These could also be shown in different graphic windows.

The 3d model and rendering: The 3d model uses the jit.gl.grishape and the jit.gl.plur objects from jitter. You can input user data to create different looking 3d models based on the objects available in jit.gl.gridshape. This is then rendered to an openGL window and then read from that window into a matrix. The matrix is then fed directly into the sequencer matrix.

Suggestions for improvement: There could be several models projected on top of each other, on different planes. each with its individual parameters controlling parameters in the sequencer and instruments which could be user defined. It could also be possible to move camera positions, ambient lightning and other textural changes on the 3d model itself. A controller could also be connected to adjust the rotation and x-y-z position of the 3d models. Another thing could be to apply anti-aliasing to 320x200 matrix before it was downscaled to 16x16, for smoother changes in parameters if that is desirable.

Knut Petter Saevik , University of Oslo 2008