Live coding of MIDI music
root
- data
- doc
- http
- src
- .gitignore
- Changes.md
- LICENSE
- Makefile
- README.md
- Report.tex
- Setup.lhs
- TODO
- live-sequencer.cabal
- shell.nix
- wxc.nix
Example usage
The live-sequencer does not make music itself, its entire task is to control other software or hardware synthesizers. That is, in order to hear something you need a working MIDI synthesizer such as the sampling based software synthesizer TiMidity. You may run TiMidity and the live-sequencer this way:
timidity -iA &
live-sequencer-gui --connect-to TiMidity Simplesong &
This should give you an ongoing stream of notes. Then change one of the numbers that appear in the lines like qn = 300
and press CTRL-R for "reloading" that module into the interpreter. This should immediately have an effect, namely increasing the tempo of the melody. You may also alter a note name like c 4
to cis 4
, then reload, then undo the modification and reload, again, after a while. This is the main idea of changing the song while it is playing. The way the changes are applied warrants that the change takes effect when the time comes. Music is not interrupted and does not need to be restarted for reacting to changes.
The overall task performed by the sequencer is to lazily evaluate a term called main
that is a list of events. The value of main
is a stream of midi events (On/Off pitch velocity
, PgmChange
, Controller
) or (Wait milliseconds
). You may wrap a MIDI event in a Channel
constructor in order to assign the event to the particular MIDI channel. If you omit this constructor then the event is put to channel 0.
In each step, the head of the main
stream gets reduced to head normal form (with :
at the top), and the first arg of the :
gets fully expanded and it must be a MIDI event.
Input language
The used language is syntactically almost a subset of Haskell with only strict pattern matching and pattern matching only at the definition level (no case), no local bindings (no lambda, let, where), no types (no type inference, type signatures and type declarations are skipped), and with diet syntax (i.e. drastically reduced syntactic sugar, like no layout rule, no do syntax, no list comprehension, no operator sections).
Semantics is similar to lazy evaluation, but we have no sharing. The design goal is that code can be changed while the program is running. This implies that evaluation of one expression may give different results at different times (e.g., during a live performance, one changes some chords of a musical theme). In turn, this implies that we do not store and share results of evaluations, hence, we don't have local bindings.
You may import and use the special functions 'Controller.checkBox', 'Controller.slider' from the "Controller" module. For every call to these functions a widget is added to the control window and the state of the widget is the result of the function call. Technically every change of these widgets internally adds or updates a rule in the "Controller" module. The effect is very similar to updating a value definition in a module and then reloading that module to the interpreter, but using the widgets is more intuitive.
Offline rendering
In the library interface of this package we provide the basic Live-Sequencer modules in order to allow offline rendering of music that you programmed within the Live-Sequencer. You may generate a standard MIDI file using functions from the "Render" module. To this end load your song module into GHCi and call
YourModule> Render.writeStream "yoursong.mid" yourSong
HTTP access
You may open a browser and view all modules under http://localhost:8080/. If the user of the GUI inserts comments like this one:
----------------
, then it is possible to modify the content below this mark via HTTP. This way multiple people can participate in the composition process. The recommended situation is a room with a data projector and a loudspeaker, where the conductor explains the functions to the auditory and the participants can watch the screen and listen to the music.
You may choose any other port using the command line option --http-port
. If you want to use a system port like the standard HTTP port 80, we recommend to configure a firewall to redirect the external port 80 to the internal user port. We discourage from starting the live-sequencer as root user. You may disable the HTTP server altogether by compiling with cabal install -f-httpServer
.
Execution modes
There are three modes of execution that you can choose from the Execution
menu:
Real-time: This is the mode for musical live performances. The interpreter waits according to the
Wait
elements in the main list.Slow motion: This mode is for demonstration and debugging. You can alter the speed using
CTRL-<
andCTRL->
.Single step: This mode is for demonstration, debugging and as a pause mode, when the interpreter reaches the end of the main list. You can trigger evaluation of the next element using
CTRL-N
. You can perform a single reduction withCTRL-U
, which also highlights the rule that will be applied next. Changes to the program are only respected when an element is completely reduced and sent via MIDI. Unfortunately it is currently not possible to undo a step.
Editing
You can change a module name by altering the module identifier between the module
and where
keywords and then triggering module reload. The same way you can load new modules by adding import lines and reloading the module. Alternatively, you may create new modules or close old ones using functions from the File
menu.
For composition it is useful to play parts of the music. You can do this by simply placing the cursor within an identifier or by marking an expression and then call Play term
from the Execution
menu. This will make the marked expression the current term and start playing.
Once the music is playing you can change it by altering the module and reload it. However you may find out that you cannot do a certain modification this way. In this case you can mark an expression that denotes a stream transformation function and call the Apply term
menu item. This will apply the marked function to the current term. Useful functions are:
merge newTrack
for adding a new track simultaneously. However, mind the latency!flip append newTrack
for appending some events to the current music.dropTime time
for skipping a part of the music. However this may skip someOff
events and this yields hanging tones. Additionally you may exceed the number of maximally allowed reductions.skipTime time
for skipping a part of the music. This one only removes or shortensWait
constructors. Thus all events are played but you risk exceeding the limit for playing many events at once.compressTime acceleration time
for accelerating the music for a certain time. This should circumvent the problems ofdropTime
andskipTime
.
Limits
Without some safety belts it would be very easy to consume all memory or all processing power by accident or by people who contribute malicious code via HTTP. Thus we have added some limits. These have reasonable default values but you can adjust them to your needs via command line options at startup. These are the limits you can set:
maximum number of reduction steps per list element: With this limit you can prevent infinite loops.
term size: With this limit you can prevent memory leaks. You may also hit the limit if you write a whole song in a big list. Better split the list up into sections and define a function for each section.
term depth: With this limit you can prevent unbalanced expression trees. Unbalanced trees do not consume more memory than balanced ones, but they consume considerably more graphical space on pretty-printing.
maximum number of events per time period: If your song is too fast or does not contain any
Wait
elements at all, your machine will run out of processing power. Thus you can restrict the number of events generated in a certain period of time. It is controlled by two options:--event-period
sets the time period in milliseconds whereas--max-events-per-period
sets the maximum number of events within this time period. In principle you can consider this a ratio but you cannot simply cancel it. E.g. both--event-period=100 --max-events-per-period=15
and--event-period=1000 --max-events-per-period=150
describe the same ratio, the difference is how liberal is the sequencer with respect to exceeding the ratio for a short time. Read the first setting as: "For 15 adjacent events, the duration between the first and the last one must be at least 100ms." That is, if you emit 20 events simultaneously every second, then the first setting will forbid this, and the second setting will allow it. Thus we recommend to first set--max-events-per-period
to the number of events that you want to emit simultaneously and then set--event-period
large enough to match the power of your machine.split wait: The interpreter cannot be interrupted if it waits in reaction to a
Wait
command. The way we implemented the interpreter, an overly longWait
duration will still allow to stop playing music immediately. However, if you restart the music you may still meet the interpreter while waiting for the end of the wait duration. To prevent this situation the interpreter will split aWait
if its duration is too long. The--split-wait
option allows you to adjust the maximum allowedWait
duration.
ALSA
Using the --new-out-port
option you may add more ALSA MIDI ports. Every port extends the range of MIDI channels by 16 new logical channels. That is Channel 40 ev
sends an event to MIDI channel 8 at the second newly added ALSA port (because 40 = 2*16+8). Every --connect-to
option refers to the latest added port. Example:
live-sequencer --connect-to Synth0 --new-out-port out1 --connect-to Synth1 --new-out-port out2 --connect-to Synth2
You do not need to connect to any synthesizer at startup. You may connect or disconnect the live-sequencer to any synthesizer once it is running using aconnect
(command line) or kaconnect
, alsa-patch-bay
, patchage
(graphical interfaces).
The live-sequencer itself can be controlled to some extent. You may start the live-sequencer this way
live-sequencer --connect-from YourMidiController
or connect to it once it is running. This enables the following functions:
If you press a key on your MIDI keyboard named YourMidiController, then the according note name is inserted in the current module. However, note durations cannot be preserved and velocities are ignored, as well. Thus don't expect that the live-sequencer captures complex songs, this function is just intended as assistance for note input.
You can control execution of the live-sequencer using MIDI Machine Control SysEx messages. Some MIDI controller keyboards have transportation buttons that support those messages.
The supported MMC commands are:
RECORD STROBE: Toggle between receiving and ignoring note input from MIDI keyboard
PLAY: Restart the interpreter
STOP: Halt the interpreter and turn sound off
PAUSE: Toggle between real time and single step mode
FAST FORWARD: Next element in single step mode
Tips & Tricks
Append with overlap
The (+:+)
operator can only handle precise concatenation of event streams. However, in common music you also have to handle upbeats and legato. Technically, this means that there may be events before and after a core part. First, let us consider the case where there are events after the core part. We define a Block
data structure containing two or more parallel Track
s. For simplicity we now use only two tracks:
type Track = [Event Message] ;
data Block = Block Track Track ;
The first track specifies the length of the block. It cannot overlap. If you have no other use for it, fill it with rests. The second track can be shorter or longer than the first track. Everything longer will be merged with subsequent Block
s. This is, how we convert a sequence of overlapping blocks to a plain Event
stream:
consBlock :: Block -> [Event Message] -> [Event Message] ;
consBlock (Block t0 t1) y = t0 +:+ y =:= t1 ;
concatBlocks :: [Block] -> [Event Message] ;
concatBlocks = foldr consBlock [] ;
Blocks with events before a core part must be handled by delaying every block by the maximum time that an event can occur before the core.
Tempo changes
Changing the tempo within a song is a bit tricky, especially in the presence of overlapping blocks (see above). You want
Consistency. Events that would occur at the same point in time without tempo changes shall occur at the same point in time in the presence of tempo changes, too. This rules out fixing the time periods in an event stream and change them later, but before merging it with other streams.
Composability. You want to play only parts of the song or concatenate multiple parts of a song. This rules out external description of tempo progression.
The best I could come up with is to extend the Midi.Message
data type by a SetTempo
constructor. After you have comletely composed the song you scan the event stream for these constructors and change the Wait
events accordingly. The LiveSequencer language is untyped, thus you could simply use a SetTempo
constructor as if it were a constructor of Midi.Message
. Though, for Haskell compatibility I suggest you wrap Midi.Message
in a custom datatype that adds the SetTempo
constructor.
The MIDI file format and ALSA sequencer even support a SetTempo
statement natively. However, we cannot easily make use of it, since it is not obvious how to merge streams containing SetTempo
in the general case.