An Envelope of Baggage

This project was undertaken during an internship at HARC with Principle Investigator Alex Warth and his group. The goal was to realise some of the ideas I'd been thinking about that were influenced by the principles of the lab (such as directness, liveness, short feedback loops) to the field of music tools.

Sometimes interfaces stick around for so long that they stop being thought of as the interface to a thing and start being thought of as the thing itself. The envelope interfaces of synthesisers are a prime example of this.

ADSR (Attack-Decay-Sustain-Release) is the default paradigm for interacting with the amplitude envelope of a synthesiser (or some subset e.g. just attack and release).

However, this is not what the envelope really is, really it is the amplitude of the signal over time. The difference being that the ADSR model implies the envelope is always made up of these four segments, with a single peak, while the more basic concept of the envelope makes no such assertions; the envelope could have multiple peaks, with different curves between them.

Within the ADSR paradigm there are a number of different interfaces used to control the four parameters. With hardware devices, this is usually four sliders or four pots.

One of the points of confusion with these interfaces is that the four identical controls do different things, namely that while the Attack, Decay and Release controls set the duration of these segments, the Sustain control sets the amplitude of the sustain segment. This could be made less confusing by grouping the duration controls away from the level control.

A bigger problem is that these controls are still quite abstract, they give us a sense of the number value of each control, but not their values in proportion to each other, and more importantly require a bit of thinking to understand the values as controlling something over time.

We can forgive older pieces of hardware for these interfaces, there are many factors (such as cost and availability) that will have dictated the type of controls that could be used. For software interfaces, we should make no such allowances.

A much better way to understand a value changing over time is to plot it.

The plot combined with direct manipulation interface controls for a much better envelope interface than pots and sliders, changing values gives you a clear view of the effect of your adjustments.

Logic Pro X's "Retro Synth"

(Ironically, in Logic Pro X one of the few synths that has a plot style interface is the “Retro Synth”

To be completely fair, it's fairly clear that in general Logic's older software synths have the 'four controls' interface, while the newer ones have the plot style, although the newest (Alchemy) bucks the trend with a major regression.

While checking out all of these interfaces I was surprised to find that the nicest envelope interface actually lived inside the Ultrabeat drum machine, which has Bezier handles for each node!)

To be completely fair, it's fairly clear that in general Logic's older software synths have the 'four controls' interface, while the newer ones have the plot style, although the newest (Alchemy) bucks the trend with a major regression.

While these are better interfaces, they still have a limited view of what an envelope is, i.e. the level goes up, then down, then down some more. With programmatic tools like SuperCollider, Extempore, the Web Audio API, PureData, Max/MSP, Reaktor, Chuck, Csound etc we can define arbitrary envelopes that do whatever we want.

These tools don’t enforce the ADSR model on your envelopes, but on the other hand are not exactly ideal interfaces, writing lists of numbers or connecting boxes with wires is an even less humane way of creating an envelope.

The job of an interface is to facilitate effective use of a thing by the operator of that thing. With envelopes the job of the interface is threefold:

1. to allow the musician to realise or approximate an envelope they have in their “mind’s ear”
2. to make adjustments to an envelope
3. a means of exploration of new envelopes

ADSR type interfaces are good for making adjustments and reasonably good for exploration (at least within the ADSR paradigm) and reasonably good for exploration, but not very good for realising an existing idea.

How can we do better?

If we think outside the world synthesisers, and imagine how we might communicate to a violinist the envelope we want them to play a note with, the quickest, easiest, most expressive, most humane way of doing it would be to just sing the envelope.

We train our brain-to-mouth interface from birth, and become truly fluent in it, some people train it to an extreme and do incredible things with it: see Beardyman

We train our brain-to-mouth interface from birth, and become truly fluent in it, some people train it to an extreme and do incredible things with it: see Beardyman

For the purpose of realising an existing idea, this method of envelope creation is much quicker, far more musical and more direct. Directness is a big topic in interface design, usually under the term direct manipulation. The input section of this interface is direct, but isn’t direct manipulation, so much as direct creation or realisation.

The advantages don’t stop at being faster and more direct, this interface makes available the true nature of an envelope: a level over time; it allows for envelopes with any number of segments and peaks.

This idea can be taken further, why sing when you can beatbox?

This interface also creates a new paradigm for envelope exploration, where any audio signal can be the source, and envelopes of existing sounds can be easily copied.

This isn’t to say that screen based interfaces aren’t useful, but they have their place. They are great for making small adjustments, copy/pasting, analysing, sharing and exploring. The second section of this interface lets the envelope be reduced to a number of points, which may be just three or four, or a many more, allowing for detailed editing of the envelope.

By breaking the envelope interface into different parts, we can have interfaces that are best suited to the different roles an interface needs to play.

Combined video

All of the demos collected as a single video:

Further work

This is part of an ongoing project, if you find these ideas interesting you may enjoy those. The goal is to re-think many of the ways in which audio software is designed.

Get in touch

Are you a musician / pro audio company / producer / software developer / DJ / interaction designer / other? Do you find this work interesting? Have you own ideas that you want help realising? If this work interests you, or gives you strong feelings (for or against…) get in touch: arthur@arthurcarabott.com