In order to analyse the relationship between musical
sound and body motion, it's important to have some way
of analysing and describing musical sound.
In this video, we will briefly introduce some useful tools
for this purpose.
When talking about music in everyday life,
we use every day words to describe the musical sound--
groovy, heavy, happy, noisy, funky, romantic, and so forth.
Notice that many of the descriptions we use
are somehow related to highly subjective,
non-musical aspects.
For instance, someone may label a certain piece
of music romantic, while others might not associate romance
with the same piece.
Other descriptions are slightly less subjective,
although still not a precise description
of the musical sound.
For example, genre related descriptions like funky,
metaphoric descriptions like punchy,
or descriptions of how we perceive
the sound such as loud or soft.
The descriptions of sound mentioned so far
are verbal descriptions.
They are words or labels that we use
to describe a sound or an entire piece of music.
Sometimes, such descriptions are what researchers
are interested in, since verbal descriptions reveal something
about how people experience the sound,
like their associations, et cetera.
However, there's often a need for more objective descriptions
of the sound signal itself.
In order to study the sound signal more closely,
we shall first have a look at three different ways
of representing the sound signal.
These are the waveform, the spectrum, and the spectrogram,
each of which gives us some visual indication of what
goes on in the sound signal.
Later, we shall have a look at how
we can extract more information from these representations.
The figures that I'm about to show
were made in Sonic Visualizer, which
is a user-friendly and free piece of software
that I would recommend you have a closer look at on your own.
This figure shows a waveform representation
of a sound signal, with time on the horizontal axis from 0
to 7 seconds and amplitude on the vertical axis.
From this representation, it is easy to spot
the duration of the sound and the periods of silence.
It's also possible to get an idea
of the dynamics of the sound, although precise or detailed
information is difficult to read by inspecting the waveform.
Let's listen to the sound
[MUSIC PLAYING]
This figure shows the waveform on top
and the spectrum of a short part of the sound on the bottom.
With frequency on the horizontal axis
and amplitude on the vertical axis,
we get information on the frequency content of the sound.
For a single tone, the fundamental frequency
and the range of overtones are easy to identify.
[MUSIC PLAYING]
A spectrogram representation shows how the frequency content
evolves over time.
The horizontal axis shows time and the vertical axis shows
frequency, from the bass on the bottom
to the treble at the top.
Amplitude is shown by colour.
The figure on the right shows the same saxophone melody,
and how the fundamental frequency and overtones change
over time.
Let's listen to it.
[MUSIC PLAYING]
Now we have three representations
of the sound signal.
Already from these representations,
we can start to say something about the sound.
The amplitude of the sound wave and the frequency content
are both varying over time.
Still, the representations we have looked at
contain large amounts of information,
and sometimes it might be necessary to narrow down
our description even further.
Sound is generally too complex to describe fully
with a single number or graph, but a good description
can be given if you decide to just look
at a single aspect of the sound at a time.
Such an aspect is what we usually call a sound feature,
or sound descriptor.
A descriptor is a numerical description
of some quantifiable aspect of the sound.
Many existing features are quite intuitive,
as they are either based on simple transformation, so
the waveform or the spectrum, or perceptual models.
Other descriptors are more technical,
but have proven useful in certain types of analysis.
Descriptors may either be global,
meaning that they describe an entire sound,
or they are time varying, meaning
that they are calculated from short, sequential, time frames,
and then forming a sequence of numbers.
Using tools like the MRI Toolbox from MATLAB,
it's possible to extract a range of descriptors
from a sound file.
Here, for instance, we see how the sound energy
of the saxophone melody changes over time.
[MUSIC PLAYING]
Other examples of some descriptors are tempo,
often described in beats per minute; pitch,
denoting the fundamental frequency of the sound;
spectral flux, describing how much the spectrum changes;
roughness, using a mathematical model to describe roughness
that occurs because of limitations in our auditory
system; spectral centroid, which denotes the very centre
of the spectrum, or in other words,
the point of the spectrum where there's an equal amount
of energy above and below.
Let's have a look at an example of how the spectral centroid
changes when the brighter parts of the sound are filtered out.
This example shows an analysis of a drumbeat,
with the waveform on top, spectrogram in the middle,
and the spectral centroid at the bottom.
[MUSIC PLAYING]
So to close up this lecture on sound analysis,
let's ask the question what numerical sound descriptors
may be useful for.
The obvious reason is that numerical descriptors tell us
different things about the sound signal than what
we are able to hear ourselves.
They may be used to compare large sets of sound files,
and as such, organise sound databases.
Furthermore, numerical sound and movement descriptors
may provide us with partial answers to big questions
like, how do people associate body movement with sound?
Is there a correlation between sound energy and movement
energy in dancers?
Or between musical tempo and movement tempo?
Or what about pitch, which we describe
as going up or down, how precise do we actually relate it
to physical or vertical movement?
Armed with numerical descriptors,
it's possible to do systematic studies
with statistical analysis and answer such questions.