So far, we have looked at the differences between motion,
action, and gesture.
But this concept of gesture is quite tricky to understand.
So now we're going to visit a colleague of mine
and hear how he is thinking about this concept of gesture.
Come.
Hello, hello.
Hi.
The baseline of all this is that music is a multimodal art.
It involves both the sense of motion and the sense of sound.
And, of course, it can involve also a sense of vision,
even maybe the sense of smell.
It's a multimodal art.
And for that reason, it's better to try to be precise
when we speak about it.
When we speak about music and body motion,
we can say we have body motion to make sound.
We have body motion to modify sound.
We have body motion that somehow complements
the sound, like in dance choreography,
and so on and so forth.
So it's clear that at the end of the day,
the essential point is precisely to understand
how music is composed of both body motion-- I
prefer that term-- and sound.
We can now say we have a basic understanding of music
as a phenomenon, well-established
both from our own research and from neuroscience.
that there is a very strong hardwired coupling,
as they say in neuroscience, between what we hear
and our sense of body motion.
And as we know from our research observing musicians
and dancers, most people tend to spontaneously
associate musical sound with some kind of body motion.
Another topic that is quite important in your research,
I know, is that of coarticulation.
It's a difficult word, but can you
try to briefly explain how you're thinking
about coarticulation in music?
Oh, yes.
Yes, yes, with pleasure.
Coarticulation means that in human motion,
and by the way, also in robotics, and also
in animation, you have the fact that body parts are constantly
on the move.
So the easiest explanation is to look at your mouth
when you are speaking.
Whenever you're pronouncing a word,
you can see the shape of the mouth, the lips,
and the tongue.
You follow the motion of the tongue.
And you'll discover that when you are saying something,
you're also preparing the next sound
that you are going to make.
And also, when you're saying something, you are, in a sense,
conditioned by what you just did.
In music, if you are going to play the piano
and hit a key way up on the keyboard,
you necessarily have to move your hand in order to hit it,
because you don't have a finger that's that long.
So this means that you always are in a context of motion.
And this is quite determining for how
music is shaped, both vocal music and instrumental music.
So if you look at the score, what
is called Western common music notations,
you have dots, C-sharp, F-sharp, G, and so on,
which are discrete events.
But when it comes to performance,
the body has to somehow move between the keys.
Or in vocal performance, you have to move from one pitch
to another.
So essentially, you always have a smearing.
And that's the word I use, meaning that you don't have
clean-cut different events.
But they tend to go into a continuous stream of sound,
exactly like in language.
And that, by the way, is one of the reasons why
it's difficult to learn foreign languages,
because speech is continuous.
In other words, speech is coarticulated, as they say.
So you have to be able to pick up the discrete events
from a continuous stream.
And in your research, I know you have been working theoretically
on this.
But also, I know that you're working
in the lab with experiments on these things.
Can you just tell a little bit about how
you're actually doing this and what you're doing in the lab?
Yes, we try to figure out exactly what musicians
are doing.
So far, we have, at least for my research,
I've focused mostly on what we call sound-producing body
motions in music.
So what we do is that we use this so-called motion capture
technology, which is essentially an infrared camera
system with markers.
And then we place markers on the fingers, hands, arms,
shoulders, the whole torso, head to feet, and so on,
depending upon what we are interested in studying.
So in the example of a piano performance,
it looks like almost like a person
having smallpox with all these markers on their hands.
And then we have very detailed information
about how this preparatory of motion
is going on all the time.
The next project will be drumming, a drum set.
Because, as you know, you have the tom-toms,
and you have the right, and hi-hat,
the bass drum kicks, snare, whatever.
And whatever rhythmical pattern the drummer is playing
needs to have this constant motion.
Because you hit the ride, and then you are hitting the snare.
And the moment you hit the ride, the stick passes off,
and you try to aim as best you can for the next event.
So in that sense, you always have this context.
And it also is mobilising the rest of the body, the torso.
The drummer would sit down on a stool like this
and try as best as he or she can to exploit the rebound.
A drummer has to conserve energy.
Otherwise, he or she would be completely exhausted
after a couple of minutes.
So again, returning to your main question,
you have all this embodiment, how
the body is an integral part of the music-making and the body
motion, of course.
And then somehow, the body has to adapt to the constraints,
as we say, of the physics of the musical instruments.
Well, thank you very much.
You're welcome.