A Musical Experiment

I made a previous post that showed a friend and professional “speaking” a Kennedy speech with American Sign Language (ASL). That was both fun and intriguing. It showed how an experienced professional is both adept and consistent in their skill. The analysis used the Dartfish (www.dartfish.com) software to take videos of two separate sessions of the same professional skill and overlay one on the other. This is the most visual way I know to show the exact form and texture of the variations that can occur between repetitive actions. It complements earlier examples (here, here and here) that show how standard video can be used to see variations in machine operation.

With that in mind, I wanted to try something a bit smaller scale and precise.  I wondered about the consistency and precision of a professional musician when he is performing a familiar piece. Bryan Crumpler (http://www.whosthatguy.com/) is a friend and professional clarinetist. He kindly agreed to be the subject of my latest mad scientist experiment. I am a musical ignoramus, so I could only trust in Bryan’s expertise, forbearance and good humor.

We video recorded Bryan performing a short (1 minute) excerpt from the exposition of the Mozart Clarinet Concerto. He repeated it 4 times, twice each with and without using a metronome.

Without a Metronome

Bryan played a 54 sec introduction to the Mozart Concerto. He played it twice and we did a blend in Dartfish to see how much variation there was in his fingering between the two sessions. In this example, he played without any external pacing or “context”. He starts with similar body position and exactly in sync, but the later parts lose sync as he takes pauses of different duration between the musical phrases.

I played with the video a bit to sync it at various points. He was very much in sync within each phrase or passage. The differences occurred in the pauses between passages.

With Metronome

Bryan played the same introduction to the Mozart Concerto. Again, he played it twice and we did a blend in Dartfish to see how much variation there was in his fingering. This time, Bryan used a metronome to give strict pacing or “context” to the piece. He started in slightly different body positions in the two sessions, but his fingering remains remarkably consistent throughout the piece.

Bryan explained to me that this is why a conductor is so important in an orchestra and why accompaniment – for the sake of “context” – is critical in the early stages of competitive auditions. A great conductor can simultaneously keep everyone anchored to the same pacing … and the conductor can systematically vary the pacing to create the dramatic impression that he or she is seeking.

Bryan is fairly passionate that absolute musical precision is too predictable and, ultimately, boring. That is what a computer does. It follows the plan with total precision … every time. A skilled musician is highly precise, but has the ability to adjust the pacing and phrasing to create the musical impression that he or she is seeking.

I never knew that. I seem to learn something new every time I take Dartfish out for a test drive.

Business Implications

It will take experts in music to know whether this type of video analysis is ever likely to be useful in routine teaching or perfecting their musical skills. I do believe, however, that video analysis can be widely used to explore actions that are difficult to see and comprehend with normal human senses. The following diagram illustrates two situations (from a long list of situations) where the human eye has difficulty seeing all of the crucial actions and/or the human brain has difficulty remembering what it has seen.



In the case of task progression, it is not that hard to remember a static image (or at least your impression of a static image). It is pretty much impossible to remember the details of all of the actions that transpired in a dynamic, moving activity. The challenge is even greater if the activity unfolds either slowly or quickly. These are the edge conditions where video analysis has the greatest value. I am increasingly convinced that industry needs to add robust video analysis tools to its everyday toolkit for forensic, investigative, and diagnostic analysis. Otherwise, there are too many important actions and behaviors that simply cannot be seen or studied in the depth they deserve. With tools like Dartfish, the process is now sufficiently easy that there really is no excuse.

Can you help?

I’m not an employee, investor or reseller for Dartfish, but they are kindly letting me play with it to explore any non-sports uses I can cook up. I am willing to play with videos sent to me by others as long as they don’t hold me to a deliverable timeline and they give me permission to post useful pieces on the blog.

Leave a Comment

Skip to toolbar