Interactive Measurement Visualization for Windows Media Player

Dan Boye

Department of Physics

Davidson College

With the support of the ACS Fellows Program, I would like to develop a new, measurement-friendly visualization plugin for Windows Media Player and a series of exercises and audio files. This new curricular material would capitalize on students’ innate interest in computers and music to examine the connections between music, science and technology. Most college students own or have immediate access to computers that have the capability of playing audio media. They are quick to admit that they spend a significant amount of time and network resources acquiring and listening to music. The media files are accessed through software players such as Microsoft Windows Media Player. These free software packages also allow the user to manipulate the audio playback by changing the frequency equalization, the reverberation time and other factors that affect the quality of the music heard. Another facility is to present a visual representation that changes with the mood and beat of the music. Different visual effects may be evoked by changing the current settings or by loading new software visualization plugins. Currently available time and frequency domain visualization plugins for media players, while imaginative, colorful and sometimes artistic, do not lend themselves to analytical thought or scientific measurement.

The new plugin and accompanying exercises would be used primarily in the Musical Technology (Physics 115) course at Davidson College. The main purpose of this course is to take advantage of the popularity of using computers to play music to expose the scientific underpinnings of music and to reveal the fascinating technology of digital music. It is a highly subscribed course covering the physics of sound, digital music basics, electronic synthesis of musical instruments, room acoustics, CD and DVD players, and compressed data formats. Non-science majors populate this course to satisfy an elective or a core requirement in the traditional liberal arts curriculum. Students directly apply material covered in the course to daily experiences and to musical performances they are required to attend. The proposed new material would be used the next time Musical Technology will be taught which is in the spring of 2003. In addition, the material developed through this fellowship would be of use in a wide range of physics and psychology courses that touch on the subjects of sound or sound perception (Phys 120: General Physics, Phys 130: General Physics with Calculus, Phys 330: Intermediate Mechanics, and Psy 101: General Psychology, Psy 276: Cognitive Psychology, and Psy 301: Research Methods – Sensation & Perception).

The plugin will be written in Visual C++ programming language and will contain commands that address simple graphics issues as well as low-level Windows event handling. Synchronized to the music, Windows Media Player will send to the plugin two 2-dimensional arrays containing the amplitude variation of the sound with time and the power variation with frequency for a particular time frame. The plugin will plot one of these arrays and allow the user to measure amplitude and time or power and frequency depending on the choice of arrays. A screenshot of a preliminary version of the new visualization plugin follows this paragraph. It shows the power vs. frequency graph for a 1kHz square wave that has been recorded in .wav audio format. Windows Media Player can interpret over 15 different types of audio formats and so this visualization will not be limited to just working with .wav files. In the lower left corner, there is a readout of a point on the graph chosen by a mouse-click. The amplitude vs. time graph will have similar capabilities.

The exercises and audio files accompanying the plugin will help the users interactively explore different ways of thinking both holistically and analytically about sound and music. An example of such an exercise would have the student measure the frequency and relative powers of the different components of the square wave shown above. Next the fundamental frequency would be ramped and the student asked to described the changing frequency spectrum. Finally, comparison of the square wave frequency ramp with the changing frequency spectrum of the opening bars of Gershwin’s “Rhapsody in Blue” which contains the clarinet solo glissando or “smear” or frequency ramp would be sought.

In order to have the broadest impact with the new plugin and accompanying exercises, a webpage will be created for the free distribution of this material to faculty at Davidson, to ACS colleagues, and to higher educators in general. In addition, the new material will be announced and demonstrated during upcoming talks by this investigator during the Musical Technology Symposium at the national meeting of the Acoustical Society of America and during the “Science is Fun” Symposium at the national meeting of the American Association for the Advancement of Science. Feedback from students in the courses where this material is used and from discussions at the meetings with specialists in the field of musical technology will help in the assessment of the learning material and guide changes and future directions for this work. The fascination of musical technology is not limited to college students; it spreads to the general public. The Microsoft Windows Media Player website (windowsmedia.com) provides a download service for plugins. The new visualization plugin will be submitted for free distribution through this service. A link to the webpage containing the accompanying exercises also will be provided. It is hoped that through these presentations and with ease of availability, science literacy will be increased in a general way.

This ACS Fellowship will provide financial support in the form of a $2500 stipend for this investigator who is on sabbatical leave for the 2001-02 academic year. The work will be accomplished in the period from January through April 2002.