By Miles Thorogood
The SpokenWeb project relies on concrete approaches to audio analysis for uncovering the patterns embedded in archival spoken word recordings. The 2019 SpokenWeb Sound Institute at Simon Fraser University highlighted the strategies that researchers are following and opportunities for research and creation projects that can use the tools from these endeavours. Specifically, two presentations demonstrate how the embedded knowledge of literary arts and state-of-the-art computational tools could combine to bring new methodologies of working with spoken web archives.
Jonathan Dick’s presentation of his MA research titled “The Mold That’s Branded On My Soul’: A Computational Approach to Racialized Voice in Jean Toomer’s ‘Kabni” was an excellent example of how close listening and manual analysis of recordings can be used to illustrate particular cultural and social motivations of the performed word. Jonathan’s process involved running audio recordings through the open-source software Sonic Visualizer to tag the interval distances between the parts of speech. He then used this information to call attention to the stresses in the voice and the concepts of the text and environment they were spoken within. Although Jonathan spent much time in annotating these interval tags into the visual representation of the sound, his argument regarding stresses was clear and informative.
Audio Signal Task Force Workshop led by Tanya Clement presented another approach to audio analysis. In this case, Tanya gave an overview of the Drift software for automatic pitch tracking and visualization that aligns the text of a poem with a poet’s intonation patterns in a given poetry recording. In the workshop, we had the opportunity to run the program with text and recordings from the spoken web archive. The software did a great job of aligning the words in a text with those in the recording and Drift came across as a useful tool in our research-creation projects.
Given these presentations, some initial ideas came to mind of how specific cultural media as presented by Jonathan Dick interacts with computational tools as from Tanya Clement. One such notion includes using Drift for pitch following for sonification of meaning and feeling. Does the poem express similar meaning and feeling when the words are removed, leaving only the abstract movement of pitch curves? Another related idea is to create a dictionary of stresses and intervals that map to particular cultural concepts that could be used to automate the exploration of the big data archive of the SpokenWeb.
Altogether, the SpokenWeb Sound Institute was an inspiration for moving forward with the research-creation as UBC Okanagan’s contribution to the overall project. Coming together as a larger group, the interactions of partners and friends created an atmosphere of positive momentum.
Miles Thorogood is an Assistant Professor of Digital Media theory and Computer Science; he is also a collaborator on the SSHRC-funded SpokenWeb Project)