8 research outputs found
Visually-Led Design for Gestural Audiovisual Instruments
No full textIn this paper we present our visually-led design method for creating gestural mappings in a new audiovisual percussion work titled Cymbalism. Unlike most audiovisual works, Cymbalism was inspired by the creation of a series of interactive visual scenes that respond to the performer’s real-time movements. In leading with the visual interaction, we discuss how this approach fostered a union between the physical, audio and visual elements of the work, creating a performance where the visualisation is not simply a feedback mechanism but fundamental in inspiring compositional concepts and new ways of interacting with sound. Through practice-based research, we use the insights gained through creative development and performance outcomes to guide the continued evolution of an established wearable gestural DMI.</p
Mixer Metaphors:audio interfaces for non-musical applications
No full textThe NIME conference traditionally focuses on interfaces for music and musical expression. In this paper we reverse this tradition to ask, can interfaces developed for music be successfully appropriated to non-musical applications? To help answer this question we designed and developed a new device, which uses interface metaphors borrowed from analogue synthesisers and audio mixing to physically control the intangible aspects of a Large Language Model. We compared two versions of the device, with and without the audio-inspired augmentations, with a group of artists who used each version over a one week period. Our results show that the use of audio-like controls afforded more immediate, direct and embodied control over the LLM, allowing users to creatively experiment and play with the device overit’s non-mixer counterpart. Our project demonstrates how crosssensory metaphors can support creative thinking and embodied practice when designing new technological interfaces.</p
Vibrato Analysis Dataset, Detection and Parameterization
No full text<p>vibrato_analysis</p>
<p>Vibrato Analysis Dataset,Detection and Parameterization</p>
<p>Introduction</p>
<p>This dataset was created as an outcome of a summer internship with Good-sounds project. Sound recordings were used for vibrato analysis. The dataset contains monophonic recordings of single notes and melodies for four instruments with annotations of four different recording sets regarding some vibrato parameters.</p>
<p>It is organized according to the owner of the recordings. Sounds with vibrato and no vibrato are presented within the folders with their annotations in cvs format. All annotations except than the alto saxophone recordings include derivative analysis of pitch parameters.</p>
<p>One of the recordings for violin was made in the Universitat Pompeu Fabra recording studio by a violist from MTG. Single notes of two octaves and four different melodies starting from different pitch are recorded in wav format sampled at 44.1 kHz. Each sound separated by semitones is recorded four times for no vibrato,slow, standard and fast rates of vibrato. Melodies were played for no vibrato and vibrato at a standard rate to have the attenuation at the end of the note.</p>
<p>One other recording set was taken from Good-sounds project. The only brass instrument within the whole dataset is this one for alto saxophone. It was again recorded in the Universitat Pompeu Fabra / Phonos recording studio. This dataset for alto saxophone not only contains vibrato and non vibrato sounds but also some different tonal characteristics.</p>
<p>Three other recordings are downloaded from a user named Carlos_Vaquero in <a href="https://freesound.org/">Freesound</a>. Violin, violoncello and transverse flute recordings were downloaded and used for non commercial purposes under creative commons license. Within the Carlos_Vaquero folder, audio files are separated according to the type of instruments.</p>
<p>Dataset Description</p>
<p>The database is meant for organizing the sounds in a handy way. It is organized according to the creator. In each three datasets, annotations and analysis parameters are available within the csv files and each has 11 field descriptor.</p>
<ul>
<li>Carlos_Vaquero
<ul>
<li>Transverse_flute</li>
<li>Violin</li>
<li>Violoncello</li>
</ul>
</li>
<li>Good-sounds-Alto sax
<ul>
<li>Alto-sax</li>
</ul>
</li>
<li>MTG - Violin
<ul>
<li>Violin</li>
</ul>
</li>
</ul>
<p>Except than alto-sax recordings, each contains following parameters:</p>
<ul>
<li><strong>Peak_Percentage:</strong> Percent wise proportion of peaks in first derivation of the pitch trajectory.</li>
<li><strong>Mean_Difference:</strong> Mean value of the differences of two consecutive peaks.</li>
<li><strong>Max_Difference:</strong> Maximum separation of side by side peaks.</li>
<li><strong>Index_Max (first one):</strong> Index value of the maximum peak in the derivative array.</li>
<li><strong>Location_Max (%):</strong> Location of the maximum peak in the array, percent wise.</li>
<li><strong>Start_End_Time (seconds):</strong> Starting and ending time instants of vibrato in the recording.</li>
<li><strong>Duration:</strong> Duration of the vibrato part of the recording.</li>
</ul>
<p>Documentation</p>
<p>Related report can be found in the GitHub repository as "Vibrato Analysis Internship Report".</p>
<p>License</p>
<p>All the software is distributed with the Affero GPL v3 license.</p>
Auditory-Tactile Narratives: Designing New Embodied Auditory-Tactile Mappings Using Body Maps
No full textThe increasing number of haptic interfaces for musical applications offers opportunities to create new auditory-tactile mappings. However, the innate, bodily correspondences between music and vibrotactile stimuli remain unexplored and a challenge to articulate. In this paper, we investigate how new mappings can be created based on the somatic, felt experiences from vibrotactile devices. We utilize an embedded HCI method to collect, articulate, and document such bodily experiences through body maps. Through an exploratory workshop with mixed-background participants and a focus group analysis, the participants reported their bodily experiences using two categories of haptic devices on the body maps and ideated new mappings through co-design. Based on the results of these sessions, we report three annotated auditory-tactile mapping schemes, each of which offers different strategies. The themes that emerged in the workshop interactions help us examine implicit associations of auditory-tactile stimuli on bodily sensations and how body maps can be used to articulate these sensations while offering an opportunity to generate new ones
The BodyHarp:Designing the intersection between the instrument and the body
No full textMutualizing the body and the instrument offers a different way of thinking about designing embodied musical interfaces. This research presents the design of the BodyHarp, a wearable instrument that combines large bodybased gestures with the fine control of hand-based instruments. This reflects a desire to create a single interface that both captures expressive, dance-like, body movement as well as nuanced gestural interactions. The BodyHarp was not designed as a separate artifact, but rather it was crafted as an augmentation to the human body. This fusion seeks to enhance the sense of intimacy between the player and the instrument and carries a different type of aesthetic - like playing a traditional instrument (the harp) but as part of the body. In other words, the BodyHarp aims to capture creative body movement and placing it in an instrumental context. In doing so, we aim to blur the transition between two gestural mediums (dance and playing an instrument) by mutualizing them - or, in a sense, by designing the interface as a part of the body.</p
Exploring Uncharted Soundscapes:Innovative Interaction and Mapping Techniques with the Bodyharp and the Teinophon
Get PDFThe evolution of Digital Musical Instruments (DMIs) has pushed the boundaries of musical expression and interaction. These instruments leverage advanced technologies to offer new dimensions of creative possibilities. This paper investigates the musical possibilities of Bodyharp [1] and the Teinophon [2], each designed to go beyond the limits of traditional instruments. We aim to uncover new perspectives on performer interaction and mapping, using the capabilities of these instruments to surpass traditional stringed-instrument boundaries with their infusion of sensor-based technologies. The study focuses on crafting and analyzing inventive mappings that connect the performer's gestures to sound, extending digital musical interactions beyond traditional instrument techniques dictated by physical and acoustical limitations. We explore uncharted sonic territories, drawing inspiration from the instruments' familiarities, yet extending them to create unique musical interactions. Through qualitative and performance-based examination of the interaction between body movements and sonic outcomes, we identify mappings that offer the most compelling and engaging results. This research recognizes the personal nature of musical preferences while encouraging creative exploration following a workshop-style evaluation method. We collected data from eight participants based on their experience with exploring both instruments' affordances and with solo and collaborative performance practices. Our work contributes to the broader discussion on sensor-based instruments, providing insights into their potential for expanding musical expression beyond established norms. By sparking further innovation and exploration, we hope to deepen our understanding of the creative possibilities embedded in Bodyharp and Teinophon, thus paving the way for new dimensions in contemporary musical expression.</p
Sonic Agency: A Group Autoethnography of Technology-mediated Performance Practice by Deaf and Hard of Hearing Musicians
No full textWhile many assistive devices and accessible technologies aim to increase access and participation of d/Deaf and Hard of Hearing (DHH) audiences, DHH people’s active participation in music performance remains limited. Through a 15-month-long group autoethnography, we present a case study of a Deaf-owned music and culture institution in a mixed-hearing, technology-mediated performance practice. We unpack how technologies are adapted and developed by DHH musicians and highlight the existing and emerging challenges of mixed hearing creative spaces. We use retrospective accounts, field notes, and performance artifacts to reveal the methods we developed for more inclusive music co-creativity. We lastly present three essential strategies that support DHH individuals’ sonic agency and access, with or without prior music knowledge, applicable by different actors who partake in the DHH or mixed hearing music performance ecosystem
