Array, the journal of the International Computer Music Association

Akkermann, Miriam

array2022
Music is in the air. Sounding per-
formances in hybrid and virtual
space
By Miriam Akkermann
While physical space is fundamental
to any sound's (physical) being –
and thus being heard – advance-
ments in technology and COVID
pandemic-related limitations to
physical travel and meeting in larger
crowds prompted debate on how to
design hybrid and virtual spaces in
which music and sound art can be
performed adequately. The question
of how to make music together
while being located at distant
places, as well as issues concerning
the integration of a wide-spread au-
dience using telecommunication
technologies, is, however, neither
completely new nor limited to digi-
tal virtuality. Currently termed as
‘telematic’and ‘networked’art works
and performances, there exists a
quite long history of using distrib-
uted sounds and sound related in-
formation in order to create artistic
settings and performances. For ex-
ample, listening to live music perfor-
mances or entertainment programs
from a distance was already possible
in the transition to the 20th century.
Facilitated by Electrophone tele-
space
phone broadcast services (which are
best documented in the UK and
France), listeners could dial in and
listen to requested performances
(theatre plays, music performances,
etc.) via telephone by means of
headphones. This was possible both
alone or in groups, the latter en-
abled by multi-headset tables
(“Electrophone tables”) at home and
in public, for example in commercial
Electrophone salons as a ‘pay as you
go’ service. This service was suc-
ceeded by radio broadcasts which
were less expensive for the audi-
ence and more easy to receive.
While telephone lines and most of
the emerging television programs
formats aimed at a single directed
(sending) broadcast, Simon Emmer-
son emphasizes the role of the radio
which “could be turned into a bi- or
multi-directional instrument, a net-
work of connections that could facil-
itate a telematic exchange between
all participants” (Emmerson 2000:
181). As an example, Emmerson
mentions Horizontal Radio (1995)
and Rivers and Bridges (1996), two
projects organized by ORF-Kunstra-
dio which explore the radio’s capac-
ity for global networking by bring-
ing remotely situated sound artists,
engineers and producers in contact
46
array2022
and interaction.
The idea of a real-time interaction
between distant partitioners was
also explored using the emerging
digital technologies towards the
end of the 20th century. Starting
with single connection lines, it was
due to Georg Hajdu in 1991/92 that
Richard Teitelbaum could partici-
pate in the anniversary concert of
the composers’collective Musica
Elettronica Viva at Mills College
while being stuck in his car due to a
blizzard using a modem connection
“that would enable him to remotely
play a MIDI keyboard while listening
to the performance over the tele-
phone” (Hajdu 2012: 2). The devel-
opments of the early 1990s were fol-
lowed by a decent number of wide-
spread, artistically and technically
diverse approaches which can be
subsumed under the terms ‘Netz-
musik‘ (Föllmer 2005), ‘internet
music’or ‘internet-based music
networks’ (Manning 2004), and
‘computer network music’
(Gresham-Lancaster 2013; for an
overview until 2008 see also Akker-
mann 2014), while other works ap-
peared in the context of multi-media
art and web-based (fine) arts, sound
art, or simply as music perfor-
mances (Wahl 2013). Not all of these
space
artworks deliberately work on space
as a fundamental, but all involve to
some extent the creative examina-
tion and handling of musical mate-
rial at geographically distant loca-
tions which are meant to share ide-
ally synchronous sound at different
locations, whether by networked
distribution or via a shared virtual
space.
Together apart in the same
(virtual) space
With the possibility of being inde-
pendent from physical (or architec-
tural) space, it becomes obvious –
even though not explicitly ad-
dressed – how strongly the aspect
of space relates to the created musi-
cal work. Being able to play or per-
ceive the same sound together
without the limitations of physical
space opens up new perspectives
on artistic processes in sound cre-
ation and its performance. Hence,
the technologies that enable real-
time and offline transmitting and re-
ceiving of data including sound and
visuals seem to lead to a situation
where temporal and geographic
boundaries start to dissolve. This
raises new questions for artistic cre-
ations: How can or should the com-
47
array2022
plex (technical and artistic) setting
be integrated in a composition’s
outline? How can sounds, musicians,
interfaces, audience, responses, etc.
be (re)present(ed) at distant loca-
tions? What does it mean to create a
shared moment in a virtual perfor-
mance? How do artistic works deal
with the relationship of physical and
virtual spaces? And how does this
influence the way an artwork can be
documented and analyzed?
A challenge that emerges for telem-
atic performances is, following Em-
merson in 2000, that it is difficult to
understand what “happens”when
‘just’ following a performance – the
interaction can seem incomprehen-
sible for the audience (Emmerson
2000, 186).
Emmerson claims that
“[t]elematic work is an offer for ex-
change. Artists and businesses alike
will have to come to terms with these
new technological and cultural facts
[… ]  [suggesting] changes in sound
art creation and perception that go
beyond modernist concepts of au-
thorship, owner-ship, artistic self-ex-
pression through the production of
unique works and ‘truthful’percep-
tion. They advocate new ways of art
making – not defined by work but
process; not through the polarisation
of creator and receiver, but rather
space
through communication and interac-
tion between them.” (Emmerson
2000, 187).
In 2000, Randall Packer and Steve
Bradley presented Telemusic#1, a
collaborative performance in which
Packer and Bradley mention seven
designers and two studios as co-
creators. In their introduction to the
premiere they explained that
„the demarcation between physical
and virtual space, between on-line
and local proximity, between the self
and the network, converges and
blurs into a shared, participatory ex-
perience through sound and our at-
tention to its spatial and transforma-
tional qualities.” (Packer and Bradley
2000)
Now, in the early 2020s, technical
developments such as real-time on-
line platforms for concerts and mu-
sic festivals, interactive virtual spa-
ces in 2D and 3D including Virtual
Reality and Augmented Reality
technologies, online platforms for
collaborative music making in real-
time and offline as well as a variety
of streaming and receiving options
for audio-visual contents exist (see
e.g. Cope 2001; Lazzaro 2001; Hugill
2005; Duckworth 2013; Serafin et al.
2016; 2017; Hamilton 2019). Mean-
while, some of the basic questions
are still in negotiation; for example:
48
array2022
How to deal with shared author-
and ownerships in a multi-dimen-
sional setting? What does a musical
or sound related work look like that
is genuinely made for virtual space
and takes all the advantages of this
setting? How does virtual space
change the roles of audience, com-
posers and musicians when a per-
formance is intended to be accessi-
ble (also) via telecommunication de-
vices – online and offline?
Settings and spaces side by side
It seems especially clear that new
digitally enabled virtual spaces
present major changes concerning
artistic, technical, and social aspects,
ranging from creating musical
works dedicated to this very specific
performance environment, estab-
lishing performances in virtual
space, and providing experiences
for the audience that include inter-
action and involvement as intended
in physical settings. This goes hand
in hand with new technological de-
velopments including virtual spa-
tialization of sounds and their repre-
sentations, leading to another new
challenge: While invisible connec-
tions enable the audience to ex-
plore and eventually interact with
space
sounds from all over the world, the
listening situation (as well as the au-
dio-visual representation at the mo-
ment of perceiving a musical work)
becomes both a tool for creation
and a requirement for the individual
audience who can not necessarily
expect a prepared and accessible lis-
tening room but who needs to facili-
tate their own personal setting for
experiencing a performance. This
again leads to a choice for the audi-
ence and to the need of a suitable
technical set-up. What equipment
does someone need to experience
the virtual space in the intended
way? Does everyone need speaker
systems or is it created to be for
headphones? This, in some ways,
would this lead us back to the idea
of the Electrophone table, maybe
now as a ‘VR headset station’provid-
ing public access to technology that
may not be at each home, thus en-
abling audiences to experience on
demand content together in virtual
space – side by side.
Notes
Science Museum Group.„Electro-
phone table, by National Telephone
Company, England, c.1895-1925“;
https://collection.sciencemuseum-
group.org.uk/objects/co8615954/
49
array2022
electrophone-table-by-national-
telephone-company-england-
c-1895-1925-electrophone-table
(last access 3 April 2023); Natasha
Kitcher.„Electrophone: the Victo-
rian-era gadget that was a precursor
to live-streaming“; The Conversation,
12. Nov. 2020; https://theconversa-
tion.com/electrophone-the-victo-
rian-era-gadget-that-was-a-precur-
sor-to-live-streaming-148944 (last
access 3 April 2023).
References
Akkermann, M. (2014), “Computer
Network Music – Approximation to
a Far-Scattered History”, Proceedings
of the EMS 2014, www.ems-
network.org/spip.php?article363.
Cope, D. (2001), Virtual Music:
Computer Synthesis of Musical Style,
MIT Press: Cambridge.
Duckworth, W. (2013), Virtual Music
How the Web Got Wired for Sound,
Taylor and Francis.
Emmerson, S. (2000), Music, Elec-
tronic Media and Culture, Ashgate:
Adlershot.
Föllmer, Golo (2005), Netzmusik,
Wolke: Hofheim.
Gresham-Lancaster, S. (2013),
“Computer Music Network”,
Leonardo, MIT Press.
space
Hajdu, Georg (2012), “Composing for
Networks”, www.georghajdu.de/gh/
fileadmin/material/articles/
Composing_for_Networks.pdf (last
access 3 June 2023).
Hamilton, Rob (2019), “Collaborative
and Competitive Futures for Virtual
Reality Music and Sound”, 2019 IEEE
Conference on Virtual Reality and 3D
User Interfaces VR: 1510–12, https://
doi.org/10.1109/VR.2019.8798166.
Hugill, Andrew (2005), “From Online
Symphonies to MusiMOO: Some
Creative Uses of the Internet in
Music and Education”, Contemporary
Music Review 24 (6): 527–40.
Lazzaro, J. (2001), “A Case for Net-
work Musical Performance”, Pro-
ceedings of the 11th international
workshop on Network and operating
systems support for digital audio and
video, NOSSDAV, New York, United
States: 157–66.
Manning, P. (2004), Electronic &
Computer Music, Oxford University
Press: Oxford.
Packer, R. and S. Bradley (2000),
“TELEMUSIC #1 – Premiere
Performance”. Rhizome, 22 October
2000. http://rhizome.org/discuss/
view/29788/ (last access 3 June
2023).
50
array2022 space
Serafin, S. Et al. (2017), “Considera-
tions on the Use of Virtual and
Augmented Reality Technologies in
Music Education”, IEEE Virtual Reality
Workshop on K-12 Embodied Learn-
ing through Virtual & Augmented
Reality (KELVAR): 1–4, https://doi.org/
10.1109/KELVAR.2017.7961562.
Serafin, S. et al. (2016). “Virtual
Reality Musical Instruments: State of
the Art, Design Principles, and
Future Directions”, Computer Music
Journal 40 (3): 22–40, https://
doi.org/10.1162/COMJ_a_00372.
Wahl, C. (2013), “Between Art
History and Media History: A Brief
Introduction to Media Art”, in:
Challenges and Perspectives, edited
by V. Hediger et al., Amsterdam:
25–58, doi:10.1515/9789048513
833-004.
51


English

Music is in the air. Sounding performances in hybrid and virtual space

Jarema, Andy

Currents: AWeather Data Sonifica-
tion Project
By Andy Jarema
Communicating the existential
threat posed by climate change is
essential to achieving a courageous
leadership and strong policy
changes needed to avoid the most
catastrophic possible futures in
store for our planet. Composers and
other artists have much to offer in
the conversation about our shifting
planet through the power of artistic
storytelling mediums. Making an
audience feel scientific data can
keep the conversation afloat about
the urgency of climate change and
the ominous shifts in weather pat-
terns on our planet.
Currents is a multifaceted and
ongoing musical work that centers
around two guiding questions: How
can we use sonification to musically
depict weather and climate data,
and how can this musical narrative
affect how we feel about the data?
Specifically, Currents sonifies a data-
base comprising forty years of data
created by the National Oceanic and
Atmospheric Administration (NOAA)
that tracks Billion-DollarWeather
and Climate Disasters in the United
States. In this article, I discuss my
first iteration of the piece, providing
insight into the sonification process
of Currents, including technical chal-
lenges in the sonification process,
how the data elements were soni-
fied, and important questions that
emerged when translating big data
into a creative musical work.
Currents
Information about climate change
traditionally relies on communicat-
ing rigorous evidence from the sci-
entific community: facts, figures,
graphs, and data. But truthfully, I
have only personally been moved to
change by artistic works. It was a
film that first lit my passion to fight
climate change (An Inconvenient
Truth, a documentary by Davis
Guggenheim from 2006), and an au-
thor that convinced me to take my
most potent individual climate ac-
tion by eating no animal products
(Jonathan Safran Foer’sWe Are The
Weather: Saving the Planet Begins at
Breakfast). As a composer, I find my-
self wondering what we artists have
to offer in terms of making people
feel something different about the
alarming climate data presented to
us by the scientific community.
52
array2023 flux
Currents is a fixed audio-visual work
created in Max/MSP that covers all
forty years of data (1980-2019) in
roughly eight minutes.1 It utilizes
visual graphs, field recordings, and
synthesized sounds to provide a
lofty overview of the entire dataset
and gives the numbers a more com-
pelling artistic narrative.
I want to begin by outlining the
initial challenging steps of working
with the data used for creating
Currents. I hope that this will help
illuminate the issues with big data
sonification other composers might
face in the future. Max/MSP was the
software I chose for the data sonifi-
cation process due to my familiarity
with the programming language,
however my technological abilities
quickly had to expand to accommo-
date the multiple moving parts
needed to access and sort such a
large web-based data set. The com-
position’s Max patch utilizes a fea-
ture introduced in Max 8 known as
“Node.js,”which is a Javascript-
based method of communicating
with web-based resources. “Node.js”
assisted with accessing a JSON data
file of the Billion-DollarWeather and
Climate Disasters database through
the NOAA website, including sorting
and parsing the data into three col-
umns of information: number of bil-
lion-dollar weather events per year,
the cost of each individual billion-
dollar weather event, and the num-
ber of deaths that occurred with
each event. From there, a “coll”ob-
ject in Max was used to store and
export the three columns of data as
three separate .txt files to my hard-
drive. By reimporting the .txt files
back into three separate “coll”ob-
jects, Max was able to travel
through the data and create the ba-
sis for translating these numbers
into sonic information.
I want to stress that accessing and
parsing this data was non-intuitive,
oblique, and challenging. It was a
frustrating monthslong endeavor
filled with failed experiments, des-
perate shifting through YouTube tu-
torials, and re-tooling existing Java
script code designed for other pur-
poses entirely. Smaller data sets
might be able to be fed into an au-
dio program like Max/MSP rather
quickly, but there seems to be no
single method of sonifying large
data sets without the use of multi-
ple programming techniques (Max/
MSP, Node.js, Javascript, JSON files)
which creates a high barrier of entry
into this process for the average
53
array2023 flux
composer.
After accessing and parsing the
data, Max/MSP was used to sonify
the weather data using field record-
ings and generative FM synthesized
sounds. The field recordings used in
this piece were recorded with my
Zoom H4 recorder in Great Smoky
Mountains National Park (crickets,
birds) and Hawai’i Volcanoes Na-
tional Park (hydrophone recording
of the ocean). The three sets of natu-
ral sounds (crickets, birds, ocean)
represent three different environ-
ments (land, sky, and sea), function-
ing as an abstract barometer that al-
lows us to check in and take read-
ings on the health of three distant
biomes on our planet.
The top graph tracks the number
of billion dollar weather events that
occur in each year (40 data points)
and is sonically depicted with crick-
ets (center pan). There is an inverse
pitch relationship between the
sound and the data, where a higher
number of weather events corre-
lates to a lower shift in the pitch of
the cricket recording. This was pri-
marily an artistic choice given that
the cricket recording takes on an
alarm bell-like quality at lower pitch
levels. The middle graph tracks the
cost of each individual weather
event (258 data points) and is soni-
cally depicted with a crackling un-
derwater hydrophone recording of
the ocean (right pan). A higher cost
triggers a longer articulation (enve-
lope release) of the sound, and sig-
nificantly high-cost events over $20
billion trigger the sound of loud
ocean waves. The bottom graph
tracks the deaths caused by each in-
dividual weather event (258 data
points) and is sonically depicted
with a bird call (left pan). There is
also an inverse pitch relationship
similar to the cricket sounds (a
higher death toll triggers a ghostly,
lower pitched bird call) and signifi-
cantly fatal events over 100 deaths
trigger the sound of chickadees
(chickadees use an intricate lan-
guage of bird calls as a way to
broadcast perceived threat levels in
its environment).
In addition to the three sonified ele-
ments operating at a micro level in
the piece, there are sonic elements
that function on a much larger time
scale to convey a broader narrative
in the data. Taken as a whole, the
forty year data set outlines an over-
all rising trend of billion dollar
weather events. To artistically reflect
the urgency of this narrative, certain
54
array2023 flux
musical elements such as tempo (a
gradual accelerando over time) and
timbre (stark FM synthesized sounds
layering in over the field recordings)
unfurl over the eight minutes time
span of the piece. Approximately
half way through the piece, another
hydrophone recording sneaks into
the right speaker. The recording
slowly begins creeping to fill the left
speaker, dropping in pitch as it goes
along. The recording wildly in-
creases in pitch and speed at the
end of the piece (the year 2020), fur-
ther signifying urgency as the sound
burns and crackles out of control.
One question that frequently
emerged in the sonification process
of Currentswas how to address the
tension between data accuracy and
creative artistry. Being faithful to the
data in order to tell a scientifically
sound narrative was a priority, but
on the other hand I found myself
taking creative licenses in the sake
of creating a compelling musical
narrative. One of these challenges
was figuring out how to scale and
calibrate the data in the sonic do-
main. For example, the number of
events per year was easy to sonify
into the pitch domain due to its rela-
tively limited range of data (there
were 0 to 16 events, which was soni-
fied into a two octave range of
notes from C2 to C4). But the cost
55
array2023 flux
Figure 1: still taken from Currents to demonstrate these visual graphs and
sonified elements
provided a unique challenge since
this data mostly operated in a nar-
row band (costs for each weather
event hovered mostly between one
and 20 billion) with occasional out-
liers stretching the upper bounds of
the data (Hurricane Katrina’s 192 bil-
lion being the highest). If this data
was sonified directly into the pitch
domain, it resulted in a flat pitch
output with mostly imperceptible
pitch changes (semitones or tones)
with occasional jumps to a higher
note when an outlier occurred. On
the other hand, adjusting my sonifi-
cation to pick up the nuance in
smaller values failed to sonically
capture the magnitude of the out-
liers when they occurred (in some
cases, the outlier was so high it pro-
duced a pitch outside the range of
human hearing). To fix this, I created
“parameter bands” (Figure 2) that
would sonify different ranges of
data to different ranges of pitches:
1-10 billion was sonified within a
one octave range (C4 to C5), 10-100
billion in another octave (C5 to C6),
and the upper outliers between
100-167 in a final octave (C6 to C7).
These types of decisions throughout
the sonification process of Currents
forced me to confront the bound-
aries between scientific accuracy
and creative artistry, ultimately
helping to shape my narrative inten-
tions and clarify the messaging be-
hind the piece.
Another question that emerged in
the sonification process was how to
utilize big data in an ethical way. A
valid critique of Currents is that the
data points scrolling across the
screen can feel objective, imper-
sonal, and disconnected from indi-
vidual experience. I myself was a
part of one of these extreme
weather events (2014 Detroit flood-
ing) and seeing the data point slip
so quickly across the screen some-
what diminishes my harrowing rec-
ollection of that day. It especially
makes me uncomfortable to see the
data points for deaths, real human
lives being lost, ticking away across
the screen with an almost impas-
sionate calculus. It reinforces the
56
array2023 flux
Figure 2:
Paramter
Bands
narrative that the problem of cli-
mate change feels too massive, its
complexities and horrors beyond
any sort of action that can possibly
reverse its course. If the objective of
climate-related art is to emotionally
move people to awareness (and
therefore- action), I sometimes won-
der if using such a large data set is
always the most potent approach.
This is where Currents blossoms into
not just one iteration, but an overar-
ching weather data sonification
project. Shortly after the premiere of
this fixed audio-visual iteration at
the 2020 Earth Day Art Model Festi-
val, I began working on the next
chapter of the Currents project. As
opposed to a fixed audio-visual
piece sweeping through mountains
of data, the next iteration of Currents
zoomed in on seven specific
weather events out of the 258 cata-
logued in the database. I crafted
notated sheet music shaped by data
sonification techniques and enlisted
seven wind/percussion players to
perform these short solo works with
live electronic processing via Max/
MSP. For example, the aforemen-
tioned 2014 Detroit flooding was
outlined by a percussion player us-
ing five different instruments to de-
pict the five inches that fell on De-
troit in roughly five hours. Using
hourly precipitation amounts from
theWeather Underground, I
mapped out the rate of rainfall and
projected this timeline overtop the
31 measures of the piece. The per-
cussion player adds a new instru-
ment into the mix after each new
inch of rain falls, leading to an even-
tual flood of five instrument sounds
bolstered by Max’s live processing
into back-masked“waves”of percus-
sion sound. This iteration of Currents
felt so much more personal: the de-
piction of individual weather events,
a community of live musicians tan-
gibly engaging with the music, the
inclusion of a single weather event
that personally affected me and my
community. Though, related to the
previous question of data accuracy
versus creative artistry, this project
took many more artistic liberties
and did not have the same rigorous
adherence to the numbers that the
first audio-visual iteration did.
To me, Currents functions as a blue-
print, an ever evolving set of works
that strives to make artistic narra-
tives out of scientific data. At its
heart it is a project concerned with
communicating an artistic narrative
57
array2023 flux
about climate change, and I hope to
continue to adapt the project into
radically new mediums to reach as
many audiences as possible. Per-
haps it could be adapted into sheet
music for young beginning band
players and include a curricular ele-
ment that could facilitate climate
change discussion in the classroom.
Or perhaps it could function as a
video game, an interactive fiction
piece using the programming lan-
guage Twine that combines written
and sonic narratives in a way that
implicates the audience into its
process (video games demand the
player’s participation for the genesis
of the story to occur). I believe we as
artists have so much to offer in the
conversation about our shifting
planet through the power of artistic
storytelling mediums, and making
an audience feel something differ-
ent then scientific data can show,
keeping the conversation afloat
about the urgency of climate
change and the ominous shifts in
weather patterns on our planet.
Notes
[1] Recording of Currents
www.youtube.com/watch?v=97J1
CG88UV0 (last access Nov 11, 2023).
58
array2023 flux


Currents: A Weather Data Sonification Project

Gerard, Garrison

Patagonian Soundscapes: An acoustic survey of sub-antarctic Chile and generative soundscape composition

Giuseppe, Pisano

array2022
Stereo and Ambisonics: A reflec-
tion over parallel spatialization
techniques in the piece Scie Lumi-
nosissime
By Giuseppe Pisano
As a composer of acousmatic music,
I have often presented my work on a
variety of different loudspeaker sys-
tems whese can be subdivided in
two macro-families: those intended
for the playback of multichannel
pieces – often symmetric systems,
in the shape of a circle, a dome, or
a regular polygon – and those in-
tended for acousmatic diffusions –
the loudspeaker orchestras, mainly
employed for the diffusion of stereo
pieces, where the performer spatial-
izes sound in real time. In this prac-
tice, it is not uncommon to listen to
re-adaptations of pieces, presented
in different formats to suit different
types of systems, and I have always
found the Ambisonics workflow a
very successful approach to scale
my pieces on systems belonging to
the first macro-family. However,
when I first faced the need of pre-
senting my multichannel pieces on
systems meant for stereo diffusion, I
was very unhappy with the results I
achieved through the typical meth-
ods employed to adapt Ambisonics
space
to stereo, such as rendering my
pieces in Ambisonics UHJ format, or
using virtual microphones to record
the sound field in stereo. Based on
this experience, I decided to com-
pose a piece in two versions in par-
allel, for stereo-setting and Am-
bisonics.
This process initiated a reflection
process on how I would manage
sonic materials differently according
to the format to emphasize their
characteristics and has become an
acquired strategy in my composi-
tional practice.
The first piece I composed in parallel
as a stereo- and Ambisonics-version
was Scie Luminosissime (2018-2020)1.
In this paper I outline the composi-
tional process of this piece, and
highlight some conceptual differ-
ences that determined my choices
in the use of the materials and the
approach to spatialization.
Scie Luminosissime was inspired by
the album Fratello Mare by Mike
Cooper and the poem Sii dolce con
me. Sii gentile by Mariangela
Gualtieri. This inspiration is rather in-
direct but provides the setting in
which I wanted to compose.
The materials used in Scie Luminosis-
sime include field recordings ob-
14
array2022
tained during the summers of 2016
and 2017 – mostly fragments of hu-
man agencies in different reverber-
ating spaces – in dialogue with con-
crete and synthesized sounds – the
scie, usually distorted and/or convo-
luted trails of synthesized sound
tones – and a few bits of sound sam-
pled from different guitar pieces by
Mike Cooper. My goal was to gener-
ate momentary and elusive visual-
izations of human interactions in an
otherwise diffused blur of gestures
and fragments.
The time-structure of the piece can
be divided in two parts, each subdi-
vided in two sections.
space
The first section is mainly character-
ized by simple tones (1) to which
other concrete elements are gradu-
ally added, occupying wider por-
tions of the spectrum (2). The cumu-
lation ends with a gesture (3) that
opens a new section and reiterates
later (4) with a different organiza-
tion of the materials. The idea was to
create a new perspective, as if the
same picture was seen from a differ-
ent angle. After this, the space dissi-
pates fading back into luminous
trails again.
In his article “Space-form and the
Acousmatic image”, Denis Smalley
Figure 1: spectrogram of Scie Luminosissime, the numbers highlight the four
sections of the piece.
15
array2022
(2007) describes a soundscape he
listens to sitting by his window. He
is inside, and the whole world is out-
side, beyond the fourth wall. The
idea of being a spectator of sound
from a vantage point reinforces
once again the parallel between
acousmatic music and cinema
which has been structural in the dis-
course originated in the French liter-
ary and musical school (Chion 1990;
Normandeau 1992). This connection
is even more important now that
the expansion of sound from stereo
to more immersive formats is be-
coming a widely available technol-
ogy, especially through the film
industry and other audio-visual
media, and demonstrates how
Smalley’s front-based approach to
the description of the soundscape
in the article is in reality much more
techno-centric than openly in-
tended.
In Smalley’s narration, the window is
the point from which the sound
radiates in the listening space, it is
both an access to the external sonic
environment and a filter that
facilitates the localization of the
different sound sources.
When we listen to music in stereo,
loudspeakers behave in a very simi-
space
lar way as this window. They define
a front from which sounds radiates
in our listening space, a two-dimen-
sional surface on which a three-
dimensional illusion is represented
as in a trompe-l’oeil (more a trompe-
l’oreille in our case) that uses per-
spective to generate depth. This
changes radically when we are sur-
rounded by loudspeakers. We do
not sit by a window, we sit under a
“glass dome”, we can still have a
frontal perspective, but our percep-
tion responds to stimuli coming
from every direction. In this situa-
tion, music is then not merely pro-
jected in the listening space, but it
becomes the listening space. The
listener is in the sound field, im-
mersed, and the sound is an archi-
tecture.
Materials in an immersive sound-
field context are hierarchically orga-
nized in a figure/ground relation-
ship. This gestalt notion, that has
been applied before to many other
fields such as soundscape analysis
(Porteous and Mastin 1985) and
acousmatic music (Santarcangelo
and Wanke 2020), is of course still
relevant for immersive formats
where music-space and listening
space overlap. A good example
might be the nomenclature used to
16
array2022
distinguish between the two types
of sound sources in Dolby Atmos:
Beds and Objects. Beds are multi-
channel sources, meant to establish
a scene, while Objects are mono
sources that would benefit from a
more precise positioning and from
panning.2 This distinction is not just
practical or operational, but reflects
a difference in the perceptual and
musical roles of the sound materials,
which can also be applied to other
immersive formats. When compos-
ing in Ambisonics, authors can
adopt a similar approach. Some
sounds can be treated as structural
pillars of a scene, establishing the
environment. Other materials can
have more prominent roles, not only
in terms of levels and clarity, but
also in their spatial behavior with
faster trajectories, swift movements,
dramatic panning. However, the ab-
sence of a structural distinction be-
tween sources and their roles allows
for more nuanced musical solutions
and alterations in the roles of the
materials.
An example of how I apply this para-
digm is at the beginning of Scie Lu-
minosissime, where, in the begin-
ning, every single tone is a fixed po-
sition, like in a constellation, where
space
every star has its own place. These
become spatial references, describ-
ing and defining the sonic environ-
ment with their presence.
As the piece develops, more reso-
nances add up together with similar
elements of different timbres, mov-
ing slightly or appearing in opposite
positions to the sound they origi-
nate from. These new materials gen-
erate a movement around the origi-
nal structure, and every new tone is
introduced with similar copies of it-
self, each presenting slightly differ-
ent characteristics. The original in-
stances are still fixed in space, as
solid statements of presence. The
other ones emerge from the back-
ground, creating a shapeshifting en-
vironment.
In the stereo version of the piece,
this effect was achieved by deter-
mining a position for each original
source on the horizontal plane,
choosing to leave low frequency
sounds in the middle and to enlarge
the picture progressively at higher
frequencies. Resonances were ob-
tained with a stereo reverb, invert-
ing channels after a low pass filter.
In the Ambisonics version, I used a
different technique encoding the
original mono sources with different
17
array2022
instances of IEM MultiEncoder,3 cre-
ating new copies of some of them
to fill the entire space.
For the additional sounds emerging,
I decided to create small alterations
in their position modulating the val-
ues for the coordinates with a ran-
dom LFO at a very small amplitude,
which generated an organic liveli-
ness.
Lower frequency sounds and reso-
nances which I wanted to sound
bigger and further away were en-
coded using the AmbiX suite by
Matthias Kronlachner (2013). This
software presents an additional
parameter called size that can be
used to adjust spatial directivity, ad-
justing the ratio between the levels
of the omnidirectional signal and
those of the other spherical har-
monic components. At higher val-
ues of size, the sound sources are
spatialized as more omnidirectional,
equally distributing the sound to all
loudspeaker, making them appear
bigger, but also less directional. Ad-
justing this parameter, I was able to
spatialize sounds with a weaker
sense of directionality which played
a different role.
space
Gestural sounds which, in the stereo
version of the piece, were spatial-
ized using traditional linear pan-
ning, in Ambisonics expand their
movement to the second and third
dimensions, swooshing from one
side to the other of the sphere.
These movements are achieved us-
ing crossfades between two en-
coded monophonic sources, and are
presented over multiple diameters,
to increase the sense of tridimen-
sionality.
Another important difference be-
tween 3D Ambisonics and stereo is
the introduction of verticality. How-
ever, while real height can only be
found in three-dimensional sound
formats, different techniques can be
employed in stereo to generate a
perceptual illusion of height. Denis
Smalley provides some good exam-
ples for this when introducing the
concept of gravitation, referring to
the work of French composers such
as François Bayle, François Dela-
lande, and in particular to Bernard
Parmegiani and his piece Geologie
Sonore (Kronlachner 2013). In his
writings, Smalley addresses height
from a spectromorphological stand-
point, claiming that the illusion of
vertical movement is based on the
18
array2022
quality and the behavior of the
sonic materials. A psychoacoustic
approach to the production of verti-
cal panning illusions is provided by
Curtis Roads in the book Computer
Music Tutorial where he stresses the
role of the pinnae in sound localiza-
tion on the vertical plane. Showing
the Head Related Transfer Function
spectra of sounds coming from
above and below ear level, he de-
scribes the functioning of vertical
panning (Roads 1996).
In Ambisonics, I found it very useful
to spatialize sounds over the vertical
plane trying to imitate or to refer
the behavior and spectral character-
istics of sounds that I am most likely
to perceive above or below ear level
in real life. For example, there are
sounds of birds or sounds that “be-
have like”birds, there is the sound of
rain and a sound that “behaves like”
rain. Exploiting the vertical plain to
spatialize sounds can provide an ex-
tra layer of realism to unreal sounds.
Similar effects are much more diffi-
cult to achieve in stereo. As pointed
out by Roads (1996, 10), realistic ver-
tical panning illusions depend on
too many factors such as speaker
placement and listener’s position,
and are therefore extremely situa-
space
tional and very fragile. But Roads
also describes that sound localiza-
tion on the vertical plane is stronger
for sounds above 6 kHz. Keeping
this in mind, I found it useful to split
sounds with very complex spectra
over different frequency bands us-
ing a set of EQs, and spatialize these
bands individually. Increasing the
width of high frequency compo-
nents did not provide any clear illu-
sion of a vertical panning move-
ment, but it widened the stereo im-
age, giving more air and a stronger
sense of openness to the sound im-
age. I found it also useful to com-
bine this frequency-based approach
to the material- and spectromor-
phology-based approach described
by Smalley, and to see them as two
sides of the same coin which
together contribute to create an
illusion of height in a stereo mix.
Spatializing filtered components of
sound sources is a technique that
was used extensively in Scie Lumi-
nosissime, especially for granular
materials, where multiple mono-
phonic sources are processed using
the GRM Space Grain4 a multi-chan-
nel granulator whose resulting sig-
nals are then filtered and encoded
using multiple IEM Multi-Encoder
with some personal presets.
19
array2022
Scie Luminosissime was the first
piece I composed using this parallel
method, which I later adopted for
other works, too. I found this com-
positional process of creating two
versions of the same piece very use-
ful to thoroughly exploit the
strengths of both formats. The ver-
sions share the same materials, the
same structure and the same dura-
tion, but manifest themselves ani-
mated by very different spirits.
While the stereo version appears
more compact, perhaps with a
stronger body, more punch and a
concrete foundation, the Ambison-
ics version is more enveloping and
relaxed: gestures are perhaps less
dramatic, but more realistic.
When working in Ambisonics, my
feeling was that certain sounds did
not require the same dynamic pres-
ence that I was using in stereo. In-
stead, they could interpret their role
in the narrative development of the
piece much more successfully with
just a different organization of the
space.
Another relevant aspect is how the
relationship between full and empty
space was managed differently in
the two versions of the piece. Filling
space in Ambisonics simply requires
“more sound”, and adding multiple
space
voices of a sound material, if cor-
rectly positioned, will rarely results
in the space feeling crowded. It was
therefore necessary to double some
sound sources in the Ambisonics
version or to cut out few others in
the stereo version to balance the
sense of fullness and emptiness in
different moments of the piece to
make sure that the same impression
was achieved.
In conclusion I think that my ap-
proach to the composition of this
piece has perfectly filled the pur-
pose of making the two versions
perfectly functional to the two dif-
ferent types of loudspeaker systems
and performance styles. Expanding
this workflow to other pieces has
made possible to present the en-
tirety of my work in all possible con-
texts.
Notes
[1] www.giuseppepisano.com/fixed-
media/ (last access Feb. 16, 2023).
[2] Dolby Atmos Renderer v3.0.
Dolby Lab. Inc, 2 August 2018, 116.
[3] IEM – Plug-in description
“multiencoder”, 2022.
https://plugins.iem.at/docs/plugin
descriptions/#multiencoder
(last access Feb. 16, 2023).
20
array2022 space
[4] InaGRM – Spaces, 2022. https://
inagrm.com/en/store/product/15/
spaces (last access Feb. 16, 2023).
References
Smalley, D. (2007). "Space-Form and
the Acousmatic Image." Organised
sound 12(1): 35-58.
Chion, M. (1990). L’audio-Vision. Son
Et Image Au Cinéma. Nathan, Paris.
Normandeau, R. (1992). "Un Cinema
Pour L’oreille." Doctoral dissertation,
Montreal: Universite de Montreal.
Porteous, J. D., and Mastin, J. F.
(1985). "Soundscape", Journal of
Architectural and Planning Research:
169-86.
Santarcangelo, V. and Wanke, R.
(2020). "The Early Stage of
Perception of Contemporary Art
Music: A Matter of Time." Organised
Sound 25 (2): 130-41.
Kronlachner, M. (2013). "Ambisonics
Plug-in Suite for Production and
Performance Usage." Paper
presented at the Linux Audio
Conference.
Roads, C. (1996). “Sound
Spatialization and Reverberation”. In
idem, The Computer Music Tutorial.
MIT press: 466-469.
21


Stereo and Ambisonics: A reflection over parallel spatialization techniques

Deal, Scott

Flux in the Anthropocene: An
Overview
By Scott Deal
Ring the bells that still can ring
Forget your perfect offering
There is a crack, a crack in everything
That's how the light gets in
(From "Anthem", by Leonard Cohen)
The Anthropocene, the current
epoch named for humanity's impact
on earth systems, is characterized
by some of the most pressing issues
of our time, including global warm-
ing, biodiversity loss, mass pollu-
tion, water shortages, and more.
Human activities, principally
through emissions of greenhouse
gases, have "unequivocally caused
global warming, with global tem-
peratures exceeding 1.1°C above
1850-1900" (United Nations IPCC,
2023). Global expansions in farming,
fishing, construction, mining, and
more is creating a crisis of biodiver-
sity where more, than one million
species are threatened with extinc-
tion (Tolleson, 2019). Fresh water re-
sources are overburdened as over-
consumption, pollution, and the de-
pletion of groundwater threaten en-
tire geographic regions. From
mountain tops to ocean bottoms,
the world is awash in plastic. As a
result of these conditions, the An-
thropocene has brought "an envi-
ronmental phase shift, where for-
merly resilient eco-systems have
been pushed into altered states"
(Kelly, J. M. et al., 2018). This shift
points to a state of “flux”, the title
word for this edition of Array.
Throughout the world, this flux con-
jures a zeitgeist of uncertainty.
Princeton scholar Richard Falk states
that "for a long time we got along
without worrying about the caring
capacity of the earth, and now we
are terrified by the limits that we are
exceeding." (Rimona et al., 2021) But
there is also room for hope, which is
where expressive aesthetics come
into play. Our turbulent world is the
starting point for creative observa-
tion, absorption, and articulation, a
process the authors in this edition of
Array have aptly engaged. Their
writings reflect personal explo-
rations into paths of inquiry, result-
ing in unique forms of musical art
that provide a basis for insight and
reflection regarding our time in this
world.
As a foundation of musical activity,
the process of listening is a broad
topic in these articles. Paulo C.
5
array2023 flux
Chagas cites listening as the funda-
mental construct of Sound Imagina-
tions: Listening Cultures and Audiovi-
sual Immersion, writing of his moti-
vation to "investigate sound phe-
nomenon from the point of view of
listening." He recorded environ-
ments in Brazil, California, and Rus-
sia, asking how "people hear the
acoustic environment in its totality
of nature and human cultural
sound". Several writers describe
their listening experiences as begin-
ning with recordings from scientific
remote devices. In Aural Snapshots
of Patagonia: An Acoustic Survey in
Sub-Antarctic Chile, Garrison Gerard
writes of documenting the acoustic
environment of Yendegaia National
Park in Chile, an extremely remote
natural setting that the Chilean gov-
ernment opened to the public by
creating a road to access it. With
plans to create a musical work from
the recordings, the effort will publi-
cally call attention to the stark real-
ity of a disappearing aural land-
scape. Amanda Stuart accessed
original audio samples from the
Canadian Array for Realtime Investi-
gations of Magnetic Activity
(CARISMA) to create her work Mag-
netosphere. In Six Seasons, Composi-
tion Inspired by Ocean Sounds From
the Arctic, Zalles Ballivian describes a
project employing hydrophones
placed on the seafloor of the
Chukchi Sea, 300 meters deep, to
capture "the sounds of sea ice, ma-
rine mammals, and the underwater
environment". Other writers pursue
less conventional modes of listen-
ing. In her article Indigeneity and
Computer Music in the Anthropocene,
Chieh Huang writes of listening be-
yond the audible spectrum into a
realm of sensations. "Vibrational
sensations, not just audibility, lie at
the core of all music". This world of
vibrations not perceived by the hu-
man ear is particularly well suited
for the technological capabilities of
the computer music genre, and its
exploration is an emerging creative
territory.
In the natural world, devoid of
any recording devices, the listening
experience is a unique, non-repeat-
able moment. In Glacial Music: Katie
Paterson’s Langjökull, Snæfellsjökull,
Solheimajökull (2007), author
Alexander Rehding describes how
Paterson exploited this factor by
recording the sounds of the three
glaciers and then pressing the audio
into phonograph records made
from ice of same glaciers. When the
ice records are played on a phono-
6
array2023 flux
graph, it melts and the "recorded
sound irrecoverably disappears".
Highlighting temporality in this
fashion recalls artists such as Andy
Goldsworthy and Robert Smithson,
whose works of natural materials
deteriorate over time. Listening con-
cepts are not limited to the experi-
ence itself but extend to aurally-
based tools. In Diverse Instrument
Symphonic Ensembles as a Strategy
for Sustainable Innovation, Jason
Palamara advocates for computer-
based ensembles that use sustain-
able and recycled devices as musical
instruments. His idea of a “Diverse
Instrument Symphonic Ensemble
(DISEnsemble)” seeks “increasingly
environmentally neutral solutions to
the task/problem of forward-look-
ing musical innovation”. In Networks
of (tele)presence: tuning in to the envi-
ronment through the INTIMAL App©,
Ximena Alarcón Díaz describes a hy-
brid telematic mobile sonic environ-
ment she created that invites peo-
ple to experience relational listening
by collectively engaging“their sense
of place and sense of presence
across distant locations”…as a “ne-
gotiation between sensing place in
a physical location, and, simultane-
ously, sensing presence and telep-
resence across distant locations”.
When harnessing scientific data
and field recordings for expression,
how do composers shape empirical
content into a musical work? What is
to make of the sometimes-
inevitable compromises that arise?
How do emerging tools/techniques
affect the musical direction and aes-
thetics of computer-based music?
These and other questions provide
valuable insight into the nature of
environmentally based electroa-
coustic process. In Currents: A
Weather Data Sonification Project,
Andy Jarema writes of struggling
how to best communicate a data-
base from the National Oceanic and
Atmospheric Administration
(NOAA), eventually settling upon a
combination of visual graphs, field
recordings, and synthesized sounds.
He also wrestled with mapping
graphs to the audible hearing range,
making decisions that “forced me to
confront the boundaries between
scientific accuracy and creative
artistry, ultimately helping to shape
my narrative intentions and clarify
the messaging behind the piece".
Stuart describes utilizing a 2-minute
sample of Orca and Ross Seal calls
that she transformed through vari-
ous audio processes into 61 differ-
ent sounds to create an imaginary
7
array2023 flux
world with visuals of her paintings.
Ballavian’s project relied upon the
creative activity of live musicians, in
this case, the Mivos Quartet, to
make choices regarding recorded
data from the sea-submerged hy-
drophones. They were presented
with a "living score" of the sounds
and were asked to then "create for
themselves a dictionary of tech-
niques that can be invoked in re-
sponse to various stimuli."
What are the conceptual intents
of environmental creators, and what
are the outcomes they seek from
their audiences? Some hope to in-
fluence people to action. Palamara
seeks to inspire sustainability, per-
haps triggering “a global movement
towards environmental awareness
and action”. Ballivian believes “we
have a responsibility to faithfully
present these sounds in an effort to
sway public opinion and affect hu-
man behavior". Stuart seeks to "draw
attention to these endangered
soundscapes, combining scientific
data and artistic perspectives to
highlight the need to preserve these
sounds - and what they represent -
before it’s too late". Jarema wonders
“what we artists have to offer in
terms of making people feel some-
thing different about the alarming
climate data presented to us by the
scientific community." However,
other intentions come into play.
Alarcón Díaz expresses hope that
users of her application explore
awareness through walking, to
“awake a sense of agency, which
might help us to become part of the
individual and collective possibili-
ties that emerge…a listening op-
portunity to become part of the
Earth’s resonance in its transition”.
Huang purposes to express connec-
tions to her indigenous Atayal cul-
ture, its teachings, and through
those concepts, to bring others into
a deeper connection to their own
environments. She invites listeners
to consider music as a bridge to cul-
tural underpinnings of those who
create it. Listeners are "ethically
obliged to immerse themselves in
the cultural context in which the
composition was created, to be con-
scious of the diverse relationships it
embodies, and to acknowledge that
the interpretation extends beyond a
mere musical performance”. Paulo
C. Chagas believes "that the funda-
mental purpose of his audiovisual
installation is to “promote connec-
tivity as an essential feature of hu-
man experience".
8
array2023 flux
Creatively engaging others with im-
portant ideas is one of the most
salient aspects of the arts. While the
number of artists within these
pages is small, they reflect a vast
global community striving to con-
nect with the environment and with
those around them. In a world pos-
sessing the greatest interconnectiv-
ity in history, the need for art to ful-
fill this function is greater than ever.
Computer music-media is what we
do well, and when it is focused on
the environment, our work certainly
impacts society’s perception of the
world. The Anthropocene has
brought threatening issues to bear.
There is uncertainty, but there is
hope.
Therefore, we ring the bells.
References
Kelly, J. M. et al. (2018). Rivers of the
Anthropocene. Oakland: University
of California Press, 2018, DOI:
10.1525/luminos.43.
Rimona A., et. al. (2021). “Ecocide/
Speciesism: Revisiting Interdepen-
dence in the Anthropocene.”Online
panel discussion at Earth Day Art
Model Festival. www.earthday
artmodel.org/index-2021.html (last
access Dec 1, 2023).
Tolleson, Jeff (2019). “Humans are
Driving One Million species to
Extinction.”Nature 569: 171. DOI:
10.1038/d41586-019-01448-4.
United Nations IPCC, (2023).
“Summary for Policymakers.” In: H.
Lee and J. Romero (eds.). Climate
Change 2023: Synthesis Report.
Contribution ofWorking Groups I, II
and III to the Sixth Assessment Report
of the Intergovernmental Panel on
Climate Change, Geneva, pp. 1-34,
DOI: 10.59327/IPCC/AR6-9789291
691647.001.
9
array2023 flux


Flux in the Anthropocene. An overview

Palamara, Jason

Diverse Instrument Symphonic
Ensembles as a Strategy for Sus-
tainable Innovation
By Jason Palamara
If the Anthropocene is characterized
by human-driven ecological trans-
formation, the traditionalWestern-
instrument orchestra was founded
as a musical analog, cutting trees
into violin bodies, animal flesh into
drumheads, and even digging up
copper and nickel to fashion into
flutes. While little research has been
done to evaluate the total ecologi-
cal impact of large musical ensem-
bles (Prado-Guerra et al., 2020), it is
logical that if traditional orchestral
instruments (instruments used and
continually valued over decades or
even centuries) have any detrimen-
tal ecological impact, non-tradi-
tional, electronic instruments such
as computers, speakers, mixers, and
MIDI controllers – each with their
requisite complement of cables and
adaptors – must present an even
more significant detriment to the
environment. Moreover, suppose
the central instrument of the com-
puter music is to remain the com-
puter. In that case, the field must ad-
dress the problems inherent in us-
ing a technology intentionally man-
ufactured to be momentary, expen-
sive, wholly toxic to the environ-
ment, and, worst of all – short-lived
due to purely capitalistic incentives
which drive manufacturers into
planned obsolescence (Aladeojebi,
2013).
The proposed model, a Diverse In-
strument Symphonic Ensemble
(DISEnsemble), is the musical in-
verse, not seeking to change the
landscape but instead seeking in-
creasingly environmentally neutral
solutions to the task/problem of for-
ward-looking musical innovation.
The sustainable focus of a DIS-
Ensemble exists throughout the
"food chain," from repertoire to in-
struments to production; the tech-
nologies must be sustainably
sourced (Trott, Even, and Frame,
2020).
This practice contradicts the general
thrust of traditional technological
musical pursuits, which generally re-
quire the best (not to mention ex-
pensive) cutting-edge technology
powered primarily by lithium batter-
ies and chips composed of similarly
rare earth metals. Sustainably sourc-
ing technological instruments and
equipment solves many problems,
77
array2023 flux
not the least of which is economic.
As with academia writ large, many
large ensemble leaders are un-
doubtedly finding that budgets are
being stretched thin as enrollments
are similarly declining (Fulk, 2023) at
a rapid pace. By contrast, sustain-
ably sourced technology is cheaper
and may save the host institution
money.
Background
Indiana University – Purdue Univer-
sity Indianapolis IUPUI's DISEnsem-
ble was started in 2018 during more
favorable economic conditions to
meet the educational needs of a
fast-growing department. While en-
rollment at IUPUI's Department of
Music and Arts Technology has con-
tinued to grow steadily even
throughout the past three years of
upheaval, the evidence nationwide
is that since the COVID-19 pan-
demic, enrollment at higher educa-
tion institutions has been decreas-
ing rapidly in the U.S. (Fischer, 2023).
With this prognosis in view, the out-
look seems dire for large musical en-
sembles of all varieties, as they are
traditionally the most complicated
and expensive musical endeavors to
support. Technology-focused en-
sembles may suffer even more
tremendous strain, as they require
more significant funding for expen-
sive equipment and, often, lower
enrollment than the traditional en-
sembles. The IUPUI DISEnsemble be-
gan with a mission of investigating
hardware hacking and musical in-
strument development as a method
for innovative music-making, so sus-
tainability was built into the group
from the beginning.
Methodology
Developing a methodology for the
success of a group such as DIS-
Ensemble is a radical and continual
process, much more so for the unini-
tiated. This group has many of the
traditional problems of a more con-
ventional laptop orchestra but also
some of the difficulties of a perform-
ing big-band jazz group (assuming
it grows to the size of IUPUI's DIS-
Ensemble). Some of the issues en-
countered by large orchestras, and
yet more problems one might only
encounter while trying to perform
at televised awards shows like the
Grammys. IUPUI's DISEnsemble has
succeeded for the past five years by
utilizing the following principles:
78
array2023 flux
1. Engage in radical acceptance of
any diverse instrument the student
plays, assuming it is logistically
possible and their skills are
acceptable. Never place the needs
of a chosen repertoire above or
below the technological forces
available at hand.
2. Approach all aspects of this group
less as a music-performing
ensemble and more as a musical
production company, which
employs performing musicians,
sound engineers (mixing live and for
documentation), visual media
designers, lighting designers,
instrument designers, and
personnel who document the
ensemble's activities for later
dissemination. A DISEnsemble is less
of a band or an orchestra and more
of an ecosystem.
3.Wherever possible, acquire
materials needed to pursue
ensemble activities from sustainable
sources. Donations, second-hand
stores, and institutional surplus are
all excellent sources that will
contain practical options for music-
making. For reference on music-
making options from dilapidated or
seemingly obsolete technology, see
Nicholas Collins' book Handmade
Electronic Music (Collins, 2009).
4. Since circuit-bending, hardware-
hacking, and instrument design
takes a very long time to produce
successful results, encourage the
long-term development of
reproducible novel instruments,
which will stay with the group as
students move on and graduate.
Students interested in this type of
work may be shunted into a
separate course (such as IUPUI's
DISEnsemble Content Creator
section) to avoid bogging down the
performing ensemble's rehearsals
with time-consuming (often failed)
experiments. Use this separate
course as a research and
development lab for the main
ensemble's activities.
5. Identify and promote promising
ensemble members who exhibit
natural leadership tendencies. A
group in this manner is too complex
to be run by one individual, which is
why traditional orchestras and wind
ensembles often developed to have
section leaders. Leadership roles can
take many forms in a group with this
much instrumental diversity, but
leaders must be clearly identified
and promoted early. Leadership
tasks and responsibilities must be
delineated and standardized at any
level possible.
79
array2023 flux
6. The group must spend a
significant amount of rehearsal and
performance time improvising,
which aids in the development of
novel instrumental performance
practice.
IUPUI's DISEnsemble (which has a
continual open-door policy regard-
ing technological donations) has of-
ten been gifted a device erro-
neously believed to be unusable
when a simple workaround or hack
was only a Google search away. Sim-
ilarly, much of what ends up at a
university surplus department is
perfectly useable for various musical
situations. Older PCs or outdated
desktop computers can be wiped,
cleaned, optimized, and used as mu-
sical instruments. Past examples of
instruments and equipment suc-
cessfully redeemed by the IUPUI
DISEnsemble include The Apoca-
lypse Piano (a novel method for pi-
ano preparation, electrification, and
amplification using old speakers
and almost-dead batteries), the
Squish controller (a MIDI controller
played with palms and fists), and a
new kind of instrument cable called
a DIS-cable (which facilitates easy
amplification of nearly any RF-emit-
ting device).
Once the DISEnsemble takes owner-
ship of a new item, an exploration of
its musical usefulness usually fol-
lows. Sometimes this process takes
considerable time, leading to sev-
eral devices sitting in a faculty office
until the right opportunity presents
itself. This approach reduces waste
and carbon emissions associated
with new instrument manufacturing
and promotes the reuse of materi-
als, extending their lifespan and
minimizing their environmental im-
pact (Brennan, 2020).
The DISEnsemble / University
Surplus Bartering Pilot Project
Those involved in this musical pur-
suit must continually seek exciting
devices or supplies. It is possible to
take this practice of material scav-
enging too far. Some devices, prod-
ucts, or surplus items are too far
gone to be musically redeemed. For
this reason, the IUPUI DISEnsemble
has recently devised a pilot program
as a bartering partnership with
IUPUI's University Surplus and the
Office of Sustainability, hopefully
serving as a pivotal new component
of DISEnsemble's regular practice.
Under the new initiative, DISEnsem-
ble will no longer seek out musical
80
array2023 flux
and audio equipment for its own
needs but for the needs of the IUPUI
community. Accordingly, DISEnsem-
ble personnel will work with Univer-
sity Surplus staff to identify musical
and audio equipment which is still
useable, and these items will be
placed in a lending library managed
by the University Library Staff and
loanable to any student or faculty
member. In this way, the university
will support the needs of its diverse
students (many of whom are under-
resourced) and cut down on the
technological waste the university
system produces. This initiative will
also establish a sustainable ex-
change system within the university
community, allowing members of
the Music and Arts Technology Fac-
ulty to trade their unused equip-
ment for new, usable equipment
from the University Surplus. If this
method of inter-departmental bar-
tering proves helpful, it should be
rolled out by the IUPUI Office of Sus-
tainability soon. This initiative will
not only make musical instruments
more accessible to artists but will
also reduce waste by repurposing
idle equipment from university in-
ventories (Cooper & Gutowski,
2017).
Future Prospects
Looking ahead, DISEnsemble aims
to expand its reach and impact. The
bartering project outlined above
will hopefully cut down on univer-
sity waste to such a significant de-
gree that it will warrant scaling up
the project's scope to the university
community. By continuing to cham-
pion sustainability and creativity in-
novation in computer music, DIS-
Ensemble seeks to inspire similar ini-
tiatives worldwide, fostering a
global movement towards environ-
mental awareness and action. Dis-
semination of these ideas could be a
good thing for the environment and
a way for computer musicians to
continue innovating sustainably.
References
Aladeojebi, T. K. (2013). “Planned
Obsolescence,” International Journal
of Scientific & Engineering Research,
4(6).
Brennan, M. (2020). “The
Environmental Sustainability of the
Music Industries.” In: Oakley, K.,
Banks, M. (eds), Cultural Industries
and the Environmental Crisis.
Springer, DOI: 10.1007/978-3-030-
49384-4_4.
81
array2023 flux
Collins, N. (2006). Handmade
Electronic Music: The Art of Hardware
Hacking. NewYork, NY: Routledge.
Cooper, D.R. and Gutowski, T.G.
(2017). “The Environmental Impacts
of Reuse: A Review.” Journal of
Industrial Ecology 21: 38-56.
DOI: 10.1111/jiec.12388.
Fischer, K. (2023). “The Shrinking of
Higher Ed,”Chronicle of Higher
Education.www.chronicle.com/
article/the-shrinking-of-higher-
ed (last access Nov 19, 2023).
Fulk, A. B. (2023). ”Is this
Armageddon?,”Chronicle of Higher
Education.www.chronicle.com/
article/is-this-armageddon? (last
access Nov 19, 2023).
Prado-Guerra, A. et al. (2020).
“Environmental impact study of
symphony orchestras and
preparation of a classification guide,”
International Journal of Environmen-
tal Studies 77 (6): 1044-1059, DOI:
10.1080/00207233.2020.1746546.
Titon, J. T. (2009). “Music and
Sustainability: An Ecological
Viewpoint.”TheWorld of Music 51(1):
119–137. www.jstor.org/stable/
41699866 (last access Nov 19, 2023).
Trott, C.D., Even, T.L. and Frame, S.M.
(2020). “Merging the arts and
sciences for collaborative
sustainability action: a methodolo-
gical framework.”Sustain Sci 15:
1067–1085. DOI: 10.1007/
s11625-020-00798-7.
82
array2023 flux


Diverse Instrument Symphonic Ensembles as a Strategy for Sustainable Innovation

Stuart, Amanda

Creating ImaginaryWorlds from
Scientific Data: An Interviewwith
Amanda Stuart
By Scott Deal
British composer and media artist
Amanda Stuart uses narratives of
form and shape, juxtaposing con-
trasting translucent colors with
dense sonic textures, woven to-
gether in an audiovisual tapestry.
From the tiniest of gestures to dra-
matic powerful climaxes, she im-
merses the listener in fantasy land-
scapes creating beautiful and dra-
matic sound worlds. Her piece Song
of the Trees won the IAWM Pauline
Oliveros Prize for Electroacoustic
Media 2015 and Not Missing You
received an Honorary Mention for
the same prize in 2014, with one
reviewer writing, …It’s as beautiful
to look at as it is to hear” (Boston
Classical Review). Her work has been
presented throughout the world in
venues that include Earth Day Art
Model Festival, New York Electroa-
coustic Music Festival, Boston New
Music Initiative, International Com-
puter Music Conference, and Under-
standing Visual Music in Buenos
Aires, to name a few. She has held
residencies for the City of Peterbor-
ough, the Cambridge Festival, Suf-
folk County Council and the Firebird
Trust. Her commissions include
compositions for the London Festi-
val Orchestra and The City of Peter-
borough Symphony Orchestra. The
following is a transcript of an inter-
view that was conducted in Octo-
ber, 2023.
Scott Deal: Amanda, in looking at
your work, it seems that you cover
a wide range of earth-related phe-
nomena. For example, Bubbleflux
explores the notion of bubbles
forming in the ocean, as well as the
functions they carry out in animal
bodies. Magnetosphere draws from
scientific data of electronic activity
in the atmosphere. These two works
seem to capture what you are doing
broadly, and I’m wondering how
would you frame them in the con-
text of your creative process and
artistic intents?
Amanda Stuart: When I first com-
posed Bubbleflux, I was certainly
attracted to the pure joy of bubbles,
but I was also incredulous at how
vital they are to life. For instance, in
medicine they are used to transport
drugs to specific parts of the body.
And it seemed to me that you could
celebrate these aspects of our
world, and not to be too grand, but
26
array2023 flux
through compositions, one could
take people into a different world
and allow them to focus on it. It's
not just another scroll of …bubbles
in this”, or …the magnetosphere is
doing that”, but rather that you are
immersed in it and that gives you
time to really think about what
you're listening to, and to think
about earth. Most importantly, it’s
about treasuring the earth, now
more than ever. With Bubbleflux, I
was attracted to the idea of creating
a video graphic score that could be
understood by non-musicians, so
we could have audience members
on stage performing spontaneously.
During the piece, the audience joins
in making popping sounds with
their mouths and there is great fun
and laughter. But they’ve also been
reading the program notes and
getting a bit more information.
Hopefully, this makes them think
a bit differently about things than
they perhaps would otherwise. And
with Magnetosphere, I think of how
incredible it was to work with the
real sounds from the atmosphere,
and to create a work that transforms
the notion from those sounds into a
whole creative world, which is quite
a different experience from under-
standing a scientific report, for in-
stance.
SD: Where did the recordings for
Magnetosphere come from?
AS: There was a call for works from
CARISMA, a project funded by the
Canadian Space Agency, using
recorded sounds they had captured
from the magnetosphere of the
earth. In the call they made clear
they were interested in music to
accompany the sounds. But I was
interested in using just the sounds
themselves, so while I knew my
work most likely would not be
selected, I thought wow, these
sounds are so amazing, what a fan-
tastic opportunity! I didn’t really
write to their brief, but I used the
material for what I wanted, which
was to take these tiny sounds and
make an incredibly imaginative
world by transforming them, giving
the agency publicity as it is cited
every time the work is presented.
SD: I’m reading on your website
about the various sounds they sup-
plied….Sferics, tweeks, whistlers,
dawn choruses and hisses. Seems like
a big variety of sounds. Perhaps you
can talk about your process. How
did you take these various sounds
and term them into a media piece?
27
array2023 flux
AS: If I remember correctly there
were four types of sounds recorded
in the magnetosphere that you
could use. The sounds were interest-
ing but limited in number – but I
like that. I wanted the piece to liter-
ally feel like you were in space and
that things were coming at you. This
is also why I chose to make it in sur-
round sound, so that you had that
real sense of being up in the atmos-
phere. After choosing sounds, I go
through every sort of transforma-
tion I can think of, using any weird
bit of software that I have that
might do something. This takes a
long time, because I’m following
my sense of what's right for the
piece and what's not. If it's too
electronic, it doesn’t work because
I want it to feel like it is in the sound
world. In essence, the first step
is just to go through all different
creative processes that I can to get
a sort of library for the piece. And
then while I’m doing that, I'm start-
ing to imagine a structure. It doesn't
have to be an obvious structure, but
it does have a sense of journey. I
think, oh, well, I want it to start re-
ally quiet. and I'm always going to
have soft bits and dramatic bits
because that's what my pieces are.
They'll take you on journeys, but
there'll be these wild bits in the mid-
dle, or it might start with a wild bit
or whatever it might be, but it's go-
ing to be big and powerful and then
completely tranquil and silent in
other bits. The sounds must have a
reason for them being there, so
there is a sense that they belong in
that place because they're part of
that storytelling, which is very im-
portant to me. Structure will create
emotional movement in the piece in
the sense that I want the listener to
respond and to be immersed in it.
Life itself is small and joyful and big
and powerful….all these things.
And somehow, I want these in all of
my pieces.
SONGOF THE TREES
SD: Moving to Song of the Trees, can
you discuss your inspiration and
process for its creation?
AS: I went to see David Hockney’s
big exhibition at the Royal College
of Art London. Walking into the ex-
hibit, there were some massive tree
paintings from his Woldgate series.
For some reason I just looked at
them and strongly sensed the struc-
ture of a musical piece. The view
was a huge tunnel of trees to walk
through which struck me as both
28
array2023 flux
imposing and peaceful. I just knew
then I wanted it to be like that, to be
peaceful and to be very dramatic in
the middle and then be peaceful
again. I also wanted the instruments
to be somehow connected with the
trees. As I was imagining these trees,
I wondered, what would they sound
like? I settled on clarinet and flute
and wrote the instrumental score
first as a duet. I wanted it to have a
sense of wind, so that the two
melodic lines would float above the
wind. The wind would come and go,
giving the listener a sense of space
and being surrounded by the
sounds. I created wind sounds by
recording an associate blowing air
into a bassoon and did the same for
the clarinet, and then processed
those sounds.
I had the wind in the trees and the
two melodic lines from the clarinet
and flute, which I decided to trans-
form in live performance. The piece
begins quite simply and the trans-
formations are quite simple, but in
the middle section there is this
whole whirly section where the two
instruments are processed live, and
it's just wild. An audience member
described it as like somebody bull
dozing down the forest in the mid-
dle of the piece, just an explosion
of sounds, all created live. At the
end you come back into a trans-
formed version of the original
melody, a return to an altered be-
ginning. Then when I did the film,
I drew from hundreds of photos I'd
taken in the village over the last few
years. I selected pictures and started
to experiment with them and real-
ized I could create a sensation flying
over the trees. It looks like you're in
a drone, but actually all the trees
aren’t moving. It’s uncanny that the
photos are all taken from the
ground, but in the film it looks like
you're flying over the trees, and
through them. I layered the images
and moved them in the software,
but it was all really tough because
I didn’t realize that I could do that
sort of thing. It was all very experi-
mental, and I was creating this all
the while I was learning how to use
the software (laughter). But it's like,
gosh, that's the joy of creating, isn't
it? You don’t know…it's like, wow,
what if I could do that! (more laugh-
ter).
SD: Oceania is derived from a two-
minute sample of Ross Seal and
Orca Whale calls. Could you discuss
a bit about your process in creating
the audio for the work, and if you
want to get a little technical, which
29
array2023 flux
software, what processes you liked,
or that you felt worked well?
AS: I was delighted to be chosen to
take part in the Polar Sounds
project, which was a collaboration
between Cities and Memory, the
Helmholtz Institute for Functional
Marine Biodiversity, and the Alfred
Wegener Institute at the Helmholtz
Centre for Polar and Marine
Research. The aim of the project was
to highlight the soundscapes of the
Antarctica and the importance of
sound communication between the
animals and how it is affected by
climate change. From what was
offered, I chose the sample of the
recording of Ross Seals and Killer
Whales because although they’re
very repetitive, two different sounds
in all, I thought I could create the
whole piece from just this one sam-
ple. Then I literally went through ev-
ery single processing software app I
have; Everything in Ableton, Max for
Live, Native Instruments, UAD and
IRCAM GRM Plug-ins. I put the single
sample through everything, which
produced around 83 sounds that I
liked and then I cut that down to 61
sounds which were then used in the
piece.
The crazy thing is that I had all these
sounds and you know, I'm starting
to get overwhelmed very quickly.
Then I divided them up into cate-
gories, like, oh, that sounds like a
washing machine. Okay, I'm gonna
have washing machine sounds, and
these are low sounds, swooping
sounds, etc. I knew I didn’t want
anything to sound electronic, be-
cause I was creating an imaginary
world underwater. I wanted the
sound to be organic and natural, of
course they weren't, but I wanted
them to feel as if you really would
hear those sounds underneath the
water. You hear the original record-
ing twice in the piece - most clearly
at one minute 15 seconds from the
start of the piece. It was very impor-
tant to me that you hear what the
whole piece was made from. The
idea is that you would start in the
depths of water. It’s an imaginary
underwater world, and you don’t re-
ally know where you are. It feels fa-
miliar but completely unfamiliar and
again, sounds are all moving so you
get sense of the whale, and it's mov-
ing further away or coming near,
and it’s swimming up to you but
you’re not certain as to what it is.
And there's a big storm after which,
you just like gently float away. So
that's how the audio of the piece
came together.
30
array2023 flux
SD: And how did you integrate the
audio part with the video?
AS: Having created the piece, I
thought it would be great to put
some pictures to it. I began
researching media of the Antarctic,
which is fascinating look at, seeing
waterfalls in the ocean and all these
things, and to see where these ani-
mals lived. But I couldn't create visu-
als that worked in the context of
Ross Seals and Killer Whales. I
returned to the idea I began with
the audio component, of just creat-
ing my own world…. the imagined
world under water. And then
I started experimenting with some
of my digital paintings and tried
transforming them and suddenly I
realized it was going to work! I used
about 13 of my paintings of various
media, digital and acrylic. They all
had been experimental paintings,
but then when I layered, trans-
formed and animated the images,
they came together into this imagi-
nary world in my mind, and that's
how the visuals came together and
were synced with the sounds.
SD: Amanda, it’s exciting to hear you
talk about your process, and I appre-
ciate how intuitively it is that you
work. Thank you for taking time to
talk with us today.
AS: My pleasure!
Music
Amanda Stuart: Oceana
World Premiere: Earth Day Art
Model Festival IUPUI Indianapolis -
April 22, 2023
http://mediathek.slub-dresden.de/
vid90003603.html
Amanda Stuart: Song of Trees
Clarinet: Gareth Stuart
Electroacoustic World Premiere:
Light and Sharpness Concert 2013,
Mumford Theatre, ARU, Cambridge;
Film Premiere: Earth Day Art Model
Festival 2022, Indiana University Pur-
due University Indianopolis (IUPUI).
http://mediathek.slub-dresden.de/
vid90003604.html
Amanda Stuart: MAGNETOSPHERE -
Sounds of the Earth
Electroacoustic World Premiere:
New York City Electroacoustic Music
Festival - 2017; Film World Premiere:
Earth Day Art Model Festival 2021,
IUPUI.
http://mediathek.slub-dresden.de/
vid90003605.html
31
array2023 flux


Creating Imaginary Worlds from Scientific Data: An Interview with Amanda Stuart

von Coler, Henrik

array2022
Chaos in the Garden. Human-
assisted AI Composition in Experi-
mental Spatial Music
By Henrik von Coler
AI and Spatialization
Applications of Artificial Intelligence
(AI) have been manifold in the past
decades. However, since the launch
of the ChatGPT1 chatbot in Novem-
ber 2022, the use of AI-based tech-
nologies has been growing expo-
nentially in any field of human pro-
ductivity and creativity. Our society
is currently debating the future of
such technologies, regarding the
effects they might have on the
structure of our working world and
private lives.
There is consensus that these sys-
tems can and will perform repetitive
and automatable tasks, even of high
complexity. How and whether they
can – or rather should – be used in
creative processes is more contro-
versial. This has already caused is-
sues with copyright and intellectual
property for AI-generated images
(Hristov, 2016).
Large Language Models (LLMs) such
as ChatGPT can be integrated into
the conceptualization phase in any
artistic discipline, by answering
space
prompts and thus influencing deci-
sions in the design process. This arti-
cle illustrates and discusses how this
tool can be used to create a text-
based score for experimental spatial
music. Related work in AI-based
composition and algorithmic meth-
ods for spatial audio practices will
be presented in the introduction.
Afterwards, the chat and the full
score are presented. This is followed
by an analysis and a report on a first
performance based on the composi-
tion.
AI and ML in Music Creation
The use of tools and auxiliary agents
in music composition is older than
computers or mechanical devices.
Dice were used to introduce chance
into classical compositions (Hedges,
1978; Ruttkay, 1997). This can be
considered an early example of
technology-aided or algorithmic
composition.
In computer music, algorithmic
composition and computer-aided
composition have been an integral
part of its history (Supper, 2001).
Early examples of computer-aided
composition produced symbolic
scores, based on algorithms and
heuristics. These include Lejaren
Hiller’s work from 1955, Moorer’s
33
array2022
rule-based systems and Xenakis’ap-
proach, who considered algorithmic
composition a ‘tool for constructing
complex structures with minimum
means’ (Roads, 1980). Lopez-Rincon
et al. (2018) present a taxonomy of
AI based music composition, divid-
ing the field into soft computing
based music composition methods
and symbolic AI based music compo-
sition methods, both with additional
subareas.
With increasing capabilities, artificial
intelligence and machine learning
(ML) can be incorporated into music
composition and performance at
many levels, including not only
score generation, but also interac-
tive performance and improvisation
systems or complete sound and mu-
sic synthesis methods.
Modern generative music systems,
such as WaveNet (Oord et al., 2016)
and OpenAI’s Jukebox (Dhariwal et
al., 2020) create complete musical
pieces in the audio domain, in the
style of artists included in the train-
ing data. Noise2Music (Huang et al.,
2023) is a diffusion model, that gen-
erates audio from text prompts, sim-
ilar to the well-known diffusion
models for image generation.
Composition is not the only applica-
tion scenario for AI and machine
space
learning in music. The integration of
ML algorithms into human-com-
puter interfaces for the control of
digital musical instruments leads to
a form of human-machine partner-
ship (Fiebrink, 2017).
Algorithmic Spatialization
In the context of electroacoustic
music, spatialization refers to the
dynamic distribution of sound in
composition or performance. Often
this involves algorithms for control-
ling the spatial distribution, or more
precisely the virtual positions of
multiple sound sources in object-
based spatialization (Schacher,
2018). Such algorithms can provide
meta-parameters and semi-auto-
mated workflows, needed to control
the many parameters involved in
these processes.
Tools like OM-Spat (Garcia et al.,
2015) and OM-Prisma (Schumacher
and Bresson, 2010) allow algorith-
mic control over spatial audio pro-
cesses in offline computer-aided
composition. Applications also in-
clude spatial sound synthesis, based
on algorithmically generated con-
trol trajectories.
Spatialization algorithms can also
be derived from natural phenom-
ena. For example, swarm behavior
34
array2022
has been used for various musical
parameters in compositions (Kali-
akatsos-Papakostas et al., 2020).
The inherent spatial qualities of
swarm motion predestine it for
spatial aspects, as for example in
swarm-based spatial granular
synthesis (Wilson, 2008). A similar
example uses Boids algorithm for
spectral and granular spatialization
(Kim-Boyle, 2006).
Other examples have more prag-
matic applications. In a self-super-
vised framework for audio spatial-
ization Lu et al., 2019, video data is
used to generate 3D spatial audio
from monophonic recordings. This
procedure can be used to create im-
mersive audio from simple record-
ings.
Spatial Performance Concepts
The performance of experimental
music on spatial sound reproduc-
tion systems opens up a wealth of
possibilities. Especially in the case of
live electronics, the usual musical
grid of time and pitch is not applica-
ble. Complex synthesis systems,
such as modular synthesizers or
custom digital musical instruments
(DMI) offer a wide range of parame-
ters that cannot be expressed in
musical notation. Additional spatial
space
parameters for each instrument re-
quire further structuring to allow for
intentional control. Concepts for a
guided improvisation are a means
for adding this structure. Such con-
cepts can be text-based instruc-
tions, graphical notations, or any
type of conceptual guideline.
The Electronic Orchestra Charlotten-
burg (EOC)2 is an ensemble that
focuses on live electronic music for
large loudspeaker systems. It has
explored various approaches to
guided improvisation, some of
which include explicit treatment of
the spatial aspect. These include
their own compositions, adapta-
tions of existing pieces and and
commissioned works.
Mark Applebaum’s S-Tog (Apple-
baum, 1996), for example, is a
graphical score, based on the train
map of Copenhagen. Musicians
travel fictitious routes on the map,
realizing playing instructions that
are connected to train stations on
the map. For the EOC adaptation,
automatic source movements were
programmed, according to the mu-
sicians’ routes on the map.
For Robert Stokowy’s Coffee Rings
(Stokowy and von Coler 2018; p. 34),
the virtual source positions of each
musician are controlled by signal
35
array2022
analysis. The detected pitches of the
instruments determine the azimuth
angle of a virtual sound source on a
fixed radius. Thus, the source distri-
bution is modulated according to
the pitch distribution of all instru-
ments.
space
planned but happened during the
chat on April 14, 2023:
Chaos in the Garden3
Concept
The goal of this experiment is to
have the AI chatbot create a score
for five arbitrary instruments, includ-
ing instructions for the spatializa-
tion. Since the score is intended to
be as open as possible, no tonal or
instrument-specific instructions
should be included. The system was
instructed to remove or replace
such content and asked to refine
certain aspects, until a useful ver-
sion was reached. Although the
process is influenced by all prompts,
the aesthetic concept was intention-
ally left to the chatbot as much as
possible.
The Chat
The following text is the original
conversation with the Large Lan-
guage Model trained by OpenAI,
based on the GPT-3.5 architecture.
None of the steps were previously
36
array2022 space
37
array2022 space
38
array2022 space
ICMA awards
By PerMagnus Lindborg and Christo-
pher Haworth
39
array2022 space
ICMA awards
40
array2022
ICMA awards
By PerMagnus Lindborg and Christo-
pher Haworth
space
placed throughout the space’. The
system was then prompted to refine
these passages, resulting in the fol-
lowing list of spatial instructions for
the sound director:
The above chat was followed by an
attempt to let the chatbot create an
animated graphical score. Although
it delivered short clips using Python
and the pygame library4, it did not
create a useful result.
Analysis
It took nine prompts from the hu-
man composer to reach the final
score. With more precise user input,
it could have been less. The struc-
ture of the piece, as well as the
topos, were not influenced by
prompts, as far as possible. How-
ever, the basic concept of an experi-
mental spatial music score for un-
specified instruments was kept as
the boundary condition.
The first version of the score with
spatial instructions had a frequent
use of the vague expression ’sounds
1. panning from left to right and
front to back
2. panning from front to back and
moving around the space in a
random pattern
3. sounds placed in different areas
of the space and moving
around in a circular pattern (2
occurrences)
4. sounds placed in different areas
of the space and moving
around in a random pattern (2
occurrences)
5. sounds placed in different areas
of the space and moving
around in a random, chaotic
pattern
6. panning from front to back and
moving around the space in a
circular pattern
7. sounds swirling around the
space in different directions
8. sounds placed in different areas
of the space and moving
around in a random, unpre-
dictable pattern
9. sounds placed in different areas
of the space and moving
41
array2022
around in a circular or random
pattern (3 occurrences)
10. sounds placed in different areas
of the space and moving
around in a random, gentle pat-
tern
11. panning from right to left and
moving around the space in a
random, celebratory pattern
12. sounds placed in different areas
of the space and moving
around in a random, energetic
pattern
The instructions listed above corre-
spond to typical methods used in
spatial music practice. Circular, ran-
dom and chaotic patterns are often
used for source movements, includ-
ing performances by the EOC.
Although attributes such as swirling,
unpredictable, gentle,
energetic and celebratory are rather
vague, they are interpretable and
conclusive. Some instructions even
include specific directions, like from
left to right and front to back.
In most cases, the spatial instruc-
tions are coherent with the overall
character of the sections and the in-
structions for the instrumental per-
formers. Hectic movements such as
swirling, unpredictable and chaotic
appear in the more dramatic Sec-
space
tions 2 and 3, combined with disso-
nant sounds, aggressive sounds and
irregular rhythms. The releasing ten-
sion of Section 4 is supported by
gentle patterns, while the uplifting
character of Section 5 is emphasized
by celebratory and energetic move-
ments.
Rehearsal and Performance
The composition was premiered on
May 22 2023 by the Electronic Or-
chestra Charlottenburg (EOC)2 as
part of the Intersonanzen festival5 in
Potsdam, Germany. During re-
hearsal and performance, the en-
semble’s main goal was to do do jus-
tice to the AI’s ’compositional inten-
tions’, in order to bring out the arc of
tension and the individual character
of the five sections.
Since some aspects of the score are
left open to a certain extent, the en-
semble made the according deci-
sions. Each phrase in the sections’
instructions was assigned to one or
more of the four musicians on stage.
The sound director used a combina-
tion of Python6, Supercollider7 and
hardware input devices to imple-
ment the spatial instructions during
the performance.
42
array2022
Discussion
Chaos in the Garden is the result of
using an AI-chatbot as a tool for cre-
ating an experimental score. Re-
gardless of the instruments used in
a performance, any interpretation
of the piece can be considered ‘com-
puter music,’since it would not have
been possible without the help of a
specific software.
The cornerstones of the concept
were specified by the user, whereas
the overarching structure and the
topos have been chosen by the sys-
tem. Being trained on existing texts
and data found in the internet, the
question arises whether the compo-
sition is original or a combination of
related material and publications.
Even this simple example raises the
question of authorship and copy-
right. A subsequent Google search
found several pieces of music with
the same title from different musical
genres, none of them related to ex-
perimental spatial music.
Bown (2021) presents a model for
human-computer interaction in cre-
ative processes. It starts with a hu-
man intention and then enters an
iterative cycle of machine output
and user evaluation, until the de-
sired result is achieved. The creation
space
of Chaos in the Garden follows a sim-
ilar pattern. However, ideally the
machine does not assist the com-
poser but the composer assists the
machine in reaching a form that can
be used as a score. In this case, the
abort criterion is in the hands of the
user, who decides whether to con-
tinue altering the score through
additional prompts. With further
prompts, the instructions might
become even more specific. This,
however, would further bias the
system and it would lose it’s claim
on authorship with every step.
The creative process involves direct
guidance, assistance or input of hu-
man beings (Hristov, 2016) and can
not be considered an autonomously
generated AI creation. Considering
the concept of CREATIVITY by
Moruzzi (2018), the system did not
act autonomously in the reception,
selection or elaboration of stimuli.
Hence, it did not create a musical
work, but was merely used as a tool
in a creative process. According to
ChatGPT 4.0, however, the ’interac-
tion in creating this piece can be con-
sidered a collaboration’. Ultimately,
Chaos in the Garden is considered a
human-assisted AI composition.
43
array2022
Notes
[1] https://chat.openai.com
[2] https://eo-charlottenburg.de
[3] Project website with recordings
and further information https://eo-
charlottenburg.de/repertoire/chaos
_in_the_garden/
[4] https://www.pygame.org
[5] https://www.potsdam-museum.
de/veranstaltung/intersonanzen
-2023
[6] https://www.python.org/
[7] https://supercollider.github.io/
References
Applebaum, M. (1996). S-Tog. http://
www.markapplebaum.com/
stog.html (last access 27 July 2023).
Bown, O. (2021). “Sociocultural and
design perspectives on AI-based
music production: why do we make
music and what changes if ai makes
it for us?” In: Handbook of Artificial
Intelligence for Music: Foundations,
Advanced Approaches, and Develop-
ments for Creativity: 1–20.
Dhariwal, P. et al. (2020). “Jukebox: A
generative model for music”. arXiv
preprint, arXiv:2005.00341.
Fiebrink, R. (2017). “Machine
learning as meta-instrument:
Human-machine partnerships
shaping expressive instrumental
space
creation”. Musical Instruments in the
21stCentury: Identities, Configurations,
Practices: 137–151.
Garcia, J. et al. (2015). “Tools and
applications for interactive-
algorithmic control of sound
spatialization in OpenMusic”.
inSONIC2015, Aesthetics of Spatial
Audio in Sound, Music and Sound Art.
Hedges, S. A. (1978). “Dice music in
the eighteenth century”. Music &
Letters 59(2): 180– 187.
Hristov, K. (2016). “Artificial
intelligence and the copyright
dilemma”. Idea 57: 431.
Huang, Q. et al. (2023). “Noise2music:
Text-conditioned music generation
with diffusion models”. arXiv
preprint arXiv:2302.03917.
Kaliakatsos-Papakostas, M. et al.
(2020). “Artificial intelligence
methods for music generation: a
review and future perspectives”.
Nature-Inspired Computation and
Swarm Intelligence: 217–245.
Kim-Boyle, D. (2006). “Spectral and
granular spatialization with boids”.
Proceedings of the International
Computer Music Conference ICMC.
Lopez-Rincon, O. et al. (2018).
“Algorithmic music composition
based on artificial intelligence: A
44
array2022
survey”. International Conference on
Electronics, Communications and
Computers: 187–193.
Lu, Y.-D. et al. (2019). “Self-super-
vised audio spatialization with
correspondence classifier”. IEEE
International Conference on Image
Processing ICIP: 3347–3351.
Moruzzi, c. (2018). “Creative AI:
Music composition programs as an
extension of the composer’s mind”.
Philosophy and Theory of Artificial
Intelligence 2017: 69–72.
van den Oord, A. et al. (2016).
“Wavenet: A generative model for
raw audio”. arXiv preprint
arXiv:1609.03499.
Roads, C. (1980). “Artificial intelli-
gence and music”. Computer Music
Journal 4(2): 13–25.
Ruttkay, Z. (1997). “Composing
Mozart variations with dice”.
Teaching Statistics 19(1): 18–19.
Schacher, J. (2018). “Algorithmic
Spatialization”. In: The Oxford
Handbook of Algorithmic Music.
edited by R. T. Dean and A. McLean.
Oxford University Press. doi:
10.5281/zenodo.3921793.
Schumacher, M. and J. Bresson
(2010). “Compositional control of
periphonic sound spatialization”.
space
2nd International Symposium on
Ambisonics and Spherical Acoustics.
Stokowy, R. and H. von Coler (2018).
Coffee Rings (EOC Version). https://
eo-charlottenburg.de/repertoire/
coffee_rings/ (last access 27 May
2023).
Supper, M. (2001). “A few remarks on
algorithmic composition”. Computer
Music Journal 25(1): 48–53.
Wilson, S. (2008). “Spatial Swarm
Granulation”. Proceedings of the
International Computer Music
Conference ICMC. Belfast, Ireland.
45


Chaos in the Garden. Human-assisted AI Composition in Experimental Spatial Music

Zalles Ballivian, Gabriel

Six Seasons: Composition Inspired by Ocean Sounds from the Arctic

Hagan, Kerry

array2022
Spatial Audio in Online Presenta-
tions
By Kerry Hagan
The pandemic forced conferences to
evaluate hybrid and online alterna-
tives for academic and artistic pre-
sentations. Although many confer-
ence activities adapted quickly to
online formats, e.g., paper presenta-
tions, some elements suffered more
severe losses when moved online. In
particular, online formats signifi-
cantly impacted spatial music con-
certs.
Considering binaural recordings are
currently the only method for repro-
ducing spatial music in two chan-
nels, it made sense that conferences
for spatial music relied on delivering
binaural audio in various ways.
Previous to ICMC 2022, I attended
several online concerts that tried to
provide spatial audio. In some cases,
artists created VR simulations of
their works. VR was especially inter-
esting for spatial compositions with
visual elements, but it was a convo-
luted method for creating binaural
recordings for audio-only pieces.
Additionally, composers without ac-
cess to VR engines could not
present with this method.
space
In other cases, the conferences re-
lied on the artists to provide two-
channel recordings of their work,
and the organizers hosted those
recordings. Several participants pro-
vided their works binaurally instead
of mixing down to stereo. Given that
most electronic and computer mu-
sic audiences would attend online
concerts with headphones, the re-
sults were relatively acceptable. The
onus to encode binaural recordings
fell on the composers.
In the case of the Klingt Gut festival
of spatial music, the organizers ac-
quired the multi-channel works
from the composers and used a
dummy head for binaural capture in
their sound dome. This stream led to
a more realistic experience, partly
because the dummy head picked up
the environment. Adding room
noise may be counterintuitive since
composers often try to reduce the
physical space to create a virtual
space in their work. But for an audi-
ence member, the experience felt
more live, present, and engaging.
Due to this experience, ICMC 2022
organizers chose to stream audio
from a dummy head for its spatial
audio concerts. The feedback from
5
array2022
online delegates matched the expe-
rience of Klingt Gut. One audience
member also mentioned that there
was more intimacy because we po-
sitioned the dummy head inside the
audience. Therefore, the stream con-
tained environmental noises and
applause from the audience’s per-
spective, giving the online delegate
a realistic concert experience.
The trade-offs of these different
ways of presenting spatial audio on-
line may be apparent. If the host re-
quires composers to mix their bin-
aural versions, some artists may find
this restrictive depending on their
software, hardware and laboratory
access. Also, composers with more
knowledge of virtual systems may
find it easier than early career re-
searchers or composers who work
entirely in audio-only media.
The dummy head solution requires
nothing more than what a com-
poser would supply in any other
multi-channel presentation.
However, despite the positives of
audience integration, composers
may wish to have more control over
the mix than the dummy head pro-
vides. Additionally, good dummy
head microphones can be prohibi-
tively expensive for concert or
space
conference organizers.
There is one more factor that none
of these solutions address. Spatial
concerts delivered online also face
the same problem as regular online
concerts: streaming audio requires
compression, and most streaming
services compress audio as MP3 or
comparable. Although some online
platforms may enable ‘original au-
dio’, other common platforms com-
press audio significantly. When com-
bined with the simulation HRTF that
binaural audio employs, compres-
sion can hurt spatial images. Many
composers are vocal in their disap-
pointment with compression
through standard channels.
Ultimately, conferences and festivals
attempt to provide the best approxi-
mation of the in-person experience.
Audiences, composers and organiz-
ers seem to have different needs re-
garding online spatial audio con-
certs, and there currently isn’t an
ideal solution that satisfies all stake-
holders.
ICMC 2022 chose the dummy head
solution. The compression that
came from streaming didn’t impact
too severely, though there were
some instances where the format
was not ideal. Though the pandemic
is slowly ending, permitting more
6
array2022 space
travel, other issues, such as climate
change, motivate organizers to con-
tinue employing online or hybrid
options for delegates. It stands to
reason that future research should
address the composer and audience
experience and the technologies for
streaming audio, especially for elec-
tronic and computer music and spa-
tial audio.
7


Not a member yet