Locus Diffuse is a networked multi-user instrument populated by a simulated slime mold and four human players. Mimicking the biological behaviour of slime mold and establishing a virtual living network between player nodes, the system sonifies interaction along these connections. Participants use a browser based interface to play the multi-user instrument, and access an accompanying stream for audio and visual output of the system. Locus Diffuse was initially developed as an in-person piece which would track physical location in space to control virtual placement of food deposits for the system’s agents, but pivoted to an network based interface due to COVID-19 restrictions.

System Overview

The project incorporates various approaches from Sonic Ecosystems, Multi-user Instruments, and Networked Music, with sonification of the resulting stigmergy and networked connectivity between players and agents at the forefront of design intentions. The central sonificiation of the system is only achieved through player (human) interaction with the simulated slime mold agents permeating the virtual environment, and the agents themselves cannot survive without the symbiotic presence of players within the space. Control is not centralized to one participant, nor surrendered to the simulated organism. Players are visualized as the named nodes and colour ranging from purple to grey depicts their current energy level. When players run out of energy, they can no longer feed the agents. Players regain energy through close proximity to each other (visible by the established lines between).

The simulation is contained within Max MSP, composed of a JS script directing agent control and control data for grain sonification, JWeb in Max for visual feedback in an HTML page, and audio synthesis control patches additionally held within Max. Online interaction is conducted through MIRA Web and live audio & video streaming was hosted on Mixer. Multiple HTML canvases are employed to keep track of the various data for the environment (one for each component of the simulation): agents, players, and trails.

Simulated Agents

Locus Diffuse’s simulated slime mold is inspired and informed through the biological behaviour of Physarum polycephalum, but does not represent a purely scientific model of the organism. Through various approximations and artistic liberties for the betterment of aesthetic outcomes, the project presents a simulation which successfully follows behaviours found in the physical organism and applies them to a dynamic multi-user instrument sonic ecosystem resulting from the generation of optimized network paths between each player.

Agents spawn in the very centre of the simulated environment and are given a random starting vector. The seed number of agents (default 500) results in a fairly evenly dispersed ring outwards from the centre. Each agent has a centre mass which is equipped with two extraneous sensors positioned at an angular offset and set distance ahead of the agent’s direction of travel. Sampling of the environment is conducted through these sensors in order to determine upcoming actions for each agent, allowing them to turn and face food once it senses a deposit close to itself.

Optimization of the agent network is achieved through a decaying chemoattractant trail deposited and sensed by each agent. Trails are deposited when an agent senses food, or senses another agent’s trail. This results in cascading trails and paths towards food deposits, following the flow of the cytoplasmic streaming. Each agent mimics this cytoplasmic streaming behaviour of a slime mold, and represents a theoretical main concentration node of this cytoplasm. A biological slime mold does not have these nodes; they are used within this simulation to approximate the flow and trajectory of such cytoplasmic streaming and searching behaviour. Agents representing concentrations of cytoplasm within the simulated slime mold body are able to traverse beyond the “edge” of the emergent slime body, seemingly alone in their search. These represent the searching tubules dispersed by the slime mold (pseudopodia), which probe the surrounding environment for nutrients. When a network is established between nodes and attempts to optimize, these biological pseudopodia retract and divert their cytoplasm along main arteries. Agents are autonomous in their understanding of the environment – no information is shared between them other than through their environmental deposits.

Like the biological drive of the slime mold, the simulated agents are in search of energy to keep moving and find more food. Energy is a value held by each agent and player attractant node which directly maps to qualities of each granular sonification. Energy also maps to motor qualities of the agents – faster movement rotation with higher energy values. Agents passively lose energy on movement and gain energy from being above/on a player location, with the chance to take a “bite” – gaining energy and additionally triggering sonification.

Sonification

The aural aesthetic goals for the project come from attempting to generate an environmental texture defined spatially by the positions of each player, by giving all players a unique voice to ensure a personally recognizable timbre. These voices are established with individual granular synthesis modules, which receive the trigger messages when an agent gains energy from a player via eating.

These trigger messages contain the energy value of the agent, and the target player being eaten from. Energy is mapped to the pitch of the fired grain, where low energy agents result in lower frequency grains, and vice versa.

Player movement also is sonified – generating beating high frequency tones from two sinusoidal oscillators within unique ranges per player. Higher movement speed is mapped to higher frequencies and lower volume, while slow movement will result in lower frequencies and more pronounced volume. This gives a sense of direct action to the players and signals a shift in the structure of the agent network due to player position changing.

All audio is spatialized to player positions using dispersion.spat – meaning all activity happening on the right side of the simulation will be weighed towards the right side of the sonic space and sound that is generated by a player, follows that player in space. This also includes verticality which can be chosen on the web interface. Player verticality is only represented spatially within the sonification of the system, and has no result on the behaviour of agent movement.

Networked Interaction & Visualization

Wide accessibility of access and control was a key consideration for the project due to its pivot to a telematic piece, accommodating different devices and network connections was an essential goal in order to allow a diverse participation and audience population. Control of player movement occurs in the browser through a provided URL, allowing access to a Max Mira interface from beyond the local network the patch is running on.

The interface contains a panel for each player consisting of identical controls. The centre black square is a mira.multitouch object for position input, and a right hand slider is for vertical movement. Mira frames are capable of mouse or touch response, so the interface may be also used on a touch enabled device.

Visual output of the system was hosted on a public live stream. This situated the stream as a centralized audio and visual hub for the experience of the instrument, with a spectroscope of each player’s respective sound on the left, and height of their position on the right.

States

Various states were established to result in distinct behaviours which altered agent trail decay, sensory distance, death threshold, birth odds, and energy decay of agents. Here are visual examples of each of these states and their qualities:

Survey and Player Responses

Play sessions were held in two waves as open calls on set dates, and public exhibitions following the weekly electro-acoustic improvisation series DisPerSion Relation X, held on Zoom. Sessions lasted roughly 25 minutes (with some lasting up to 60 minutes)  and allowed players time to explore various interactions. These were presented in two waves, where the first wave of play sessions focused on a single behaviour state of the agents, the second wave presented players with the four varied states.

1. What was your sense of playing in this virtual environment?
2. What was your sense of connection to the others in the virtual space? (Other players or agents)
3. How did you perceive your own “voice” while playing? (Location, timbre, relation to environment and others)
4. How would you describe your ability (or lack of) to perform expressive musical action?

Responses were recorded anonymously on Google Forms and questions were not shown to participants prior to playing. As Wave 2 was centered around four different behavior states, the response form was updated to accommodate this change. These questions were aimed at attempting to get a sense of their perception of the system in terms of relations with other players and the resulting sonification. Players were not primed on the behaviour of each of these states but the transition between each was announced to prompt the players that they will be interacting with a new behavior. States moved sequentially through 1-4, but in free play after experiencing each, players could request to revisit certain states

For wave 2 responses, questions from wave 1 were all asked again, including “For each state:” before a given question. One additional question was asked:

• How would you describe the behaviours of each state? (changes in response, characteristics, etc)

Key Observations

• The perception of a sonic ecosystem was tied to variation in sound sources intersected with agent behaviour
• Narrative-based personal connection between players and agents mediated interaction characteristics
• There was a bias towards a visual understanding of the system vs an aural one
• The “locus” of experiences of connection were more varied, ranging from a lack of systemic connection, focus on inter-human collaboration, to human-agent collaborative sounding

Publication

Hoy R., D. Van Nort, “Locus Diffuse: An Agent-Based Sonic Ecosystem for Collaborative Musical Play”, in Proc. of the 15th International Symposium on Computer Music Multidisciplinary Research (CMMR), 2021. [PDF]

---