Ensemble Resonanz
2020/21
35'
Video of the Digital Version:
Video of the Live Version:
Versions:
Two versions of the piece exist: The original staged version (premiered in Bonn October 2020, revised Dresden 2023) and a digital only streamed version (premiered in February 2021).
Program Notes:
„Convergence“ uses the concept of Artificial Intelligence to learn features of human musicians and then recreate new entities based on these recordings. In the piece the players interact with their generated counter-parts. They see theirselves transform and reshape. The technology used is centered around Auto-Encoders (and GANs). Metaphorically they demonstrate a world that is constructed and parametric. The friction between machine perception and human world perception is the starting point for questions that address the fluidity of the self and the restrictions of perception.
Human world and self models are parametric systems that make abstract assumptions and classifications of our surroundings. These processes happen party subconscious and unreflected. They give us the impression of an absolute truth or reality, as the constructed concepts, identities and beliefs are persuasive and internal. That these constructive models are fluid and subject to change is examined through the use of AI in this context. Auto-Encoders allow a formalization of the input data – in this case faces, bodies and voices. The deep learning yields a low-dimensional – abstract or high-level – description of the input. Opposed to our black box human mind the high level parameters in the algorithm are accessible and can be edited and transformed.
In this sense the AI systems allows to warp the representation of the human performers and thus stressing the fluidity of the modeling: A different person, character trait, evaluation or gender is far less substantially disparate as the subject would anticipate. The transformation of the parameters posses the character of (social, societal, clinical or biological) mind-altering states. The AI system is used to enable this altering with the aim to question the robustness and immobility of identity and world models. It tries to expose the internal constructiveness and in this sense works as a mirroring device. It recreates partial aspects of our perception and classification and through its alteration offers the viewer to draw a parallel to our own mental world building processes.
Presentation:
Credits:
Ensemble Resonanz
Co-developed with IRCAM, Paris
Audio Deep Learning Programming:
Antoine Caillon, Philippe Esling, Benjamin Levy (Ircam)
Video Deep Learning Programming:
Jorge Davila-Chacon (Heldenkombinat)
„Convergence“ was developed as part of #bebeethoven, a project of PODIUM Esslingen.
Funded by Kulturstiftung des Bundes.
Digital version commissioned by Eclat Festival.
Training Epochs:
Randomly Generated Outputs:
Audio Outputs:
Input Scream Recording:
Reconstruction with Model trained on Extended String Playing Techniques:
Input Scream Recording:
Reconstruction with Model trained on Extended String Playing Techniques:
Input Scream Recording:
Reconstruction with Model trained on Extended String Playing Techniques:
Input Violin Recording:
Reconstruction with Model trained on Extended String Playing Techniques:
Reconstruction with Model trained on Screams:
Reconstruction with Model trained on Singing:
Reconstruction with Model trained on Speaking Voices:
Input Violin Recording:
Reconstruction with Model trained on Extended String Playing Techniques:
Reconstruction with Model trained on Screams:
Performance Images / Screenshots:
