IPEM: Institute for Psychoacoustics and Electronic Music

Music offers profound pleasure and value to human beings; it supports expression and emotion, leads to strong experiences, and has powerful effects on social, cognitive and motor functions.
At IPEM, we study music from the viewpoint of humans’ lived, embodied interactions with music; ‘the body in action’.

IPEM’s research vision

XR: New spaces for embodied interaction with music and creative culture

XR create immersive and multisensory environments, objects, avatars, and agents. Using bodily interaction technologies Encompass both simulations of existing physical spaces, or spaces that spring from creative imagination . At IPEM, we explore how these spaces may function as innovative research labs, museums, and concert halls of the future.

MusiXR (a music research lab of the future)

Human music interaction exhibits various patterns of coordinated activity; in sound, bodily movement and (neuro)physiological responses. Adopting XR as core methodological tool in empirical-cognitive musicology, we try to understand the foundations of how coordinated patterns emerge and evolve over time, and how this creates value in humans’ subjective experience.

HEALTH AND WELL-BEING:

Improving a healthy lifestyle through creativity and art

To build and sustain a healthy and active life, motivation and successful motor control are of utmost importance. As music taps into the human cognitive, emotional, social, and sensorimotor capacities of human beings, we belief that it can contribute substantially to an active and healthy life.

At IPEM, we explore how music and musical feedback systems (sonification strategies) may be of benefit for sports activities (running, cycling, body building, etc.) as well as for cognitive and motor rehabilitation in clinical populations, such as Parkinson disease, Alzheimer, stroke, multiple sclerose, and developmental coordination disorder patients.

XRhive (a museum of the future)

Cultural heritage connects us to our past, shapes individual and collective narratives, and raises cross-cultural awareness. Using XR as technological framework, our ambition is to develop an interaction-based approach to cultural heritage preservation, embedding and valuing heritage in a living culture of human interaction; what we call a “living archive”.

RELIVING CULTURAL HERITAGE:

Embodied interaction with the cultural past in meaningful ways

Cultural heritage provides concrete and tangible connections to our past. It shapes individual and collective narratives and values, and raises cross-cultural awareness and exchanges.

At IPEM, we develop an art-science-technology framework for innovating cultural heritage preservation in the broad domain of performance art. The central ambition is to complement traditional “document-based heritage” with “interaction-based heritage”, embedding and valuing heritage in a living culture of human interaction and experience; what we call a “living archive”.

We use cutting-edge digital tools and technologies to foster this interaction-based approach to cultural heritage and the re-creation of cultural value. This endeavor is fundamentally interdisciplinary as technology-development is informed by knowledge on the cultural and historical object and art praxis (art-science) and by knowledge on principles drive human interaction with art and culture, such as expressivity, reward, etc.

XRt (a concert hall of the future)

Spurred by the creative impulses of artists, IPEM wants to be at the forefront of art-science research on XR-based forms of experience and interaction in the arts. By exploring the possibilities of immersive audiovisual displays, bodily interaction technologies, and data networks, we want to proactively innovate practices in the cultural-creative sector.

XR – THE CONCERT HALL OF THE FUTURE:

Studying and augmenting human interaction in XR musical environments

XR multimodal displays allow simulating existing real-life musical environments, creating for users the feeling of actually being present in this context. XR displays may also create radically new immersive, multimodal and imaginary environments for users. Complemented with motion capture technologies and high-speed data networks, it becomes possible for users to (socially) interact in a highly embodied manner, within an XR environment, even when they are not physically together. In addition, virtual agents in all shapes and colors may be created to interact creatively in real time with users. Given these possibilities, we consider XR environments as a radically new space for the study and practice of creative and artistic interactions.

[ video ]

At IPEM, we want to study human interactions within these XR musical environments. The ability to accurately simulate multimodal environments, to control the behavior of virtual agents therein, and to create scenarios which are otherwise impossible in our physical environment, turns XR into a performance space that radically extents research frontiers in empirical and cognitive musicology.

From churches in Medieval times to classical concert halls and digital IPods, musical practices and experiences have always been shaped by the spaces in which they occurred. XR unlocks a new space that holds enormous possibilities for music experience and interaction. By integrating scientific knowledge with the creative spark of artists and the possibilities of new technologies, IPEM wants to be at the forefront in developing new, XR-based forms of experience and interaction, innovating practices in the cultural-creative sector.

Music & Wellbeing  (Improving a healthy lifestyle through creativity and art)

To build and sustain a healthy and active life, motivation and successful motor control are of utmost importance. As music taps into the human cognitive, emotional, social, and sensorimotor capacities of human beings, we belief that it can contribute substantially to an active and healthy life.

At IPEM, we explore how music and musical feedback systems (sonification strategies) may be of benefit for sports activities (running, cycling, body building, etc.) as well as for cognitive and motor rehabilitation in clinical populations, such as Parkinson disease, Alzheimer, stroke, multiple sclerose, and developmental coordination disorder patients.

  • Low impact runner: a music based bio-feedback system

  • The D-Jogger

    • Publication: Moens, B., Muller, C., Van Noorden, L., Franěk, M., Celie, B., Boone, J., … & Leman, M. (2014). Encouraging spontaneous synchronisation with D-Jogger, an adaptive music player that aligns movement and music. PloS one, 9(12), e114234.
    • In this study we explore how music can entrain human walkers to synchronise to the musical beat without being instructed to do so. For this, we use an interactive music player, called D-Jogger, that senses the user’s walking tempo and phase. D-Jogger aligns the music by manipulating the timing difference between beats and footfalls. Experiments are reported that led to the development and optimisation of four alignment strategies. The first strategy matched the music’s tempo continuously to the runner’s pace. The second strategy matched the music’s tempo at the beginning of a song to the runner’s pace, keeping the tempo constant for the remainder of the song. The third alignment starts a song in perfect phase synchrony and continues to adjust the tempo to match the runner’s pace. The fourth and last strategy additionally adjusts the phase of the music so each beat matches a footfall. The first two strategies resulted in a minor increase of steps in phase synchrony with the main beat when compared to a random playlist, the last two strategies resulted in a strong increase in synchronised steps. These results may be explained in terms of phase-error correction mechanisms and motor prediction schemes. Finding the phase-lock is difficult due to fluctuations in the interaction, whereas strategies that automatically align the phase between movement and music solve the problem of finding the phase-locking. Moreover, the data show that once the phase-lock is found, alignment can be easily maintained, suggesting that less entrainment effort is needed to keep the phase-lock, than to find the phase-lock. The different alignment strategies of D-Jogger can be applied in different domains such as sports, physical rehabilitation and assistive technologies for movement performance.
  • 3Mo: a model for music-based biofeedback

    • Publication: Maes, P. J., Buhmann, J., & Leman, M. (2016). 3Mo: a model for music-based biofeedback. Frontiers in neuroscience, 10, 548.
    • In the domain of sports and motor rehabilitation, it is of major importance to regulate and control physiological processes and physical motion in most optimal ways. For that purpose, real-time auditory feedback of physiological and physical information based on sound signals, often termed “sonification,” has been proven particularly useful. However, the use of music in biofeedback systems has been much less explored. In the current article, we assert that the use of music, and musical principles, can have a major added value, on top of mere sound signals, to the benefit of psychological and physical optimization of sports and motor rehabilitation tasks. In this article, we present the 3Mo model to describe three main functions of music that contribute to these benefits. These functions relate the power of music to Motivate, and to Monitor and Modify physiological and physical processes. The model brings together concepts and theories related to human sensorimotor interaction with music, and specifies the underlying psychological and physiological principles. This 3Mo model is intended to provide a conceptual framework that guides future research on musical biofeedback systems in the domain of sports and motor rehabilitation.
  • The SoundBike: musical sonification strategies to enhance cyclists’ spontaneous synchronization to external music

    • Publication: Maes, P. J., Lorenzoni, V., & Six, J. (2019). The SoundBike: musical sonification strategies to enhance cyclists’ spontaneous synchronization to external music. Journal on Multimodal User Interfaces, 13(3), 155-166.
    • The spontaneous tendency of people to synchronize their movements to music is a powerful mechanism useful for the development of strategies for tempo adaptation of simple repetitive movements. In the current article, we contribute to such strategies—applied to cycling—by introducing a new strategy based on the sonification of cyclists’ motor rhythm. For that purpose, we developed the SoundBike, a stationary bike equipped with sensors that allows interactive sonification of cyclists’ motor rhythm using two distinct but compatible sonification methods. One is based on the principle of step sequencers, which are frequently used for electronic music production. The other is based on the Kuramoto model, allowing automatic and continuous phase alignment of beat-annotated music pieces to cyclists’ motor rhythm, i.e., pedal cadence. Apart from an in-depth presentation of the technical aspects of the SoundBike, we present an experimental study in which we investigated whether the SoundBike could enhance spontaneous synchronization of cyclists to external music. The results of this experiment suggest that sonification of cyclists’ motor rhythm may increase their tendency to synchronize to external music, and helps to keep a more stable pedal cadence, compared to the condition of having external music only (without sonification). Although the results are preliminary and should be followed-up by additional experiments to become more conclusive, SoundBike seems anyhow a promising interactive sonification device to assist motor learning and adaptation in the field of sports and motor rehabilitation.
  • The Augmented Movement Platform for Embodied Learning (AMPEL)

    • Publication: Moumdjian, L., Vervust, T., Six, J., Schepers, I., Lesaffre, M., Feys, P., & Leman, M. (2021). The Augmented Movement Platform for Embodied Learning (AMPEL): development and reliability. Journal on Multimodal User Interfaces, 15, 77-83.
    • Balance and gait impairments are highly prevalent in the neurological population. Although current rehabilitation strategiesfocus on motor learning principles, it is of interest to expand into embodied sensori-motor learning; that is learning through acontinuous interaction between cognitive and motor systems, within an enriched sensory environment. Current developmentsin engineering allow for the development of enriched sensory environments through interactive feedback. The AugmentedMovement Platform for Embodied Learning (AMPEL) was developed, both in terms of hardware and software by an inter-disciplinary circular participatory design strategy. The developed device was then tested for in-between session reliability forthe outcome measures inter-step interval and total onset time. Ten healthy participants walked in four experimental paths onthe device in two different sessions, and between session correlations were calculated. AMPEL was developed both in termsof software and hardware, with three Plug-In systems (auditory, visual, auditory + visual). The auditory Plug-In allows forflexible application of augmented feedback. The in-between session reliability of the outcomes measured by the system werebetween high and very high on all 4 walked paths, tested on ten healthy participants [mean age 41.8±18.5; BMI 24.8±6.1]. AMPEL shows full functionality, and has shown between session reliability for the measures of inter-step-intervals andtotal-onset-time in healthy controls during walking on different paths.
  • Embodied learning in multiple sclerosis using melodic, sound, and visual feedback: a potential rehabilitation approach

    • Publication: Moumdjian, L., Six, J., Veldkamp, R., Geys, J., Van Der Linden, C., Goetschalckx, M., … & Feys, P. (2022). Embodied learning in multiple sclerosis using melodic, sound, and visual feedback: a potential rehabilitation approach. Annals of the New York Academy of Sciences, 1513(1), 153-169.
    • Given the prevalence of motor and cognitive functions in persons with multiple sclerosis (PwMS), we proposed thatthetheoreticalframeworkofembodimentcouldprovidearehabilitationavenuetotrainthesefunctionsasonefunc-tional unit. PwMS (n=31) and age- and gender-matched healthy controls (n=30)underwentanembodiedlearn-ing protocol. This involved learning a cognitive sequence while performing it through bodily stepping movementunder three feedback conditions (melody, sound, and visual). Cognitive and movement performance was assessedby a delayed recall 15 min after undergoing the embodied learning protocol. Half of participants correctly recalledthe sequence in all three conditions, while 70% of healthy controls achieved correct recall within the melody condi-tion. Balance impairment predicted the speed of executing the sequence irrespective of learning, most apparent inthe melody condition. Information processing speed predicted the speed of executing the sequence in the melodyand sound conditions between participants as well as over time. Those who learned performed the sequence fasterin the melody condition only and overall were faster over time. We propose how embodied learning could expandthe current context of rehabilitation of cognitive and motor control in PwMS.
  • The effect of the severity of neurocognitive disorders on emotional and motor responses to music

    • Publication: Hobeika, L., Ghilain, M., Schiaratura, L., Lesaffre, M., Puisieux, F., Huvent‐Grelle, D., & Samson, S. (2022). The effect of the severity of neurocognitive disorders on emotional and motor responses to music. Annals of the New York Academy of Sciences, 1518(1), 231-238.
    • The successful design of musical interventions for dementia patients requires knowledge of how rhythmic abilities change with disease severity. In this study, we tested the impact of the severity of the neurocognitive disorders (NCD) on the socioemotional and motor responses to music in three groups of patients with Major NCD, Mild NCD, or No NCD. Patients were asked to tap to a metronomic or musical rhythm while facing a live musician or through a video. We recorded their emotional facial reactions and their sensorimotor synchronization (SMS) abilities. Patients with No NCD or Mild NCD expressed positive socioemotional reactions to music, but patients with Major NCD did not, indicating a decrease in the positive emotional impact of music at this stage of the disease. SMS to a metronome was less regular and less precise in patients with a Major NCD than in patients with No NCD or Mild NCD, which was not the case when tapping with music, particularly in the presence of a live musician, suggesting the relevance of live performance for patients with Major NCD. These findings suggest that the socio-emotional and motor reactions to music are negatively affected by the progression of the NCD.

Publications

Sorry, no publications matched your criteria.

Contact Details

Contact details