Development of a Multisensory Immersive System and Educational Solution for People with Hearing Impairments (Apr. 2023 – Dec. 2025)

This project aims to improve music accessibility for people with hearing impairments by developing technologies that enable comprehensive and immersive perception of musical elements—such as rhythm, melody, harmony, and timbre—through alternative senses including vision and touch. Based on these technologies, an educational solution supporting musical activities is also being developed.

Music education is effective for auditory training and speech-language rehabilitation, and has positive emotional impacts in daily life. However, the lack of technologies that enable musical and cultural participation for people with hearing impairments has limited improvements in quality of life. Furthermore, many existing approaches emphasize engineering performance without considering appropriate technology principles, resulting in limited social sustainability.

To address these challenges, this project will develop real-time music element recognition algorithms and visual/tactile transformation technologies, as well as a flexible platform for immersive music systems that supports various adaptations to ensure long-term sustainability. In addition, an educational music solution based on the developed technologies will be designed and validated through empirical studies in real educational settings.