Multimodal Interaction and Sensor Integration

About This Course

Over the 12 weeks, you will be fully immersed in the world of multimodal interaction. The course is structured with 2 Preparation weeks, 8 course weeks, and 2 Reflection weeks, allowing you to build a solid foundation, explore sensor technologies, and apply your knowledge in a capstone project. Whether you are aiming to work in XR, HCI, or applied sensor technologies, this course will give you both scientific insights and practical skills.

You will develop both individual skills and collaborative competencies, learning how to analyse multimodal systems, design adaptive interfaces, and evaluate user experiences. The goal is to equip you with the knowledge and tools to innovate in next-generation interactive systems.

Learning Objectives

By the end of this course, you will:

• Understand the scientific and technical foundations of multimodal interaction and sensor integration, including human perception and multisensory processing.
• Develop practical skills in designing, implementing, and evaluating interactive systems that combine multiple sensory modalities and integrate diverse sensor data.
• Apply principles of sensor fusion and multisensory perception to create adaptive, accessible, and immersive interfaces for XR, robotics, and smart environments.

Course structure

• Preparation Weeks (2 Weeks): Orientation, environment setup, and background skills for multimodal interaction.
• Course Weeks (8 Weeks): Foundations, perception, sensor technologies, haptics, multimodal feedback, gesture interaction, adaptive systems, and applications.
• Reflection Weeks (2 Weeks): Integration of assignments into a portfolio, peer reviews, and reflection on future applications of multimodal systems.

Study Programs

This course is mandatory for the following study programs:

• M.Sc. Advanced Digital Reality

This course is offered as an elective for the following study programs:

• M.Sc. Cybersecurity
• M.Sc. Applied AI
• M.Sc. Digital Leadership
• MBA Digital Technologies
• MBA Digital Transformation

¹: Generally all mandatory modules of a study program can be selected as an elective in all study programs in which they are not offered as a mandatory module. However, there are several rules that apply. First, at least half of the selected electives have to be taken from your study program. Second, all electives are only available in either odd or even quarters. Therefore, not all electives are available in all programs (given that you want to finish the program within the regular time frame and do not want to take that elective in parallel to your impact project.)

Micro Degree

• This course is offered as a micro degree.
• German UDS Micro Degrees are compatible with the European MOOC Consortiums Common Micro Credentials Framework.
• Micro Degrees will be rewarded with an equivalent of 5 ECTS.
• Micro Degrees are offered to non-regular students and require a fee of €900.

Prerequisites

To prepare, we recommend refreshing your knowledge of:

• Programming basics (C# and Unity scripting)
• Signal processing and data visualisation
• Human–computer interaction fundamentals
• Basic XR concepts (VR/AR principles, interaction methods)

Optional resources:

• Unity Learn (beginner/intermediate scripting)
• Introductory podcasts or videos on multimodal perception and XR systems
• Review of basic mathematics for coordinate systems and 3D transformations

General Information

• Teaching Format: Video lectures, hands-on assignments, discussion forums (Yellowdig), and office hours.
• Total Workload Master: 125h (40h/85h) / 5 ECTS
• Total Workload MBA: 100h (40h/60h) / 4 ECTS
• Total Workload Micro Degree: 125h (40h/85h) / Equivalent to 5 ECTS
• Module coordinator: Dr. Daniele Di Mitri
• Examinations: Weekly graded assignments, quizzes, project work, and a capstone project portfolio.
• Offered: Even quarters