Multimodal Interaction and Sensor Integration

About This Course

In today’s fast-evolving business world, digital capabilities are crucial for success. This module offers an in-depth understanding of the role that digital technologies play in improving business processes, enhancing customer value, managing risks, and creating new revenue opportunities. We will explore everything from the basics of digital media, how data is encoded and compressed, to the fundamentals of the Internet and emerging digital technologies such as AI and blockchain.

Against the backdrop of global digitization, this course provides a holistic view of how digital technologies impact not only industries but society at large. Through a combination of lectures, practical exercises, and project-based learning, we will dive into the nature of the Internet, the World Wide Web, and the future potential of digital innovations.

Learning Objectives

• Understand the scientific and technical foundations of multimodal interaction and sensor integration, including human perception and multi-sensory processing.
• Develop practical skills in designing, implementing, and evaluating interactive systems that combine multiple sensory modalities and integrate diverse sensor data.
• Apply principles of sensor fusion and multi-sensory perception to create adaptive, accessible, and immersive interfaces for XR, robotics, and smart environments.

Course structure

• Preparation Weeks (2 Weeks): Orientation, environment setup, and background skills for multimodal interaction.
• Course Weeks (8 Weeks): Foundations, perception, sensor technologies, haptics, multimodal feedback, gesture interaction, adaptive systems, and applications.
• Reflection Weeks (2 Weeks): Integration of assignments into a portfolio, peer reviews, and reflection on future applications of multimodal systems.

Study Programs

This course is mandatory for the following study programs:

• M.Sc. Advanced Digital Reality

This course is offered as an elective for the following study programs:

• M.Sc. Cybersecurity
• M.Sc. Applied AI
• M.Sc. Digital Leadership
• MBA Digital Technologies
• MBA Digital Transformation

¹: Generally all mandatory modules of a study program can be selected as an elective in all study programs in which they are not offered as a mandatory module. However, there are several rules that apply. First, at least half of the selected electives have to be taken from your study program. Second, all electives are only available in either odd or even quarters. Therefore, not all electives are available in all programs (given that you want to finish the program within the regular time frame and do not want to take that elective in parallel to your impact project.)

Micro Degree

• This course is offered as a micro degree.
• German UDS Micro Degrees are compatible with the European MOOC Consortiums Common Micro Credentials Framework.
• Micro Degrees will be rewarded with an equivalent of 5 ECTS.
• Micro Degrees are offered to non-regular students and require a fee of €900.

Prerequisites

• Programming basics (especially C# and Unity scripting)
• 3D geometry and transformations
• Core concepts in computer graphics and human-computer interaction

General Information

• Teaching Format: Mastery:Coding
• Total Workload Master: 125h (40h/85h) / 5 ECTS
• Total Workload MBA: 100h (40h/60h) / 4 ECTS
• Total Workload Micro Degree: 125h (40h/85h) / Equivalent to 5 ECTS
• Module coordinator: Dr. Daniele Di Mitri
• Examinations: Quizzes, presentation(s), essay(s)/paper(s), project report(s), written exam (tbd) - Details will be announced with course start.
• Offered: Odd quarters