多模態人機互動
Multimodal Human-Computer Interaction
開課學期 Semester： 114-2
課號 Class number：CSIE7641 資工系研究所選修（歡迎外系與大學部加選）
授課教授 Instructor： 鄧善元 Shan-Yuan Teng
上課時間 Time： 星期二 Tuesday 789
上課地點 Location：學新 523 (CSIE Make Reality Space)
上課語言 Language：English
開課人數 Student Quota：30
Synopsis
Humans perform physical tasks with abundant multimodal information from visual, auditory, tactile, and more senses. Mainstream computers still offer very limited input and output capabilities when interacting with the users (e.g., multitouch gestures, speech input). This course provides project-based knowledge on human sensory perception & cognition and technologies that accommodate these, including topics on eye-tracking, auditory UI, robotic touch interfaces. Students will conduct user studies and develop software-hardware systems that explore and leverage multimodal interaction in these projects. Later in the course, students will explore an open-ended research project guided by the instructor.
Prerequisites
No prerequisites but basic programming skills are recommended.
Syllabus
| Week | Date | Content | Resources |
|---|---|---|---|
| 1 | 2/24 | Intro to multimodalities in Human-Computer Interaction & Class logistics | |
| 2 | 3/3 | Visual perception I: eye-tracking | |
| 3 | 3/10 | Visual perception II: user study in class | |
| 4 | 3/17 | Visual perception III: application demo | |
| 5 | 3/24 | Auditory perception I: audio UI | |
| 6 | 3/31 | Auditory perception II: user study in class | |
| 7 | 4/7 | Auditory perception III: application demo | |
| 8 | 4/14 | Final Project proposal | |
| 9 | 4/21 | Tactile perception I: robotic touch | |
| 10 | 4/28 | Tactile perception II: user study in class | |
| 11 | 5/5 | Tactile perception III: application demo | |
| 12 | 5/12 | Final project: paper writing | |
| 13 | 5/19 | Final project: prototyping | |
| 14 | 5/26 | Final project: user study | |
| 15 | 6/2 | Final presentation | |
| 16 | 6/9 | Paper submission |
Policies
- You must come to the first class to be enrolled in this class. If you are not enrolled but want to take this class, come to the first class. We will evaluate the number of students and let you know the availability before the second week.
- Physically attending each class is important (in class user studies and demos, both graded). If you cannot attend for any reason, you should email the instructor directly and it will be handled individually.
- No auditting. Our resources and space are limited.
- Students should prepare their own laptops (if needed, please contact the department).
AI policy
- Feeding assignment requirements into online AI models (e.g., ChatGPT) is strongly prohibited. You should not share slides outside of the class.
- References should be cited and manually checked (“ChatGPT” is not a valid reference, just like “Google”).
Copyright
Otherwise stated, the copyright of the materials in the class belongs to Shan-Yuan Teng.