多模態人機互動
Multimodal Human-Computer Interaction
開課學期 Semester： 114-2
課號 Class number：CSIE7641 資工系研究所選修（歡迎外系與大學部加選）
授課教授 Instructor： 鄧善元 Shan-Yuan Teng
上課時間 Time： 星期二 Tuesday 789
上課地點 Location：學新 523 (CSIE Make Reality Space)
上課語言 Language：English
開課人數 Student Quota：30
Synopsis
Humans perform physical tasks with abundant multimodal information from visual, auditory, tactile, and more senses. Mainstream computers still offer very limited input and output capabilities when interacting with the users (e.g., multitouch gestures, speech input). This course provides project-based knowledge on human sensory perception & cognition and technologies that accommodate these, including topics on eye-tracking, auditory UI, robotic touch interfaces. Students will conduct user studies and develop software-hardware systems that explore and leverage multimodal interaction in these projects. Later in the course, students will explore an open-ended research project guided by the instructor.
Prerequisites
Basic programming skills are required.
Syllabus
| Week | Date | Content | Resources |
|---|---|---|---|
| 1 | 2/24 | Intro to class | |
| 2 | 3/3 | Auditory modality I: intro | Install Processing Audio programming Audio games |
| 3 | 3/10 | Auditory modality II: user study in class | |
| 4 | 3/17 | Auditory modality III: application demo | |
| 5 | 3/24 | Visual modality I: intro | |
| 6 | 3/31 | Visual modality II: user study in class | |
| 7 | 4/7 | Visual modality III: application demo | |
| 8 | 4/14 | Robotic workshop | |
| 9 | 4/21 | Haptic modality I: intro | |
| 10 | 4/28 | Haptic modality II: user study in class | |
| 11 | 5/5 | Haptic modality III: application demo | |
| 12 | 5/12 | Final project: brainstorming | |
| 13 | 5/19 | Final project: prototyping | |
| 14 | 5/26 | Final project: user study | |
| 15 | 6/2 | Final presentation | |
| 16 | 6/9 | Paper submission |
Policies
- You must come to the first class to be enrolled in this class. If you are not enrolled but want to take this class, come to the first class. We will evaluate the number of students and let you know the availability before the second week.
- Physically attending each class is important (in class user studies and demos, both graded). If you cannot attend for any reason, you should email the instructor directly and it will be handled individually.
- No auditting. Our resources and space are limited.
- Students should prepare their own laptops with webcam and wired headphones (if needed, please contact the department).
Questions
Email multimodal-hci@csie.ntu.edu.tw; for personal request, please email the instructor directly.
Copyright
Otherwise stated, the copyright of the materials in the class belongs to Shan-Yuan Teng.