Heads-up Computing

Towards to the next interaction paradigm via wearable intelligent assistants

Meet Shengdong (Shen) Zhao

I am a full professor in the School of Creative Media and Computer Science Department at the City University of Hong Kong (CityU). Previously, I held the position of Associate Professor in the Computer Science Department of the National University of Singapore (NUS). I obtained my PhD in computer science from the University of Toronto and my master’s degree from the School of Information Management & Systems at the University of California, Berkeley. My undergraduate education included a dual major in computer science and biology from Linfield College, Oregon, USA. In January 2009, I founded the Synteraction (formerly NUS-HCI) Lab. My research focuses on developing innovative interface tools and applications that simplify and enhance people’s lives, such as Draco, which won the Best iPad App of the Year in 2016. My publications regularly occur in top HCI conference proceedings and journals, including ToCHI, CHI, UbiComp, CSCW, UIST, and IUI. I am also passionate about bridging academic research with industry and have served as a senior consultant with the Huawei Consumer Business Group. Additionally, I actively participate in program committees for leading HCI conferences and have served as the paper co-chair for the ACM SIGCHI conferences in 2019 and 2020. During my leisure time, I enjoy reading, running, and exploring the beauty of nature.

Latest News and Blogs

...
05 Dec
Celebrating Major Milestones in HCI
Learn more
...
15 Nov
Award-Winning Research at ISS 2024
Learn more
...
03 Oct
Prof. Shengdong Zhao and Prof. Mark Billinghurst Dine in Adelaide
Learn more
...
07 Aug
UIST 2024: GPTVoiceTasker: Advancing Multi-step Mobile Task Efficiency Through Dynamic Interface Exploration and Learning
Learn more
...
23 Jul
The Conclusion of the 2024 Chiang Mai Research Camp
Learn more
...
24 Jun
UbiComp/ISWC 2024 Workshop on Heads-Up Computing
Learn more

Publications

2022 Visual Behaviors and Mobile Information Acquisition Nuwan Janaka, Xinke Wu, Shan Zhang, Shengdong Zhao, Petr Slovák
CHI  2022 Paracentral and near-peripheral visualizations: Towards attention-maintaining secondary information presentation on OHMDs during in-person social interactions Nuwan Janaka, Chloe Haigh, Hyeong Cheol Kim, Shan Zhang, Shengdong Zhao View PDF
CHI  2022 Does Dynamically Drawn Text Improve Learning? Investigating the Effect of Text Presentation Styles in Video Learning YAshwin Ram, Shengdong Zhao View PDF
ACM  2022 SSpoon: A Shape-changing Spoon That Optimizes Bite Size for Eating Rate Regulation. Yang Chen, Katherine Fennedy, Anna Fogel, Shengdong Zhao, Chao Zhang, Lijuan Liu, Ching Chiuan Yen View PDF
2023 GestureGPT: Zero-shot Interactive Gesture Understanding and Grounding with Large Language Model Agents. JXin Zeng, Xiaoyu Wang, Tengxiang Zhang, Chun Yu, Shengdong Zhao, Yiqiang Chen
2023 Impact of Human-AI Interaction on User Trust and Reliance in AI-Assisted Qualitative Coding. Jie Gao, Junming Cao, ShunYi Yeo, Kenny Tsu Wei Choo, Zheng Zhang, Toby Jia-Jun Li, Shengdong Zhao, Simon Tangi Perrault View PDF
SUI  2023 GlassMessaging: Supporting Messaging Needs During Daily Activities Using OST-HMDs Nuwan Janaka, Jie Gao, Lin Zhu, Shengdong Zhao, Lan Lyu, Peisen Xu, Maximilian Nabokow, Silang Wang, Yanch Ong View Video
2023 Supporting Artefact Awareness in Partially-Replicated Workspaces. Emran Poh, Anthony Tang, Jeannie Su Ann Lee, Shengdong Zhao View Link
INTERACT  2023 AdaptReview: Towards Effective Video Review Using Text Summaries and Concept Maps. Shan Zhang, Yang Chen, Nuwan Janaka, Chloe Haigh, Shengdong Zhao, Wei Tsang Ooi View Link
CHI  2023 ParaGlassMenu: Towards Social-Friendly Subtle Interactions in Conversations Runze Cai, Nuwan Nanayakkarawasam Peru Kandage Janaka, Shengdong Zhao, Minghui Sun
CHI  2023 Not All Spacings are Created Equal: The Effect of Text Spacings in On-the-go Reading Using Optical See-Through Head-Mounted Displays. Chen Zhou, Katherine Fennedy, Felicia Fang-Yi Tan, Shengdong Zhao, Yurui Shao View PDF
CHI  2023 Can Icons Outperform Text? Understanding the Role of Pictograms in OHMD Notifications. Nuwan Nanayakkarawasam Peru Kandage Janaka, Shengdong Zhao, Shardul Sapkota View PDF
CHI  2023 NotiFade: Minimizing OHMD Notification Distractions Using Fading. Nuwan Janaka, Shengdong Zhao, Samantha W. T. Chan View Video
2023 Mindful Moments: Exploring On-the-go Mindfulness Practice On Smart-glasses. Felicia Fang-Yi Tan, Ashwin Ram, Chloe Haigh, Shengdong Zhao View PDF
2023 Balancing Accuracy and Speed in Gaze-Touch Grid Menu Selection in AR via Mapping Sub-Menus to a Hand-Held Device Yang Tian, Yulin Zheng, Shengdong Zhao, Xiaojuan Ma, Yunhai Wang View PDF
2023 BackTracer: Improving ray-casting 3D target acquisition by backtracking the interaction history. Kaiqi Chen, Huagen Wan, Shengdong Zhao, Xutao Liu View PDF
2023 VidAdapter: Adapting Blackboard-Style Videos for Ubiquitous Viewing Ashwin Ram, Han Xiao, Shengdong Zhao, Chi-Wing Fu View PDF
2023 GlassMessaging: Towards Ubiquitous Messaging Using OHMDs Nuwan Janaka, Jie Gao, Lin Zhu, Shengdong Zhao, Lan Lyu, Peisen Xu, Maximilian Nabokow, Silang Wang, Yanch Ong View PDF
2023 Heads-Up Computing Moving Beyond the Device-Centered Paradigm Shengdong Zhao, Felicia Tan, Katherine Fennedy View PDF
2024 GPT VoiceTasker: LLM-Powered Virtual Assistant for SmartphoneWeb Development Success Minh Duc Vu, Han Wang, Zhuang Li, Jieshan Chen, Shengdong Zhao, Zhenchang Xing, Chunyang Chen View PDF
2024 AudioXtend: Assisted Reality Visual Accompaniments for Audiobook Storytelling During Everyday Routine Tasks Felicia Fang-Yi Tan, Peisen Xu, Ashwin Ram, Wei Zhen Suen, Shengdong Zhao, Yun Huang, Christophe Hurter View PDF
2024 Facilitating Virtual Reality Integration in Medical Education: A Case Study of Acceptability and Learning Impact in Childbirth Delivery Training Chang Liu, Felicia Fang-Yi Tan, Shengdong Zhao, Abhiram Kanneganti, Gosavi Arundhati Tushar, Eng Tat Khoo View PDF
2024 PANDALens: Towards AI-Assisted In-Context Writing on OHMD During Travels Runze Cai, Nuwan Janaka, Yang Chen, Lucia J. Wang, Shengdong Zhao, Can Liu View PDF
2024 Demonstrating PANDALens: Enhancing Daily Activity Documentation with AI-assisted In-Context Writing on OHMD Nuwan Janaka, Runze Cai, Shengdong Zhao, David Hsu View PDF
2024 Heads-Up Multitasker: Simulating Attention Switching On Optical Head-Mounted Displays Yunpeng Bai, Aleksi Ikkala, Antti Oulasvirta, Shengdong Zhao, Lucia J. Wang, Pengzhi Yang, Peisen Xu View PDF
2024 Navigating Real-World Challenges: A Quadruped Robot Guiding System for Visually Impaired People in Diverse Environments Shaojun Cai, Ashwin Ram, Zhengtai Gou, Mohd Alqama Wasim Shaikh, Yu-An Chen, Yingjia Wan, Kotaro Hara, Shengdong Zhao, David Hsu View PDF
2024 GlassMail: Towards Personalised Wearable Assistant for On-the-Go Email Creation on Smart Glasses Chen Zhou, Zihan Yan, Ashwin Ram, Yue Gu, Yan Xiang, Can Liu, Yun Huang, Wei Tsang Ooi, Shengdong Zhao View PDF
2024 Kine-Appendage: Enhancing Freehand VR Interaction Through Transformations of Virtual Appendages Yang Tian, Hualong Bai, Shengdong Zhao, Chi-Wing Fu, Chun Yu, Haozhao Qin, Qiong Wang, Pheng-Ann Heng View PDF
2024 SelfIE: Self-Initiated Explorable Instructions Towards Enhanced User Experience Hyeongcheol Kim, Katherine Fennedy, Georgia Zhang, Can Liu, Shengdong Zhao View PDF
2024 Demonstrating PilotAR: A Tool to Assist Wizard-of-Oz Pilot Studies with OHMD Nuwan Janaka, Runze Cai, Shengdong Zhao, David Hsu View PDF
2024 TOM: A Development Platform For Wearable Intelligent Assistants Nuwan Janaka, Shengdong Zhao, David Hsu, Sherisse Tan Jing Wen, Chun-Keat Koh View PDF
2024 PilotAR: Streamlining Pilot Studies with OHMDs from Concept to Insight Nuwan Janaka, Runze Cai, Ashwin Ram, Lin Zhu, Shengdong Zhao, Kai Qi Yong View PDF
2024 Demonstrating TOM: A Development Platform for Wearable Intelligent Assistants in Daily Activities Nuwan Janaka, Shengdong Zhao, David Hsu, Sherisse Tan Jing Wen, Chun-Keat Koh View PDF
2024 What's this? Understanding User Interaction Behaviour with Multimodal Input Information Retrieval System Silang Wang, Hyeongcheol Kim, Nuwan Janaka, Kun Yue, Hoang-Long Nguyen, Shengdong Zhao, Haiming Liu, Khanh-Duy Le View PDF
2024 GPTVoiceTasker: Advancing Multi-step Mobile Task Efficiency Through Dynamic Interface Exploration and Learning Minh Duc Vu, Han Wang, Jieshan Chen, Zhuang Li, Shengdong Zhao, Zhenchang Xing, Chunyang Chen View PDF
2024 GestureGPT: Toward Zero-Shot Free-Form Hand Gesture Understanding with Large Language Model Agents Xin Zeng, Xiaoyu Wang, Tengxiang Zhang, Chun Yu, Shengdong Zhao, Yiqiang Chen View PDF
2024 Collective Creation of Intimacy: Exploring the Cosplay Commission Practice within the Otome Game Community in China Yihao Zhou, Haowei Xu, Lili Zhang, Shengdong Zhao View PDF
2025 SimulataR: Rapid Assisted Reality Prototyping using Design-Blended Videos Ashwin Ram, Yue Gu, Bowen Wang, Sneha Jaikumar, Youqi Wu, Benjamin Tan Kuan Wei, Qingyang Xu, Haiming Liu, Shengdong Zhao View PDF
2025 AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glasses Runze Cai, Nuwan Janaka, Hyeongcheol Kim, Yang Chen, Shengdong Zhao, Yun Huang, David Hsu View PDF
2025 SummonBrush: Enhancing Touch Interaction on Large XR User Interfaces by Augmenting Users' Hands with Virtual Brushes Yang Tian, Zhao Su, Tianren Luo, Teng Han, Shengdong Zhao, Youpeng Zhang, Yixin Wang, Boyu Gao, Dangxiao Wang View PDF
2025 ViFeed: Promoting Slow Eating and Food Awareness through Strategic Video Manipulation during Screen-Based Dining Yang Chen, Felicia Fang-Yi Tan, Zhuoyu Wang, Xing Liu, Jiayi Zhang, Yun Huang, Shengdong Zhao, Ching Chiuan Yen View PDF
2025 Bridging Gaps in HCI: Advancing Education, Research, and Careers in Asia Dilrukshi Gamage, Shiwei Cheng, Preeti Mudliar, Zhicong Lu, Shengdong Zhao, Nova Ahmed, Xiaojuan Ma, Uichin Lee, Ding Wang View PDF
2025 Designing Interactive Multimodal Information Retrieval and Access for Heads Up Computing (DIMIRA-HUC) Haiming Liu, Shengdong Zhao, Silang Wang, Preben Hansen, Ian Oakley, Khanh-Duy Le View PDF
15 Years of
experience
120 + Publications
48 Collaborators
15 PhD
graduates

Research Team

Ashwin Ram Postdoc
Nuwan Janaka Postdoc
Bai Yunpeng Phd Student
Cai Runze Phd Student
Luoying Phd Student
Zhou Chen Phd Student
Young Research Assistant
Yaorui Research Assistant