|
Lai Sum Yim, Quang TN Vo, Ching-I Huang, Chi-Ruei Wang, Wren McQueary, Hsueh-Cheng Wang, Haikun Huang, Lap-Fai Yu
WFH-VR: Teleoperating a Robot Arm to set a Dining Table across the Globe via Virtual Reality.
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022)
Paper, Project Page Video, Bibtex
VR teleoperation of a robot arm over the globe to set a dining table.
|
|
Changyang Li, Wanwan Li, Haikun Huang, Lap-Fai Yu
Interactive Augmented Reality Storytelling Guided by Scene Semantics.
ACM Transactions on Graphics (Proceeding of SIGGRAPH 2022)
Featured in SIGGRAPH Blog
Project Page, Paper, Supplementary Material, Video, Bibtex
Deploying AR stories to different scenes.
|
|
Wanwan Li, Haikun Huang, Tomay Solomon, Behzad Esmaeili, Lap-Fai Yu
Synthesizing Personalized Construction Safety Training Scenarios for VR Training.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2022
(Special Issue on IEEE Virtual Reality 2022)
Project Page, Paper, Video, Bibtex
Synthesizing personalized construction safety VR training scenarios.
|
|
Michael S Solah*, Haikun Huang*, Jiachuan Sheng, Tian Feng, Marc Pomplun, Lap-Fai Yu
*Equal contributors
Mood-Driven Colorization of Virtual Indoor Scenes.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2022
(Special Issue on IEEE Virtual Reality 2022)
Best Journal Paper Nominee
Project Page, Paper, Video, Invited Talk at IEEE Vis 2022, Bibtex
Synthesizing virtual environments w.r.t. target moods.
|
|
Changyang Li, Haikun Huang, Jyh-Ming Lien, Lap-Fai Yu
Synthesizing Scene-Aware Virtual Reality Teleport Graphs.
ACM Transactions on Graphics (Proceeding of SIGGRAPH Asia 2021)
Project Page, Paper, Supplementary Material, Video, Bibtex
Conditionally accepted. More details coming soon.
|
|
Yongqi Zhang, Haikun Huang, Erion Plaku, Lap-Fai Yu
Joint Computational Design of Workspaces and Workplans.
ACM Transactions on Graphics (Proceeding of SIGGRAPH Asia 2021)
Project Page, Paper, Supplementary Material, Video, Bibtex
Conditionally accepted. More details coming soon.
|
|
Wei Liang, Xinzhe Yu, Rawan Alghofaili, Yining Lang, Lap-Fai Yu
Scene-Aware Behavior Synthesis for Virtual Pets in Mixed Reality.
Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2021)
Project Page, Paper, Video, Bibtex
Conditionally accepted. More details coming soon.
|
|
Yujia Wang, Wei Liang, Haikun Huang, Yongqi Zhang, Dingzeyu Li, Lap-Fai Yu
Toward Automatic Audio Description Generation for Accessible Videos.
Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2021)
Project Page, Paper, Video, Bibtex
Conditionally accepted. More details coming soon.
|
|
Biao Xie, Huimin Liu, Rawan Alghofaili, Yongqi Zhang, Yeling Jiang, Flavio Destri Lobo, Changyang Li, Wanwan Li, Haikun Huang, Mesut Akdere, Christos Mousas, Lap-Fai Yu
A Review on Virtual Reality Skill Training Applications.
Frontiers in Virtual Reality, 2021
Paper, Bibtex
|
|
Jixuan Zhi, Lap-Fai Yu, Jyh-Ming Lien
Designing Human-Robot Coexistence Space.
IEEE Robotics and Automation Letters (RA-L), 2021
(IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021))
Project Page, Paper, Video, Bibtex
|
|
Rawan Alghofaili, Matthew Fisher, Richard Zhang, Michal Lukáč, Lap-Fai Yu
Exploring Sketch-based Character Design Guided by Automatic Colorization.
Proceedings of Graphics Interface (GI 2021)
Project Page, Paper, Video, Bibtex
A GAN-driven user interface for automatic colorization of character sketches.
|
|
Amilcar Gomez Samayoa*, Javier Talavera*, Siem G. Sium, Biao Xie, Haikun Huang, Lap-Fai Yu
*Equal contributors
Building a Motion-Aware, Networked Do-It-Yourself Holographic Display.
IEEE International Conference on Intelligent Reality (ICIR 2021)
Second Place Award, Application Competition on Intelligent Reality (Students Track)
Paper, Video, Bibtex
A simple DIY networked holographic display.
|
|
Hanqing Wang, Wei Liang, Lap-Fai Yu
Scene Mover: Automatic Move Planning for Scene Arrangement by Deep Reinforcement Learning.
ACM Transactions on Graphics (Proceeding of SIGGRAPH Asia 2020)
Project Page, Paper, Video, Bibtex
Conditionally accepted. More details coming soon.
|
|
*Wanwan Li, *Biao Xie, Yongqi Zhang, Walter Meiss, Haikun Huang, Lap-Fai Yu
*Equal contributors
Exertion-Aware Path Generation.
ACM Transactions on Graphics (Proceeding of SIGGRAPH 2020)
Project Page, Paper, Video, Bibtex
Generating exertion-aware paths over 3D terrains for VR exercising.
|
|
Wanwan Li*, Javier Talavera*, Amilcar Gomez Samayoa*, Jyh-Ming Lien, Lap-Fai Yu
*Equal contributors
Automatic Synthesis of Virtual Wheelchair Training Scenarios.
IEEE Virtual Reality (VR 2020)
Project Page, Paper, Video, Bibtex
Generating VR scenarios for wheelchair training.
|
|
Yujia Wang, Wei Liang, Wanwan Li, Dingzeyu Li, Lap-Fai Yu
Scene-Aware Background Music Synthesis.
ACM Multimedia (ACM MM 2020)
Oral presentation
Project Page, Paper, Video, Bibtex
Our approach automatically retrieves and assembles decent music pieces to fit with (game) scenes.
|
|
Haikun Huang, Yuxuan Zhang, Tomer Weiss, Rebecca W Perry, Lap-Fai Yu
Interactive Design of Gallery Walls via Mixed Reality.
IEEE Artificial Intelligence and Virtual Reality (AIVR 2020)
Project Page, Paper, Video, Bibtex
A novel mixed reality interface for gallery wall design.
|
|
Yujia Wang, Wenguan Wang, Wei Liang, Lap-Fai Yu
Comic-Guided Speech Synthesis.
ACM Transactions on Graphics (Proceeding of SIGGRAPH Asia 2019)
Project Page, Paper, Video, Bibtex
Automatic narration of characters in comics, mangas, etc. Conditionally accepted, more details coming soon.
|
|
Rawan Alghofaili, Yasuhito Sawahata, Haikun Huang, Hsueh-Cheng Wang, Takaaki Shiratori, Lap-Fai Yu
Lost in Style: Gaze-driven Adaptive Aid for VR Navigation.
Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2019)
Project Page, Paper, Video, Bibtex
Feel lost when navigating in a virtual environment (such as when playing a VR game)? This tool helps you get back on track by predicting when you need navigation help and popping up wayfinding aids.
|
|
*Yongqi Zhang, *Biao Xie, Haikun Huang, Elisa Ogawa, Tongjian You, Lap-Fai Yu
*Equal contributors
Pose-Guided Level Design.
Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2019)
Best Paper Honorable Mention Award
Project Page, Paper, Video, Bibtex
Procedural game level generator for motion/exercise games (e.g., Just Dance, Speed of Light).
|
|
*Haikun Huang, *Michael S. Solah, Dingzeyu Li, Lap-Fai Yu
*Equal contributors
Audible Panorama: Automatic Spatial Audio Generation for Panorama Imagery.
Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2019)
Project Page, Paper, Video, Bibtex
This work automatically synthesizes realistic spatial sounds for 360° panorama images for delivering immersive VR experiences
|
|
Yining Lang, Wei Liang, Yujia Wang, Lap-Fai Yu
3D Face Synthesis Driven by Personality Impression.
AAAI Conference on Artificial Intelligence (AAAI 2019)
Project Page, Paper, Video, Bibtex
Our approach automatically modifies a 3D face to give different personality impressions.
|
|
Yung-Shan Su, Shao-Huang Lu, Po Sheng Ser, Wei-Ting Hsu, Wei-Cheng Lai, Biao Xie, Hong-Ming Huang, Teng-Yok Lee, Hung-Wen Chen, Lap-Fai Yu, Hsueh-Cheng Wang
Reasoning Pose-aware Placements of Objects with Semantic Labels.
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019)
Paper, Project Page Video, Bibtex
Training a robotic arm to manipulate and place 3D objects with brand name labels. Conditionally accepted. More details coming soon.
|
|
Qianwen Chao, Xiaogang Jin, Hen-Wei Huang, Shaohui Foong, Lap-Fai Yu, Sai-Kit Yeung
Force-based Heterogeneous Traffic Simulation for Autonomous Vehicle Testing.
IEEE International Conference on Robotics and Automation (ICRA 2019)
Paper, Video, Bibtex
A force-based, heterogeneous traffic simulator that models the interactions between different road users in a simple and unified way.
|
|
Rawan Alghofaili, Michael S. Solah, Haikun Huang, Yasuhito Sawahata, Marc Pomplun, Lap-Fai Yu
Optimizing Visual Element Placement via Visual Attention Analysis.
IEEE Virtual Reality (VR 2019)
Project Page, Paper, Video, Bibtex
Populating visual elements (e.g., ads) in 3D virtual environments based on a visual attention predictor trained by eye-gaze data.
|
|
Yining Lang, Wei Liang, Lap-Fai Yu
Virtual Agent Positioning Driven by Scene Semantics in Mixed Reality.
IEEE Virtual Reality (VR 2019)
Project Page, Paper, Video, Bibtex
Positioning virtual agents in mixed reality by considering scene semantics and user interaction tasks.
|
|
Wei Liang, Jingjing Liu, Yining Lang, Bing Ning, Lap-Fai Yu
Functional Workspace Optimization via Learning Personal Preferences from Virtual Experiences.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2019
(Special Issue on IEEE Virtual Reality 2019)
Project Page, Paper, Video, Bibtex
By learning working habits of users from virtual environments, our approach generates personalized workspace designs automatically.
|
|
Yujia Wang, Wei Liang, Jianbing Shen, Yunde Jia, Lap-Fai Yu
A Deep Coarse-to-Fine Network for Head Pose Estimation from Synthetic Data.
Pattern Recognition (PR), 2019
Project Page, Paper, Video, Bibtex
Training a deep learning classifier using rendered head pictures to estimate head poses for applications such as VR/AR.
|
|
Lifeng Zhu, Benyi Xie, Yongjie Zhang, Lap-Fai Yu
Cartonist: Automatic Synthesis and Interactive Exploration of Nonstandard Carton Design.
Computer-Aided Design (CAD), 2019
Paper, Video, Bibtex
An interactive tool for exploring and creating interesting nonstandard carton designs.
|
|
Tian Feng, Quang-Trung Truong, Thanh Nguyen, Jing Yu Koh, Lap-Fai Yu, Alexander Binder, Sai-Kit Yeung
Urban Zoning Using Higher-Order Markov Random Fields on Multi-View Imagery Data.
European Conference on Computer Vision (ECCV 2018)
Paper, Bibtex
Automatic urban zoning using HO-MRF built on multi-view imagery data including street-view photos and top-view satellite images.
|
|
Tzu-Kuan Chuang, Ni-Ching Lin, Jih-Shi Chen, Chen-Hao Hung, Yi-Wei Huang, Chunchih Teng, Haikun Huang, Lap-Fai Yu, Laura Giarre, Hsueh-Cheng Wang
Deep Trail-Following Robotic Guide Dog in Pedestrian Environments for People who are Blind and Visually Impaired - Learning from Virtual and Real Worlds.
IEEE International Conference on Robotics and Automation (ICRA 2018)
Project Page, Paper, Video, Bibtex
Training a robotic guidedog that can help blind people to navigate in the real world
|
|
Yining Lang, Wei Liang, Fang Xu, Yibiao Zhao, Lap-Fai Yu
Synthesizing Personalized Training Programs for Improving Driving Habits via Virtual Reality.
IEEE Virtual Reality (VR 2018)
Project Page, Paper, Video, Bibtex
Synthesizing personalized driving training programs for VR-based training.
|
|
Biao Xie*, Yongqi Zhang*, Haikun Huang, Elisa Ogawa, Tongjian You, Lap-Fai Yu
*Equal contributors
Exercise Intensity-driven Level Design.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2018
(Special Issue on IEEE Virtual Reality 2018)
Featured on IEEE Xplore Innovation Spotlight
Project Page, Paper, Video, Bibtex
Our approach generates game levels that carry the right amount of exercise intensity one needs to meet his or her training goal. Have fun while exercising in virtual reality!
|
|
Chenfanfu Jiang, Siyuan Qi, Yixin Zhu, Siyuan Huang, Jenny Lin, Lap-Fai Yu, Demetri Terzopoulos, Song-Chun Zhu
Configurable 3D Scene Synthesis and 2D Image Rendering with Per-Pixel Ground Truth using Stochastic Grammars.
International Journal of Computer Vision (IJCV), 2018
Paper, Video, Bibtex
By sampling stochastic scene grammars, different realistic scenes can be synthesized for purposes such as graphics for vision and virtual reality.
|
|
Tomer Weiss, Alan Litteneker, Noah Duncan, Masaki Nakada, Chenfanfu Jiang, Lap-Fai Yu, Demetri Terzopoulos
Fast and Scalable Position-Based Layout Synthesis.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2018
Paper, Video, Bibtex
We present a fast, physics-motivated, continuous layout synthesis technique that can arrange hundreds of objects (e.g., chairs in a theater) in a few seconds.
|
|
Quang-Hieu Pham, Minh-Khoi Tran, Binh-Son Hua, Lap-Fai Yu, Duc Thanh Nguyen, Sai-Kit Yeung, et al.
SHREC'18: RGB-D Object-to-CAD Retrieval.
Eurographics Workshop on 3D Object Retrieval, 2018
Project Page, Paper, Bibtex
A report of our RGB-D Object-to-CAD retrieval contest.
|
|
Noah Duncan, Lap-Fai Yu, Sai-Kit Yeung, Demetri Terzopoulos
Approximate Dissections.
ACM Transactions on Graphics (Proceeding of SIGGRAPH Asia 2017)
Project Page, Paper, Video, Bibtex
Inspired by tangrams, our approach can automatically generate pieces that can be reassembled into different interesting shapes.
|
|
Hanqing Wang, Wei Liang, Lap-Fai Yu
Transferring Objects: Joint Inference of Container and Human Pose.
IEEE International Conference on Computer Vision (ICCV 2017)
Project Page, Paper, Video, Bibtex
Our approach jointly infers the human pose and the container for performing a transfer task.
|
|
Carla Aravena*, Mark Vo*, Tao Gao, Takaaki Shiratori, Lap-Fai Yu
*Equal contributors
Perception Meets Examination: Studying Deceptive Behaviors in VR.
Proceedings of the 39th Annual Meeting of the Cognitive Science Society (CogSci 2017)
Project Page, Paper, Video, Bibtex
An interesting study on human deceptive behavior based on the Theory of Mind inspired by the game Classroom Aquatic.
|
|
Haikun Huang, Ni-Ching Lin, Lorenzo Barrett, Darian Springer, Hsueh-Cheng Wang, Marc Pomplun, Lap-Fai Yu
Automatic Optimization of Wayfinding Design.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2017
Project Page, Paper, Supplementary, Video, Bibtex
An automatic tool to help wayfinding designers create better wayfinding designs for architectural spaces, and to help level designers create better game levels.
|
|
Changyang Li, Wei Liang, Chris Quigley, Yibiao Zhao, Lap-Fai Yu
Earthquake Safety Training through Virtual Drills.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2017
(Special Issue on IEEE Virtual Reality 2017)
Project Page, Paper, Video, Bibtex
Experience the shake and learn to protect yourself via immersive virtual environments.
|
|
Mahdi Jampour, Chen Li, Lap-Fai Yu, Kun Zhou, Stephen Lin, Horst Bischof
Face Inpainting based on High-Level Facial Attributes.
Computer Vision and Image Understanding (CVIU), 2017
Project Page, Paper, Supplementary, Video, Bibtex
Combining high-level and low-level cues for inpainting occluded face images.
|
|
Mahdi Jampour, Mohammad-Shahram Moin, Lap-Fai Yu, Horst Bischof
Mapping Forests: a Comprehensive Approach for Non-linear Mapping Problems.
Journal of Mathematical Imaging and Vision (JMIV), 2017
Paper, Bibtex
A new and robust mapping approach based on regression transformations for computer vision applications.
|
|
Duc Thanh Nguyen, Binh-Son Hua, Lap-Fai Yu, Sai-Kit Yeung
A Robust 3D-2D Interactive Tool for Scene Segmentation and Annotation.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2017
Project Page, Paper, Video, Bibtex
Details of our 3D scene annotation tool for annotating the SceneNN dataset.
|
|
Binh-Son Hua, Quang-Trung Truong, Minh-Khoi Tran, Quang-Hieu Pham, Lap-Fai Yu, Duc Thanh Nguyen, Sai-Kit Yeung, et al.
SHREC'17: RGB-D to CAD Retrieval with ObjectNN Dataset.
Eurographics Workshop on 3D Object Retrieval, 2017
Project Page, Paper, Bibtex
A report of our RGB-D object-to-CAD object retrieval contest.
|
|
Noah Duncan, Lap-Fai Yu, Sai-Kit Yeung
Interchangeable Components for Hands-On Assembly Based Modelling.
ACM Transactions on Graphics (Proceeding of SIGGRAPH Asia 2016)
Featured in SIGGRAPH Asia 2016 technical papers video trailer
Project Page, Paper (High Res), Paper (Low Res), Video, Fast Forward, Bibtex
This paper may be regarded as a co-fabrication approach for generating a set of seamlessly interchangeable components from 3D Models. It can be applied for toy and product design. From a small set of components generated by our approach, you can assemble thousands of different objects by hand. Have fun!
|
|
Tian Feng, Lap-Fai Yu, Sai-Kit Yeung, KangKang Yin, Kun Zhou
Crowd-driven Mid-scale Layout Design.
ACM Transactions on Graphics (Proceeding of SIGGRAPH 2016)
Project Page, Paper (High Res), Paper (Low Res), Video, Bibtex
This paper proposes to use crowd simulations to procedurally generate crowd-aware layout designs.
|
|
Binh-Son Hua, Quang-Hieu Pham, Duc Thanh Nguyen, Minh-Khoi Tran, Lap-Fai Yu, Sai-Kit Yeung
SceneNN: a Scene Meshes Dataset with aNNotations.
IEEE International Conference on 3D Vision (3DV 2016)
Oral presentation
Best Paper Honorable Mention
Project Website, Paper, Supplementary Material, Bibtex
We provide 100+ scene meshes with detailed annotations. The dataset will keep updating and expanding. Stay tuned on our project website.
|
|
Haikun Huang, Ni-Ching Lin, Lorenzo Barrett, Darian Springer, Hsueh-Cheng Wang, Marc Pomplun, Lap-Fai Yu
Analyzing Visual Attention via Virtual Environments.
Virtual Reality meets Physical Reality Workshop, SIGGRAPH Asia, 2016
Paper, Bibtex
|
|
Noah Duncan, Lap-Fai Yu, Sai-Kit Yeung, Demetri Terzopoulos
Zoomorphic Design.
ACM Transactions on Graphics (Proceeding of SIGGRAPH 2015)
Project Page, Paper, Video, Bibtex
This paper describes a computational approach to generate some cool animal-inspired designs.
|
|
|
Chen Li, Lap-Fai Yu, Zhichao Lu, Yasuyuki Matsushita, Kun Zhou, Stephen Lin
Continuous Symmetric Stereo with Adaptive Outlier Handling.
IEEE International Conference on 3D Vision (3DV 2015)
Paper, Supplementary Material, Bibtex
A stereo matching technique that provides a good combination of accuracy and speed, allowing one to generate high-quality reconstruction of outdoor images even with complex occlusions. More details coming soon.
|
|
Lap-Fai Yu, Noah Duncan, Sai-Kit Yeung
Fill and Transfer: A Simple Physics-based Approach for Containability Reasoning.
IEEE International Conference on Computer Vision (ICCV 2015)
Project Page, Paper, Supplementary Material, Video, Bibtex
We use simple physics to reason about the fill and transfer directions of containers. Many thanks to the chef for cooking our food slowly while Noah and I were baking this idea in a food court.
|
|
|
Lap-Fai Yu, Sai-Kit Yeung, Demetri Terzopoulos
The Clutterpalette: An Interactive Tool for Detailing Indoor Scenes.
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2014
Project Page, Paper (High Res), Paper (Low Res), Video, Bibtex
Having trouble detailing your virtual worlds? The Clutterpalette helps you quickly add scene details by using the clutter object distributions learnt from real-world RGB-D scene data.
|
|
Lap-Fai Yu, Sai-Kit Yeung, Yu-Wing Tai, Stephen Lin
Shading-based Shape Refinement of RGB-D Images.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013)
Paper (High Res), Paper (Low Res), Video, Bibtex
I had much fun playing with the Microsoft Kinect. The RGB-D images were captured in across countries.
|
|
Lap-Fai Yu, Sai-Kit Yeung, Yu-Wing Tai, Demetri Terzopoulos, Tony F. Chan
Outdoor Photometric Stereo.
IEEE International Conference on Computational Photography (ICCP 2013)
Oral presentation
Paper (High Res), Paper (Low Res), Video, Data, Bibtex
This work brings photometric stereo out of the lab.
|
|
Lap-Fai Yu
Data-driven Optimization for Modeling in Computer Graphics and Vision.
Ph.D. Dissertation, Computer Science Department, UCLA, 2013
Advisor: Demetri Terzopoulos
|
|
Lap-Fai Yu, Sai-Kit Yeung, Demetri Terzopoulos, Tony F. Chan
DressUp! Outfit Synthesis through Automatic Optimization.
ACM Transactions on Graphics (Proceeding of SIGGRAPH Asia 2012)
Featured in SIGGRAPH Asia 2012 technical papers video trailer
Project Page, Paper (High Res), Paper (Low Res), Bibtex
In addition to dressing someone up nicely, one can also apply the technique to avoid having repeated guys in
video game scenes, e.g. GTA
|
|
Lap-Fai Yu, Sai-Kit Yeung, Chi-Keung Tang, Demetri Terzopoulos,
Tony F. Chan, Stanley J. Osher
Make it Home: Automatic Optimization of Furniture Arrangement.
ACM Transactions on Graphics (Proceeding of SIGGRAPH 2011)
Project Page, Paper, Bibtex
This idea is rooted from the old days when I was tired of the monotonous interior layouts of role-playing game worlds. Coincidentally, our goal fitted perfectly well with SIGGRAPH 2011's slogan, Make it home.
|
|
Lap-Fai Yu
3D Reconstruction and Synthesis of Facial Expressions using a Manifold Alignment Framework.
M.Phil. Thesis, Department of Computer Science and Engineering, HKUST, 2009
Advisor: Dit-Yan Yeung
My first attempt to apply data-driven techniques to computer graphics, in this case, to train a controller to synthesize 3D facial expression. The inspiration was from the dull faces I always saw in video games.
|