Hung-Jui (Joe) Huang

I am a 5th-year Ph.D. student at CMU RI, co-advised by Prof. Michael Kaess and Prof. Wenzhen Yuan. I am generally interested in robotics, computer vision, and machine learning. My research is about superhuman tactile sensing, including actively estimating physical properties and creating high-resolution 3D reconstructions. Recently, I completed a six-month internship at Google DeepMind in London, where I worked on multimodal vision-language-action (VLA) models for contact-rich manipulation tasks.

I received my B.Sc. in EECS at MIT. During my undergrad, I worked with Prof. Gregory Stein and Prof. Nicholas Roy. After B.Sc., I worked at ISEE for two years with Dr. Chris Baker and Prof. Ying Nian Wu. In the past, I have worked on robot navigation and self-driving trucks.

I am named a Presidential Fellow at CMU and my research is supported by the Paul and James Wang/Sercomm Presidential Scholarship.

Email / CV (May 25) / Scholar / Github

DeepMind
Student Researcher
July 25 - Jan. 26

CMU
Ph.D. in RI
Aug. 21 - Present

ISEE
Research Engineer
Aug. 19 - May 21

Nvidia
Intern
Jun. 18 - Aug. 18

MIT
B.Sc. in EECS
Sep. 14 - Jun. 19

Publications

	GelSLAM: A Real-Time, High-Fidelity, and Robust 3D Tactile SLAM System Hung-Jui Huang, Amin Mirzaee, Michael Kaess, Wenzhen Yuan Under Review project page / paper / code / dataset / bibtex Extremely high-fidelity, real-time 3D reconstruction using only tactile input.
	GelBelt: A Vision-based Tactile Sensor for Continuous Sensing of Large Surfaces Amin Mirzaee, Hung-Jui Huang, Wenzhen Yuan RA-L, 2024 project page / paper / video / bibtex Our tactile sensor is designed for efficient, continuous scanning over large surfaces.
	NormalFlow: Fast, Robust, and Accurate Contact-based Object 6DoF Pose Tracking with Vision-based Tactile Sensors Hung-Jui Huang, Michael Kaess, Wenzhen Yuan RA-L, 2024 Best Poster Award, ICRA 2025 Dexterity in Multi-Fingered Hands Workshop project page / paper / code / dataset / bibtex Real-time, robust, and accurate tactile-based tracking for novel objects, including low-textured ones like ping pong balls and eggs. Useful for manipulation, in-hand manipulation, and 3D reconstruction tasks.
	FusionSense: Bridging Common Sense, Vision, and Touch for Robust Sparse-View Reconstruction Irving Fang, Kairui Shi, Xujin He, Siqi Tan, Yifan Wang, Hanwen Zhao, Hung-Jui Huang, Wenzhen Yuan Chen Feng, Jing Zhang, ICRA, 2025 project page* / paper / code / bibtex Robot reconstructing visually and geometrically accurate surroundings with sparse visual and tactile data.
	An Intelligent Robotic System for Perceptive Pancake Batter Stirring and Precise Pouring Xinyuan Luo, Shengmiao Jin, Hung-Jui Huang, Wenzhen Yuan [IROS Best Entertainment and Amusement Papers Finalists] IROS, 2024 project page / video / paper / bibtex Our pancake preparation robot integrates advanced sensory and control algorithms to mix batter with precise uniformity, estimate its properties, and pour it into specified shapes.
	Kitchen Artist: Precise Control of Liquid Dispensing for Gourmet Plating Hung-Jui Huang, Jingyi Xiang, Wenzhen Yuan ICRA, 2024 project page / video / paper / bibtex Our sauce plating robot can precisely control the thickness of squeezed liquids on a surface, even when dealing with unseen liquids.
	Estimating Properties of Solid Particles Inside Container Using Touch Sensing Xiaofeng Guo, Hung-Jui Huang, Wenzhen Yuan IROS, 2023 paper / project page / bibtex Estimating solid particle properties (e.g., size and shape) inside a container using tactile and force-torque sensing.
	Understanding Dynamic Tactile Sensing for Liquid Property Estimation Hung-Jui Huang, Xiaofeng Guo, Wenzhen Yuan RSS, 2022 video / project page / paper / bibtex High-precision liquid viscosity and volume estimation using only tactile sensing. Our approach can even estimate sugar concentration within water based on slight viscosity variations.
	Planning on a (Risk) Budget: Safe Non-Conservative Planning in Probabilistic Dynamic Environments Hung-Jui Huang, Kai-Chi Huang, Michal Čáp, Yibiao Zhao, Ying Nian Wu, Chris L. Baker ICRA, 2021 video / paper / bibtex A planning algorithm with guaranteed bounds on the probability of safety violation, which nonetheless achieve non-conservative performance. Tested on a self-driving truck in a real-world environment.
	Omnidirectional CNN for Visual Place Recognition and Navigation Tsun-Hsuan Wang, Hung-Jui Huang, Juan-Ting Lin, Chan-Wei Hu, Kuo-Hao Zeng, Min Sun (* indicates equal contribution) ICRA, 2018 project page / video / code / paper / bibtex O-CNN compares omnidirectional visual images to a database of images to determine the robot's location and helps navigation.