Zichen "Charles" Zhang
zcczhang1 at gmail dot com
I'm the Member of the Technical Staff at Skild AI Inc. supervised by Deepak Pathak and Abhinav Gupta. Previously, I was the Predoctoral Young Investigators (PYI) on the PRIOR team at AI2 supervised by Luca Weihs. I graduated from Macalester College in Dec. 2022 advised by Prof. Andrew Beveridge and Susan Fox.
My research lies at the intersection of vision, learning, and robotics.
While I can't yet share the work I'm most excited about at Skild AI, I've been working on feed-forward 3D/4D foundation models in my spare time. As of October 19, 2026, I've achieved SoTA, even against methods published just days ago, across a wide range of 3D/4D vision tasks: offline 3D reconstruction (sparse, dense, and long-range pose/depth estimation), online streaming reconstruction, metric 3D reconstruction, dense 3D point tracking and SE(3) estimation, feed-forward 3D segmentation, panoramic depth estimation, stereo depth estimation, multi-view scene and mesh reconstruction, robotic task adaptation (RoboCasa, RoboDojo), RGB-only navigation, and more... Most results outperform previous SoTA by a large margin across multiple evaluation protocols and datasets, on every metric. Since my bandwidth for side projects is limited, if you're interested in collaborating, please reach out!
Google Scholar / GitHub / Twitter (X) / Email / IM
- 05/2025: I've joined Skild AI - grounding my research in the real world!
- 11/2024: PoliFormer is accepted at CoRL 2024 (all strong accepts), and won the Outstanding Paper Award 🎉
- 07/2024: Release PoliFormer, an RGB-only indoor navigation agent trained end-to-end with on-policy RL that generalizes to the real world without adaptation, and beats previous SoTA with an improvement of up to 28.5% across LoCoBot and Stretch RE-1.
- 04/2024: Unified-IO 2 is accepted at CVPR 2024, and selected as Highlight. My first time received 5-5-5 (max 5) review scores!
- 04/2024: UVD is accepted at ICRA 2024, and selected as the IEEE ICRA Best Paper Award in Robot Vision Finalist.
- 01/2024: Release Unified-IO 2, the first autoregressive multimodal model that is capable of understanding and generating images, text, audio, and action, unifying over 200 understanding and generation tasks with emergent abilities.
- 09/2023: Release Universal Visual Decomposer (UVD), an off-the-shelf method immediately enables RL from vision without reward engineering and compositional generalizations in IL for long-horizon manipulation tasks in sim and real, without any extra training, data, task knowledge, and costs.
- 04/2023: VIMA is accepted at ICML 2023.
- 04/2023: Release When Learning Is Out of Reach, Reset: towards only request interventions when necessary, with generalizations.
- 12/2022: I've completed my undergraduate studies and join PRIOR @ AI2 as Predoctoral Young Investigator (PYI)!
- 10/2022: Release VIMA, a single agent can solve visual goal, one-shot video imitation, novel concept grounding, visual constraint, etc.