Zichen "Charles" Zhang

charlesz at allenai dot org

I am Predoctoral Young Investigators (PYI) on the PRIOR team at AI2 supervised by Luca Weihs. Previously, I graduated from Macalester College in Dec. 2022 advised by Prof. Andrew Beveridge and Susan Fox.

My research lies at the intersection of vision, learning, and robotics.

Google Scholar  /  GitHub  /  Twitter (X)  /  Email  /  IM

News
  • 11/2024: PoliFormer is accepted at CoRL 2024 (all strong accepts), and won the Outstanding Paper Award 🎉
  • 07/2024: Release PoliFormer, an RGB-only indoor navigation agent trained end-to-end with on-policy RL that generalizes to the real world without adaptation, and beats previous SoTA with an improvement of up to 28.5% across LoCoBot and Stretch RE-1.
  • 04/2024: Unified-IO 2 is accepted at CVPR 2024, and selected as Highlight. My first time received 5-5-5 (max 5) review scores!
  • 04/2024: UVD is accepted at ICRA 2024, and selected as the IEEE ICRA Best Paper Award in Robot Vision Finalist.
  • 01/2024: Release Unified-IO 2, the first autoregressive multimodal model that is capable of understanding and generating images, text, audio, and action, unifying over 200 understanding and generation tasks with emergent abilities.
  • 09/2023: Release Universal Visual Decomposer (UVD), an off-the-shelf method immediately enables RL from vision without reward engineering and compositional generalizations in IL for long-horizon manipulation tasks in sim and real, without any extra training, data, task knowledge, and costs.
  • 04/2023: VIMA is accepted at ICML 2023.
  • 04/2023: Release When Learning Is Out of Reach, Reset: towards only request interventions when necessary, with generalizations.
  • 12/2022: I've completed my undergraduate studies and join PRIOR @ AI2 as Predoctoral Young Investigator (PYI)!
  • 10/2022: Release VIMA, a single agent can solve visual goal, one-shot video imitation, novel concept grounding, visual constraint, etc.
  • Research (Highlighted / All)
    (* indicates equal contribution, indicates equal advising)

    Loading...
    Service
    Reviewer
  • NeurIPS 2024, ICRA 2024, CVPR 2024, NeurIPS 2023, CoRL 2023
  • Talks
  • 05/2024: "Universal Visual Decomposer" and "Unified-IO 2" at Tsinghua University hosted by Prof. Huaping Liu.
  • 01/2024: "Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action" at Google Deepmind with Jiasen Lu, hosted by Fei Xia and Brian Ichter .
  • 04/2023: "Autonomous Visuomotor Reinforcement Learning" at Ranjay Krishna group.
  • Teaching
  • TA at Macalester College : Comp 484 Intro to Artificial Intelligence, MATH 378 Complex Analysis, STAT/COMP 112 Intro to Data Science, COMP 394 Topics Course: Reinforcement Learning, and COMP 128 Data Structure
  • Others

    Home  /  Posts

    Pageviews


    Inspired by this and this.