ABOUT ME!
I am a Machine Learning Research Scientist at Toyota Research Institute working on Large Behavioral Models and Deep-learning based 3D perception systems for Robotics. I received my PhD in the George W. Woodruff School of Mechanical Engineering at Georgia Institute of Technology. I was advised by Dr. Zsolt Kira from the Robotics, Perception and Learning (RIPL) Lab. My PhD thesis is titled Learning 3D Robotics Perception using Inductive Priors. My thesis is available here and the dissertation defense video here. My current research focuses on 3D perception, GenAI and Multimodal AI and covers the following topics:
- 3D Perception: 3D Reconstruction, NeRF, Gaussian Splatting, Representation Learning, 6D pose estimation
- Generative AI for Robotics: Diffusion Models for Pose Estimation, Data Augmentation for Policy Learning
- Multimodal AI : Vision-and-Language, Embodied AI, Semantic Understanding, Spatio-temporal learning
I’m a technical reviewer for Machine Learning and Robotics Conferences including CVPR, ECCV, ICCV, ICLR, Neurips, ICRA, IROS, RSS and Siggraph and the lead co-organizer of RoboNeRF Workshop at ICRA’24 and Robo 3D-VLM workshop at CVPR’25! Below you will find my projects portfolio. You can find my updated resume here.
Affiliations
TRI
2021 – Present
Georgia Tech
2017 – Present
Fulbright
2017 – 2019
SRI International
Summer 2020
GIKI
2011-2015
NEWS
[Apr 2025]
Released Posed-DROID, improved camera calibration for the DROID dataset. Updated DROID paper, blog post and annotations!
[Feb 2025]
Two CVPR 2025 papers accepted on Zero-shot novel-view and depth synthesis and Zero-shot robotics grasping!
[Dec 2024]
CVPR Workshop accepted on 3D Vision Language Models (VLMs) for Robotics. Call for papers coming soon!
[Dec 2024]
Invited talk at Woven by Toyota (Robotics) on Learning 3D Robotics Perception using Inductive Priors!
[Jun 2024]
Three IROS’24 papers on Diffusion-based 6D Pose Estimation, Language-embedded Gaussian Splat and Interactive Perception!
[May 2024]
Our paper Open X-Embodiment, a large-scale robotics learning dataset collaboration, got best paper awards at ICRA 2024!
[Jun 2024
Attending CVPR’24 remotely. We will present two workshop posters, NeRF-MAE and ICE-Gaussian!
[May 2024]
NeRF-MAE and ICE-Gaussian accepted to CVPR Neural Rendering Intelligence and AI4CC Workshops.
[May 2024]
Attended ICRA’24 in Yokohama, Japan to help present FSD and co-organize RoboNerF Workshop
[Mar 2024]
Gave an Invited talk at Stanford’s Computer Vision: Foundations and Applications class on Neural Fields in Vision and Beyond
[Jan 2024]
Gave an invited talk at Shuran Song’s Robot Perception Class at Stanford. Topic: Neural Fields in Robotics and beyond.
[Dec 2023]
Passed my PhD defense and received my doctorate! My thesis is titled “Learning 3D Robotics Perception using Inductive Priors”
[Jul 2023]
Our Paper, NeO 360, accepted to ICCV’23! Grateful to have trio of papers accepted to ECCV, CVPR and now ICCV
[Jun 2023]
Gave invited talks on Neural Fields in Robotics (Part 1 and 2) at 3D Deep Learning Reading Group
[Apr 2023]
Passed my PhD proposal defense titled ‘Inductive biases for object and agent-centric neural 3D scene representations’
[Apr 2023]
Guest lecture at Georgia Tech’s Deep learning Class on ‘Learning Object-centric Centric Neural 3D Scene Representations’
[Jan 2022]
Started my second internship at Toyota Research Institute, with Machine Learning team in Bay Area, California
[Jul 2021]
Started my first internship at Toyota Research Institute, with Robotics perception team in Bay Area, California.
[Aug 2019]
The beginning of my PhD program
FEATURED PUBLICATIONS
(For a complete list, please see my Google Scholar Profile)
Posed-DROID: Scaling-Up Automatic Camera Calibration for DROID dataset
Muhammad Zubair Irshad, Vitor Guizilini, Alexander Khazatsky, Karl Pertsch
Part of DROID paper, published at Robotics: Science and System, RSS 2024
Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion
Vitor Guizilini, Muhammad Zubair Irshad, Dian Chen, Greg Shakhnarovich, Rares Ambrus
Computer Vision and Pattern Recognition, CVPR 2025
Real2Render2Real: Scaling Robotic Manipulation Data Without Dynamics Simulation or Robot Hardware
Justin Yu, Letian Fu, Huang Huang, Karim El-Refai, Rares Andrei Ambrus, Richard Cheng, Muhammad Zubair Irshad, Ken Goldberg
arXiv 2025
FastMap: Revisiting Dense and Scalable Structure from Motion
Jiahao Li, Haochen Wang, Muhammad Zubair Irshad, Igor Vasiljevic, Matthew R. Walter, Vitor Campagnolo Guizilini, Greg Shakhnarovich
arXiv 2025
Neural Fields in Robotics: A Survey
Muhammad Zubair Irshad, Mauro Comi, Yen-Chen Lin, Nick Heppert, Abhinav Valada, Zsolt Kira, Rares Ambrus, Johnathan Trembley
In Submission, 2025
NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fields
Muhammad Zubair Irshad, Sergey Zakharov, Vitor Guizilini, Adrien Gaidon, Zsolt Kira, Rares Ambrus
European Conference on Computer Vision, ECCV 2024
CVPR Neural Rendering Intelligence Workshop, 2024
ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Grasping
Shun Iwase, Muhammad Zubair Irshad, Katherine Liu, Vitor Guizilini, Robert Lee, Takuya Ikeda, Amma Ayako, Koichi Nishiwaki, Kris Kitani, Rares Ambrus, Sergey Zakharov
Computer Vision and Pattern Recognition, CVPR 2025
POGS: Persistent Object Gaussian Splat for Tracking Human and Robot Manipulation of Irregularly Shaped Objects
Justin Yu, Kush Hari, Karim El-Refai, Arnav Dalil, Justin Kerr, Chung-Min Kim, Richard Cheng, Muhammad Zubair Irshad, Ken Goldberg
International Conference on Robotics and Automation, ICRA 2025
RoVi-Aug: Robot and Viewpoint Augmentation for Cross-Embodiment Robot Learning
Lawrence Chen*, Chenfeng Xu*, Karthik Dharmarajan, Muhammad Zubair Irshad, Richard Cheng, Kurt Keutzer, Masayoshi Tomizuka, Quan Vuong, Ken Goldberg
Conference on Robot Learning, CoRL 2024 (Oral Presentation, Top 4.3%)
Language-Embedded Gaussian Splats (LEGS): Incrementally Building Room-Scale Representations with a Mobile Robot
Justin Yu*, Kush Hari*, Kishore Srinivas*, Adam Rashid, Chung Min Kim, Justin Kerr, Richard Cheng, Muhammad Zubair Irshad, Ashwin Balakrishna, Thomas Kollar, Ken Goldberg
International Conference on Intelligent Robots and System, IROS 2024
DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset
Alexander Khazatsky*, Karl Pertsch*, Suraj Nair, Ashwin Balakrishna … Muhammad Zubair Irshad et al.
Robotics: Science and Systems, RSS 2024
DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation
Takuya Ikeda, Sergey Zakharov, Tianyi Ko, Muhammad Zubair Irshad, Robert Lee, Katherine Liu, Rares Ambrus, Koichi Nishiwaki
International Conference on Intelligent Robots and System, IROS 2024
ECCV Workshop on Recovering 6D Object Pose, 2024
Open X‑Embodiment: Robotic Learning Datasets and RT‑X Models
Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, Abhinav Gupta, … Muhammad Zubair Irshad et al.
IEEE International Conference on Robotics and Automation, ICRA 2024 (Best Paper Award)
FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects
Mayank Lunayach, Sergey Zakharov, Dian Chen, Rares Ambrus, Zsolt Kira, Muhammad Zubair Irshad
International Conference on Robotics and Automation, ICRA 2024
ICE-G: Image Conditional Editing of 3D Gaussian Splats
Vishnu Jaganathan, Hannah Huang, Muhammad Zubair Irshad, Varun Jampani, Amit Raj, Zsolt Kira
CVPR AI for Content Creation Workshop, 2024
Learning 3D Robotics Perception using Inductive Priors
Muhammad Zubair Irshad
PhD Thesis, Georgia Institute of Technology 2023
NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes
Muhammad Zubair Irshad, Sergey Zakharov, Katherine Liu, Vitor Guizilini, Thomas Kollar, Adrien Gaidon, Zsolt Kira, Rares Ambrus
International Conference on Computer Vision, ICCV 2023
CARTO: Category and Join Agnositc Reconstruction of Articulated Objects
Mayank Lunayach, Sergey Zakharov, Dian Chen, Rares Ambrus, Zsolt Kira, Muhammad Zubair Irshad
Computer Vision and Pattern Recognition, CVPR 2023
ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimization
Muhammad Zubair Irshad*, Sergey Zakharov*, Rares Ambrus, Thomas Kollar, Zsolt Kira, Adrien Gaidon
European Conference on Computer Vision, ECCV 2022
CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimation
Muhammad Zubair Irshad, Thomas Kollar, Michael Laskey, Kevin Stone, Zsolt Kira
IEEE International Conference on Robotics and Automation, ICRA 2022
Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation
Muhammad Zubair Irshad, Chih-Yao Ma, Zsolt Kira
IEEE International Conference on Robotics and Automation, ICRA 2021
SASRA: Semantically-aware Spatio-Temporal Reasoning Agent for Vision-and-Language Navigation
Muhammad Zubair Irshad, Niluthpol Mithun, Zachary Seymour, Han-Pang Chiu, Supun Samarasekera, Rakesh Kumar
International Conference on Pattern Recognition, ICPR 2022
MACHINE LEARNING WORKSHOPS
RoboNerF: 1st Workshop On Neural Fields In Robotics
Muhammad Zubair Irshad, Nick Heppert, Jonathan Tremblay, Shreyas Kousik, Zsolt Kira, Abhinav Valada
IEEE International Conference on Robotics and Automation (ICRA), 2024
3D Vision Language Models (VLMs) for Robotics Manipulation: Opportunities and Challenges
Jiafei Duan, Muhammad Zubair Irshad, Ishika Singh, Vitor Guizlini, Rares Ambrus, Zsolt Kira
Computer Vision and Pattern Recognition (CVPR), 2025
MACHINE LEARNING SOFTWARE & REPO
Deep Reinforcement Learning based control of complex robotic agents
Details
Embodied Visual Navigation in Habitat
Details
Learning inverse dynamics of 7-DOF Robot Arm
Details
Complex robot maze navigation using image classification and ROS
Details
Vehicle Control for Autonomous Driving
Details
Environment perception stack for Self Driving Cars
Details
Visual Odometry for Autonomous Driving
Details