ABOUT ME!
I am a Machine Learning Research Scientist at Toyota Research Institute working on deep-learning based 3D perception systems for robotics. I received my PhD in the George W. Woodruff School of Mechanical Engineering at Georgia Institute of Technology. I was advised by Dr. Zsolt Kira from the Robotics, Perception and Learning (RIPL) Lab. My PhD thesis is titled Learning 3D Robotics Perception using Inductive Priors. My thesis is available here and the dissertation defense video here. My current research focuses on 3D perception, GenAI and Multimodal AI and covers following topics:
- 3D Perception: 3D Reconstruction, NeRF, Gaussian Splatting, Representation Learning, 6D pose estimation
- Generative AI for Robotics: Diffusion Models for Pose Estimation, Data Augmentation for Policy Learning
- Multimodal AI : Vision-and-Language, Embodied AI, Semantic Understanding, Spatio-temporal learning
I’m a technical reviewer for Machine Learning and Robotics Conferences including CVPR, ECCV, ICCV, ICLR, Neurips, ICRA, IROS and Siggraph and the lead organizer of RoboNeRF Workshop at ICRA’24! Below you will find my projects portfolio. You can find my updated resume here.
Affiliations
TRI
2020 – Present
Georgia Tech
2017 – Present
Fulbright
2017 – 2019
SRI International
Summer 2020
GIKI
2011-2015
NEWS
[Jun 2024]
Three IROS’24 papers on Diffusion-based 6D Pose Estimation, Language-embedded Gaussian Splat and Interactive Perception!
[May 2024]
NeRF-MAE and ICE-Gaussian accepted to CVPR Neural Rendering Intelligence and AI4CC Workshops.
[May 2024]
Attended ICRA’24 in Yokohama, Japan to help present FSD and co-organize RoboNerF Workshop
[Mar 2024]
Gave an Invited talk at Stanford’s Computer Vision: Foundations and Applications class on Neural Fields in Vision and Beyond
[Jan 2024]
Gave an invited talk at Shuran Song’s Robot Perception Class at Stanford. Topic: Neural Fields in Robotics and beyond.
[Dec 2023]
Passed my PhD defense and received my doctorate! My thesis is titled “Learning 3D Robotics Perception using Inductive Priors”
[Jul 2023]
Our Paper, NeO 360, accepted to ICCV’23! Grateful to have trio of papers accepted to ECCV, CVPR and now ICCV
[Jun 2023]
Gave invited talks on Neural Fields in Robotics (Part 1 and 2) at 3D Deep Learning Reading Group
[Apr 2023]
Passed my PhD proposal defense titled ‘Inductive biases for object and agent-centric neural 3D scene representations’
[Apr 2023]
Guest lecture at Georgia Tech’s Deep learning Class on ‘Learning Object-centric Centric Neural 3D Scene Representations’
[Jan 2022]
Started my second internship at Toyota Research Institute, with Machine Learning team in Bay Area, California
[Jul 2021]
Started my first internship at Toyota Research Institute, with Robotics perception team in Bay Area, California.
[Aug 2019]
The beginning of my PhD program
FEATURED PUBLICATIONS
NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fields
Muhammad Zubair Irshad, Sergey Zakharov, Vitor Guizilini, Adrien Gaidon, Zsolt Kira, Rares Ambrus
European Conference on Computer Vision, ECCV 2024
CVPR Neural Rendering Intelligence Workshop, 2024
RoVi-Aug: Robot and Viewpoint Augmentation for Cross-Embodiment Robot Learning
Lawrence Chen*, Chenfeng Xu*, Karthik Dharmarajan, Muhammad Zubair Irshad, Richard Cheng, Kurt Keutzer, Masayoshi Tomizuka, Quan Vuong, Ken Goldberg
Conference on Robot Learning, CoRL 2024 (Oral Presentation)
Language-Embedded Gaussian Splats (LEGS): Incrementally Building Room-Scale Representations with a Mobile Robot
Justin Yu*, Kush Hari*, Kishore Srinivas*, Adam Rashid, Chung Min Kim, Justin Kerr, Richard Cheng, Muhammad Zubair Irshad, Ashwin Balakrishna, Thomas Kollar, Ken Goldberg
International Conference on Intelligent Robots and System, IROS 2024
DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation
Takuya Ikeda, Sergey Zakharov, Tianyi Ko, Muhammad Zubair Irshad, Robert Lee, Katherine Liu, Rares Ambrus, Koichi Nishiwaki
International Conference on Intelligent Robots and System, IROS 2024
ECCV Workshop on Recovering 6D Object Pose, 2024
FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects
Mayank Lunayach, Sergey Zakharov, Dian Chen, Rares Ambrus, Zsolt Kira, Muhammad Zubair Irshad
International Conference on Robotics and Automation, ICRA 2024
ICE-G: Image Conditional Editing of 3D Gaussian Splats
Vishnu Jaganathan, Hannah Huang, Muhammad Zubair Irshad, Varun Jampani, Amit Raj, Zsolt Kira
CVPR AI for Content Creation Workshop, 2024
NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes
Muhammad Zubair Irshad, Sergey Zakharov, Katherine Liu, Vitor Guizilini, Thomas Kollar, Adrien Gaidon, Zsolt Kira, Rares Ambrus
International Conference on Computer Vision, ICCV 2023
CARTO: Category and Join Agnositc Reconstruction of Articulated Objects
Mayank Lunayach, Sergey Zakharov, Dian Chen, Rares Ambrus, Zsolt Kira, Muhammad Zubair Irshad
Computer Vision and Pattern Recognition, CVPR 2023
ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimization
Muhammad Zubair Irshad*, Sergey Zakharov*, Rares Ambrus, Thomas Kollar, Zsolt Kira, Adrien Gaidon
European Conference on Computer Vision, ECCV 2022
CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimation
Muhammad Zubair Irshad, Thomas Kollar, Michael Laskey, Kevin Stone, Zsolt Kira
IEEE International Conference on Robotics and Automation, ICRA 2022
Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation
Muhammad Zubair Irshad, Chih-Yao Ma, Zsolt Kira
IEEE International Conference on Robotics and Automation, ICRA 2021
SASRA: Semantically-aware Spatio-Temporal Reasoning Agent for Vision-and-Language Navigation
Muhammad Zubair Irshad, Niluthpol Mithun, Zachary Seymour, Han-Pang Chiu, Supun Samarasekera, Rakesh Kumar
International Conference on Pattern Recognition, ICPR 2022
MACHINE LEARNING WORKSHOPS
RoboNerF: 1st Workshop On Neural Fields In Robotics
Muhammad Zubair Irshad, Nick Heppert, Jonathan Tremblay, Shreyas Kousik, Zsolt Kira, Abhinav Valada
IEEE International Conference on Robotics and Automation (ICRA), 2024