About         Portfolio         Resume

Zubair Irshad
 

ZUBAIR IRSHAD

Research Scientist 
Toyota Research Institute 

ABOUT ME!

I am a Machine Learning Research Scientist at Toyota Research Institute working on deep-learning based 3D perception systems for robotics. I received my PhD in the George W. Woodruff School of Mechanical Engineering at Georgia Institute of Technology. I was advised by Dr. Zsolt Kira from the Robotics, Perception and Learning (RIPL) Lab. My PhD thesis is titled Learning 3D Robotics Perception using Inductive Priors. My thesis is available here.

My current research focuses on 3D perception, Scene Understanding and Embodied AI and covers topics such as neural implicit reconstruction (i.e. NeRF, Gaussian Splatting), efficient 3D object detection, 6D pose estimation and visual embodied navigation. Feel free to contact me to talk anything related to 3D Vision, Robotics, NeRFs, Deep Learning or related to some of my projects. Below you will find my projects portfolio. You can find my resume here.

Affiliations

Toyota Research Institute
2020-Present

Georgia Tech
2017-Present

Fulbright
2017-2019

SRI International
Summer 2020

GIKI
2011-2015

NEWS

[Mar 2024]

[Jan 2024]

Started as a Research Scientist at Toyota Research Institute in Bay Area, California.

[Jan 2024]

Gave an invited talk at Shuran Song’s Robot Perception Class at Stanford. Topic: Neural Fields in Robotics and beyond.

[Jan 2024]

Our paper FSD accepted to ICRA’24.

[Dec 2023]

Our workshop Neural Fields in Robotics accepted to ICRA’24! Call for papers live here.

[Dec 2023]

Passed my PhD defense and received my doctorate! My thesis is titled “Learning 3D Robotics Perception using Inductive Priors”

[Jul 2023]

[Jul 2023]

Our Paper, NeO 360, accepted to ICCV’23! Grateful to have trio of papers accepted to ECCV, CVPR and now ICCV

[Jul 2023]

Awesome Implicit NeRF Robotics reached 800 stars on Github ⭐

[Jul 2023]

Served as a reviewer and reviewed 19 papers this year, so far for NeurIPS’23CVPR’23ICCV’23 and ICRA’23

[Jun 2023]

Attended CVPR 2023, Virtually. (Poster presentation of our paper, CARTO)

[Jun 2023]

Gave invited talks on Neural Fields in Robotics (Part 1 and 2) at 3D Deep Learning Reading Group

[Apr 2023]

Started as a mentor at Fatima Fellowship, supported by Huggingface

[Apr 2023]

Passed my PhD proposal defense titled ‘Inductive biases for object and agent-centric neural 3D scene representations’

[Apr 2023]

Guest lecture at Georgia Tech’s Deep learning Class on ‘Learning Object-centric Centric Neural 3D Scene Representations’

[Feb 2023]

Our paper, CARTO, on fast articulated object reconstruction, accepted into CVPR’23

[Oct 2022]

Attended ECCV’22 virtually (Poster presentation of our paper, ShAPO)

[Aug 2022]

Awarded GRA Funding (with Dr. Zsolt Kira) from Toyota Research Institute for my PhD

[Jul 2022]

Our paper, ShAPO on categorical object reconstruction and 6D pose estiamation, accepted into ECCV’22

[May 2022]

Our paper, SASRA on semantic mapping for Vision-and-Language Navigation, accepted to ICPR’22

[May 2022]

Attended ICRA’22 in person. Gave a talk on our paper, CenterSnap

[Jan 2022]

Started my second internship at Toyota Research Institute, with Machine Learning team in Bay Area, California

[May 2021]

Attended ICRA’21 virtually. Gave a talk on our paper, Robo-VLN

[Jul 2021]

Started my first internship at Toyota Research Institute, with Robotics perception team in Bay Area, California.

[Jan 2021]

Our paper, Robo-VLN, accepted to ICRA’21

[May 2020]

Started summer internship at SRI International, with CVT team in Princeton, New Jersey

[Nov 2019]

Passed PhD Qualifying Exams at Georgia Tech

[Aug 2019]

The beginning of my PhD program

FEATURED PUBLICATIONS

NeRF MAE teaser

NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D representation Learning for Neural Radiance Fields

Muhammad Zubair Irshad, Sergey Zakharov, Vitor Guizilini, Adrien Gaidon, Zsolt Kira, Rares Ambrus

CVPR Neural Rendering Intelligence Workshop, 2024

DiffusionNOCS

DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation

Takuya Ikeda, Sergey Zakharov, Tianyi Ko, Muhammad Zubair Irshad, Robert Lee, Katherine Liu, Rares Ambrus, Koichi Nishiwaki

Preprint, 2024

FSD Pose teaser

FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects

Mayank Lunayach, Sergey Zakharov, Dian Chen, Rares Ambrus, Zsolt Kira, Muhammad Zubair Irshad

International Conference on Robotics and Automation, ICRA 2024

NeRO360

NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenes

Muhammad Zubair Irshad, Sergey Zakharov, Katherine Liu, Vitor Guizilini, Thomas Kollar, Adrien Gaidon, Zsolt Kira*, Rares Ambrus*

International Conference on Computer Vision, ICCV 2023

CARTO CVPR 2023

CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects

Nick Heppert, Muhammad Zubair Irshad, Sergey Zakharov, Katherine Liu, Rares Ambrus, Jeannette Bohg, Abhinav Valada, Thomas Kollar

Conference on Computer Vision and Pattern Recognition, CVPR 2023

ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimization

Muhammad Zubair Irshad*, Sergey Zakharov*, Rares Ambrus, Thomas Kollar, Zsolt Kira, Adrien Gaidon

European Conference on Computer Vision, ECCV 2022

CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimation

Muhammad Zubair Irshad, Thomas Kollar, Michael Laskey, Kevin Stone, Zsolt Kira

IEEE International Conference on Robotics and Automation, ICRA 2022

Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation

Muhammad Zubair Irshad, Chih-Yao Ma, Zsolt Kira

IEEE International Conference on Robotics and Automation, ICRA 2021

SASRA: Semantically-aware Spatio-Temporal Reasoning Agent for Vision-and-Language Navigation

Muhammad Zubair Irshad, Niluthpol Mithun, Zachary Seymour, Han-Pang Chiu, Supun Samarasekera, Rakesh Kumar

International Conference on Pattern Recognition, ICPR 2022

MACHINE LEARNING SOFTWARE & REPO

Deep Reinforcement Learning Agents

Deep Reinforcement Learning based control of complex robotic agents

Details
Habitat Point Goal Navigation

Embodied Visual Navigation in Habitat

Details

Learning inverse dynamics of 7-DOF Robot Arm

Details

Complex robot maze navigation using image classification and ROS

Details

Vehicle Control for Autonomous Driving

Details

Environment perception stack for Self Driving Cars

Details

Visual Odometry for Autonomous Driving

Details

End to end imitation learning of dynamically unstable systems

Details