Jump to: Photos · Principal Investigator · Current PhD Students · MS Students · Undergrads · Collaborators · PhD/Postdoc Alumni · MS Alumni · BS Alumni · Theses · Funding

photos — scroll →

Welcome to the Perception, Action, & Learning research group at UPenn (PennPAL). While AI has become ubiquitous on our screens, it largely remains absent from our real, physical world. Two important blockers stand in the way: robots are difficult to train for new tasks, and they struggle to operate under the strict size, weight, and power constraints of physical embodiment. Our research addresses both barriers, spanning robot learning and computer vision with two core focus areas: (1) making robots easier to teach, and (2) co-designing their hardware and software for resource-efficiency.

To scale robot learning to new robots, tasks, and environments, we leverage foundation models to automate the tedious aspects of robotic engineering. We build autonomous loops where AI writes the code for reward functions, simulation environments, hardware configurations, and controllers. By removing humans from the loop, these frameworks allow robots to rapidly master complex locomotion and intricate manipulation.

Computational efficiency is just as vital. We investigate the minimal information a robot actually needs to sense and retain at each stage of its control loop. Rather than building overly complex systems, we ask a fundamental question: How should we co-design a robot’s physical form, sensor suite, compute hardware, and software for a specific task? Our work opens new avenues toward minimalistic, dramatically more resource-efficient robot learners of the future.

Note: A more complete research statement, written in August 2025, is here.

Some recent talks that might give you more of a sense of what we have been working on over the years:

• What Information Does a Robot Learner Need, Penn ASSET AI Mixer (15 mins starting at 2:03:00, Sep 2025)
• Engineering Better Robot Learners, GRASP Research Overviews (7 mins, Sep 2025)
• Tapping Foundation Models for Representations, Rewards, and Policies, ICRA Foundation Models and Neurosymbolic Systems Workshop (25 mins, May 2025)
• Engineering Better Robot Learners, Worldwide Robotics Workshop (2 mins, Apr 2025)
• Exploring and Exploiting Robot Learning, Michigan AI Symposium (35 mins, Oct 2024)
• How to Train Your Robot, Nemertes “What’s Next” Industry Conference Talk (for a general audience) (40 mins, Sep 2024)
• Polyglot Robots, MIT Embodied Intelligence Seminar (60 mins, Mar 2023)
• Scaling Vision-Based Robot Learning, GRASP Faculty Research Talk (25 mins, Aug 2022)
• Scaling Vision-Based RL for Robotics, MIT Computational Sensorimotor Learning Seminar (55 mins, Feb 2022)
• Perception Through Action and For Action, AAAI New Faculty Highlight Talk (30 mins, Feb 2021)

Principal Investigator

• Dinesh Jayaraman

Current PhD Students

• Edward Hu
• Aurora Qian
• Leon Kim (co-advised with Michael Posa)
• Pratik Kunapuli (co-advised with Vijay Kumar)
• Junyao Shi
• Arjun Krishna
• Chris Watson (co-advised with Rajeev Alur)
• Sagnik Anupam (co-advised with Osbert Bastani)
• Subin Kim
• Aileen Liao (co-advised with Michael Posa)

Current MS Students

• Arjun Arasappan (BS+MS)
• Brandon Yang
• Carlos Vega
• Kaitian Chao
• Qinhe Peng
• Selina Wan

Current Undergraduate Students

• Sam Wang
• Lilian Li

External Collaborators

• Tianyu Li (PhD student advised by Nadia Figueroa)
• Long Le (PhD student advised by Eric Eaton)
• Jiani Huang (PhD student advised by Mayur Naik)
• Yifei Simon Shao (PhD student advised by Vijay Kumar)
• Yunzhou Tim Song (PhD student advised by Kostas Daniilidis)
• Siddharth Ancha (Research Scientist, UC Berkeley)
• Vincent Pacelli (Postdoctoral Fellow, Georgia Tech)

PhD and Postdoc Alumni

MS Alumni

BS and Visiting Alumni

Research Theses Supervised

PhD

• Jason Ma (2025): Foundation Reward Models for Robot Learning (co-advised with Osbert Bastani)

MS

• Tony Wang (2026): Evaluating, Enhancing, and Explaining Robotics Foundation Models (co-advised with Kostas Daniilidis)
• Anh-Quan Pham (2026): Automatically Improving Simulation Physics for Articulated Objects
• Hungju “Johnny” Wang (2025): Foundation Models for Real-World Robotics
• James Springer (2024): Leveraging Privileged Information for Sample-Efficient Reinforcement Learning

BS

• Will Liang (2025): Correspondence-Driven Trajectory Warping for Data-Efficient Imitation and Autonomous Play
• Fiona Luo (2025): Student-Aligned Teachers for Policy Distillation
• Chuning Zhu (2021): Deep Reinforcement Learning via State Marginal Estimation

Funding & Support

Our work is possible thanks to the support of: