Embodied Spatio-Temporal AI for Human-centric Robot Autonomy

The ability to build an actionable representation of the environment of a robot is crucial for autonomy and prerequisite to a large variety of applications, ranging from home, service, care and consumer robots to autonomous vehicles, augmented reality, and disaster response. Notably, much of this depends on long-term operation in human-centric domains that are complex, widely diverse, and highly dynamic.

This talk presents an autonomy pipeline to address these challenges. At the core, I argue that time, and thus memory, dynamics, and adaptation, should be an integral component of AI systems. I will introduce methods to capture the present and past of complex and dynamic scenes through symbolic abstractions, which further facilitate predicting future scene outcomes. I will show how robots as embodied agents can leverage our actionable scene representations and predictions to complete tasks such as actively gathering data that helps them improve their scene models and perception capabilities, and how all these tools can combine for robots to fully autonomously learn over time.
The presented methods are demonstrated on-board fully autonomous aerial and ground robots, run in real-time on the limited hardware available, and are released as open-source software.

Biosketch

Lukas Schmid is a Tenure-Track Professor of Machine Intelligence at UTN. 

Before that, he was a Research Scientist and Postdoctoral Fellow at the SPARK Lab led by Prof. Luca Carlone at MIT, and a Postdoctoral Researcher at the Autonomous Systems Lab (ASL) led by Prof. Roland Siegwart at ETH Zürich. He earned his PhD in 2022 from ASL at ETHZ, where he also was a visiting researcher at the Microsoft Spatial AI Lab led by Prof. Marc Pollefeys.

His work has been recognized by several honors, including RSS Pioneers 2025, NOKOV New Generation Star at IROS 2025, the RSS Outstanding Systems Paper Award 2024, two ETH Medals for outstanding PhD and M.Sc. Theses, the Willi Studer Prize for the best graduate of the year at ETHZ, the first place in the 2024 Hilti SLAM challenge, and a Swiss National Science Foundation (SNSF) Postdoc Fellowship.

His research focuses on embodied spatio-temporal AI for human-centric robot autonomy. This includes research on scene representations and abstraction, on detection, prediction, and understanding of moving and changing entities, active perception and information gathering, as well as lifelong learning for continuous adaptation to the robot environment, embodiment, task, and human preference.