How machines perceive, reconstruct, and reason about the 3D world — from classical geometry and image formation, through modern neural rendering (NeRF and 3D Gaussian Splatting), to lifting semantics into 3D and agents that act through world models.
This module follows a single thread: turning 2D observations into structured, queryable, and actionable 3D. We begin with how images are formed and the classical ways of representing 3D geometry, move through depth estimation and multi-view reconstruction and state estimation (Structure-from-Motion and SLAM), and then into the neural rendering methods: volumetric fields, NeRF, and 3D Gaussian Splatting, that now define the field.
The final weeks lift language and semantics into these 3D representations for open-set queries, and connect perception to action through active perception and world models. Teaching combines lectures with hands-on practicals: PyTorch3D, COLMAP, NeRFStudio, language-embedded Gaussians, and a DreamerV3 agent, and the module is assessed through a short quiz and an end-to-end group project. It is one module of the 12 CFU Elective in Artificial Intelligence.
A few glimpses of what we will build together. These are only a taste of what the module covers.
The same office scene reconstructed two ways: a classical mesh and rendering (3D Gaussian Splatting). Drag the slider to wipe between them.
Lifting language into the map: the robot is given an object described in free-form text and actively explores to find and map it, deciding where to look next from open-vocabulary semantics.
Choosing where to look to localize reliably: your robot should see only places that help it localize! Selecting viewpoints that maximize camera pose-estimation accuracy.
Six weeks, three sessions per week, with a hands-on practical each week. Slides are linked as PDF after each lecture. SCHEDULE CAN BE SLIGHTLY CHANGED.
Assessment has two parts: a quiz (13 questions, max 10 points) and a group project built on the practical sessions (max 20 points, plus up to +3 bonus for reimplementing something from a paper method or a top-3 leaderboard finish).
The quiz is held on the last day of lessons, with further sittings in the standard exam sessions. The project can be presented in a 10-minute talk at the end of December or during standard exam sessions.
This grade counts toward the overall mark of the 12 CFU Elective in Artificial Intelligence.
