Back
2026INF005.pdf
Doctoral thesis

Learning on the fly : from domain adaptation to on-device learning on autonomous nano-UAVs

  • 2026

PhD: Università della Svizzera italiana

Autonomous nano-UAV
Artificial intelligence
TinyML
Domain adaptation
On-device learning
Self-supervised learning
Ultra-low power embedded systems
English Autonomous pocket-sized unmanned aerial vehicles (nano-UAVs) are exciting testbeds for embodied Tiny Machine Learning (TinyML) algorithms. Thanks to their compact form factor (~10cm diameter, sub-50g), nano-UAVs excel at navigating tight spaces, safely operating near humans, and acting as ubiquitous intelligent sensors. However, nano-UAVs come with strict payload and power budgets, i.e., a few grams and a few hundred mW, restricting onboard processors to resource-constrained embedded microcontroller units. Despite recent advances in nanorobotics research, autonomous nano-UAVs still struggle to operate reliably in challenging real-world scenarios beyond controlled laboratory environments. TinyML approaches suffer from domain shift, i.e., perception models that work well in their training domain and controlled settings often degrade sharply in new deployment domains. This Ph.D. Thesis aims to push intelligent nano-UAVs out of the lab and closer to the real world. We demonstrate advancements spanning the full autonomous stack of our nano-UAV platform and showcase their benefits on real-world applications, including human pose estimation, monocular depth estimation, and autonomous drone racing. This Thesis revolves around three main contributions. First, a review of existing research on nano-UAVs highlights gaps and significant functional inefficiencies in their software infrastructure, making them suboptimal for hosting complex, high-performance, real-time applications. Therefore, we propose a performance-optimized application framework for AI-based autonomous nano-UAVs that enables researchers to prototype new applications faster than before, while simplifying efficient use of precious hardware resources. Further, we demonstrate that our framework directly translates to measurable improvements in a set of real-world applications. Building upon our optimized software infrastructure, we study training strategies for vision-based TinyML perception models to reliably generalize under domain shift, even with limited training data. Due to limited model capacity and scarce real-world training data, these models often generalize poorly beyond controlled settings. High-quality training labels typically require collecting data from expensive infrastructure (e.g., motion capture systems) or accurate simulators, making the ideal training dataset very expensive and even impossible for all those cases in which the deployment scenario is not known a priori. To cope with this challenging scenario, we introduce three train-time techniques to mitigate domain shift in both novel unseen environments and drone attitude changes, and we demonstrate their benefits through end-to-end in-field experiments. Then, we further extend our methodology to mitigate domain shift through on-device self-supervised learning on resource-constrained microcontroller units (MCUs): a more powerful and general concept than training-time techniques, as it dynamically adapts a robot's models in the field. To enable effective on-device learning on MCUs, we tackle several challenges: i) how to provide supervision information for the onboard training (i.e., fine-tuning); ii) given ultra-tight onboard memories, we need decide which data---and how many---are more valuable for our fine-tuning; iii) finally, we need to identify the best backpropagation strategy compatible with the limited battery lifetime of the nano-UAV. To the best of our knowledge, our contribution marks the first demonstrations of on-device self-supervised learning on a nano-robotic platform. Our contributions advance the level of autonomous intelligence achievable by nano-UAVs in real-world conditions. The proposed application framework is being integrated into the official software of the leading nano-UAV manufacturer, demonstrating its relevance and maturity. Our know-how enabled us to win the first international nano-UAV AI competition with a fully autonomous vision-based sim-to-real obstacle avoidance system, successfully transferring from simulation to the physical race arena with no prior exposure to real-world data. Altogether, our results lay the groundwork for self-improving nano-robotics, capable of autonomously adapting to new and unknown environments under severe resource constraints.
Collections
Language
  • English
Classification
Computer science and technology
License
License undefined
Open access status
green
Identifiers
Persistent URL
https://n2t.net/ark:/12658/srd1335227
Statistics
Document views: 0 File downloads:
  • 2026INF005.pdf: 0