Skypilot: Fine-Tuning LLM with Physical Grounding for AAV Coverage Search

Zhongkai Chen¹ · Yihao Sun¹ · Chao Yan² · Han Zhou¹ ·Xiaojia Xiang¹· Jie Jiang³
¹College of Intelligence Science and Technology, National University of Defense Technology
²College of Automation Engineering, Nanjing University of Aeronautics and Astronautics
³China Academy of Launch Vehicle Technology
Zhongkai Chen and Yihao Sun contributed equally to this work.

Abstract

Autonomous aerial vehicles (AAVs) have played a pivotal role in coverage operations and search missions. Recent advances in large language models (LLMs) offer promising opportunities to augment AAV intelligence. These advances help address complex challenges like area coverage optimization, dynamic path planning, and adaptive decision-making. However, the absence of physical grounding in LLMs leads to hallucination and reproducibility problems in spatial reasoning and decision-making. To tackle these issues, we present Skypilot, an LLM-enhanced two-stage framework that grounds language models in physical reality by integrating monte carlo tree search (MCTS). In the first stage, we introduce a diversified action space that encompasses generate, regenerate, fine-tune, and evaluate operations, coupled with physics-informed reward functions to ensure trajectory feasibility. In the second stage, we fine-tune Qwen3-4B on 23,000 MCTS-generated samples, achieving substantial inference acceleration while maintaining solution quality. Extensive numerical simulations and real-world flight experiments validate the efficiency and superiority of our proposed approach.

Keywords: Coverage search, LLM, Monte carlo tree search, Physical grounding.

Method

Skypilot Framework MCTS
Figure 1: The workflow of Skypilot. 1 / 2

Pipeline Overview

1

Radar-Based Localization

2

Monte Carlo Tree Search

3

Full Parameter Fine-tuning

4

Deployment

Results

Indoor Experiment Outdoor Experiment Ablation Experiment
Figure 1: Indoor coverage search experiments. 1 / 3

Flight Video

SkyPilot deployment - indoor and outdoor flight experiments.

Code

Code is coming soon.