Nahid's Notes

Introducing AerialSim: Training and Evaluating AI Agents for Drone Control

Train and evaluate vision-language agents on realistic drone missions through interactive control, code generation, and verifiable rewards.

October 2026

Vision-language models have become increasingly capable of understanding images and videos. AerialSim extends these capabilities to autonomous drone missions that require visual perception, spatial reasoning, planning, and physical action.

Consider a drone approaching a landing pad. The model must identify the pad, navigate toward it, detect obstacles, and decide whether landing is safe. If the pad becomes obstructed during the approach, the model needs to adjust its plan. These tasks bring together visual understanding, sequential decision-making, and feedback from the physical environment.

Today, we're introducing AerialSim, a reinforcement learning environment developed with Prime Intellect for training and evaluating vision-language agents on autonomous drone missions. Built on Cosys-AirSim and Unreal Engine 5.5, AerialSim supports two approaches to autonomous flight. In visual mode, the agent observes the environment through drone cameras and controls the drone one action at a time, using feedback to adjust its decisions. In coding mode, the agent writes a Python program to control the drone, tests and revises the program, and submits it for autonomous execution. Both modes share the same simulation environment, reproducible scenarios, and verifiable rewards based on flight outcomes.

One Simulator, Two Ways to Control a Drone

AerialSim runs drone missions in a photorealistic urban park with landing pads, obstacles, and changing environmental conditions. Both control modes operate in the same 3D environment and are evaluated using recorded flight trajectories.

Diagram comparing interactive control and code generation in AerialSim
Figure 1: One simulator, two ways to control.

AerialSim-Visual: Interactive Drone Control

In AerialSim-Visual, the model controls the drone through a sequence of camera observations and actions. At each step, it receives a camera image and selects one of six actions: take off, move to a position, adjust the camera, hover, land, or abort. The simulator executes the action and returns a new observation showing the drone's updated position and surroundings, including any changes in obstacles or landing conditions. The model uses this feedback to select its next action, creating a closed-loop control system that allows it to adapt its flight decisions throughout the mission. The mission ends when the drone lands, aborts, or reaches the maximum number of allowed turns.

Drone camera view of a landing pad with an obstruction on it
Figure 2: The drone approaches a landing pad with an obstruction. The agent uses visual observations to determine whether landing is safe.

AerialSim-Coding: Programmatic Flight Control

In AerialSim-Coding, the model generates a Python program using a drone-control API. It can test the program in a practice flight, inspect the results, and revise the code before submission. The final program runs in a fresh simulator instance without further model intervention. This evaluates the model's ability to translate mission instructions and visual observations into executable flight-control code.

Seven Scenario Families for Autonomous Flight

AerialSim includes seven scenario families designed to evaluate different aspects of autonomous flight, including navigation, landing decisions, obstacle avoidance, and multi-step mission execution.

Table 1: The seven scenario families, and what each mission asks the agent to do.
ScenarioMission
Land or HoldLand on a clear pad or hold position when it is obstructed
Find PadLocate a landing pad using camera observations
Obstacle BypassNavigate around an obstacle blocking the route
Alternate PadSelect a usable landing pad from two candidates
Patrol InspectionVisit inspection points in the required order
Go AroundBreak off an unsafe approach and repeat the approach route
CompoundComplete multiple objectives as conditions change during flight
Three camera frames from each of the seven scenario families, one family per row
Figure 3: Visual samples of seven mission families. Frames as the model sees them from drone cameras. One mission family per row.

Example Scenarios

AerialSim uses predefined seeds to generate reproducible missions. Each seed defines a base scenario, including the drone's starting position, landing-pad locations, and flight constraints. The environment then creates scenario variants by changing specific conditions, such as obstacle placement. Training and evaluation use separate seed ranges to prevent overlap.

For example, in land_or_hold, seed 50 generates two variants of the same base scenario: one with a clear landing pad and another with an obstruction. Both share the same mission instructions, drone starting position, and landing-pad location. The agent flies each variant independently using camera observations, without being told which condition is present. Comparing the outcomes helps determine whether the model adjusts its decisions appropriately when the environment changes. Other families, such as alternate_pad, generate three variants per seed to test different landing conditions.

The following example shows two alternative land or hold scenarios:

{
  "name": "land_or_hold_0050_clear",
  "family": "land_or_hold",
  "seed": 50,
  "pad": [19.2, -0.1],
  "blocked": false,
  "heading": 0.0
}

{
  "name": "land_or_hold_0050_blocked",
  "family": "land_or_hold",
  "seed": 50,
  "pad": [19.2, -0.1],
  "blocked": true,
  "heading": 0.0
}

Example Rollouts

Interactive Control

AerialSim-Visual provides six actions for controlling the drone:

{"action": "takeoff", "height": 5}
{"action": "goto", "x": 20, "y": -5, "z": -8, "speed": 3}
{"action": "look", "pitch": -50, "yaw": 0}
{"action": "hover", "seconds": 2}
{"action": "land"}
{"action": "abort"}

The environment uses North-East-Down (NED) coordinates, where negative Z values indicate positions above the reference origin. Each movement is limited to 30 meters and a maximum speed of 5 meters per second.

The following example shows a successful seven-turn rollout from the alternate_pad family. The agent navigates through two checkpoints, adjusts its camera to inspect the landing area, and lands on a usable pad.

Table 2: A successful seven-turn rollout in AerialSim-Visual, from the alternate_pad family. The agent flies both checkpoints, tilts the camera down to inspect the pads, and lands on the one that is usable.
TurnActionPurpose
1takeoffAscend to 5 meters
2goto(6, 0, -2)Navigate to the first checkpoint
3goto(17, 0, -2)Navigate to the second checkpoint
4look(pitch=-30)Adjust the camera toward the landing area
5goto(22.2, -3.5, -2)Approach the selected landing pad
6look(pitch=-80)Inspect the landing pad from above
7landLand on the usable pad and complete the mission

Code-Based Control

In AerialSim-Coding, the agent generates a Python program to control the drone using the provided API. The following example shows a successful flight program:

from aerialsim_flight.drone import connect

d = connect()
d.takeoff(height=8)
d.goto(18.3, 3.7, -5.5, speed=3)
d.hover()

d.camera_control(pitch=-90, yaw=0)
image = d.camera()

state = d.state()
print(
    f"Position: x={state['x']:.1f}, "
    f"y={state['y']:.1f}, "
    f"altitude={state['altitude']:.1f}m"
)

# Approach the landing pad
d.goto(18.3, 3.7, -2.5, speed=2)
image2 = d.camera()
d.land()

The agent can test its program in a practice flight, inspect the results, and revise the code before submission. The final program runs in a fresh simulator instance without further model intervention. Program execution uses an API proxy and import restrictions to limit access to supported drone-control operations and simulator state.

Reward Design: Verifying Flight Outcomes

AerialSim is designed with binary success failure rewards (1/0). The environment evaluates each flight using recorded simulator data. The recorder captures the drone's position, velocity, orientation, and collision state at approximately 10 Hz. A mission succeeds when all applicable conditions are satisfied.

The evaluator checks four categories of conditions:

Table 3: The four categories of condition the evaluator checks. A flight scores 1 only when every applicable condition across all four categories passes.
CategoryEvaluation criteria
Mission outcomeCorrect landing, hover, or abort decision based on scene conditions
Target completionReaching the designated landing region, completing the required action, and recording the flight
Flight safetyCollision avoidance, altitude limits, checkpoints, and navigation corridors
Scenario constraintsFollowing mission-specific rules, event timing, and inspection requirements

A mission receives a reward of 1 only when all applicable conditions are satisfied. The evaluator also records the outcome of each condition to support failure analysis.

For example, a drone may reach the correct landing pad and follow the required route but abort when landing is safe. The evaluator records the successful navigation and the incorrect final decision, resulting in a mission reward of 0.

These detailed evaluation results help identify failures in navigation, planning, and decision-making beyond the overall mission success rate.

Training Strategies: Reinforcement Learning with GRPO

AerialSim integrates with Prime Intellect's prime-rl framework and supports Group Relative Policy Optimization (GRPO). Each flight rollout receives a binary reward of 1 if the mission succeeds while satisfying all required safety and scenario constraints, or 0 otherwise. GRPO uses relative rewards within groups of rollouts to update the policy.

The model reported in our results was trained with a sampling temperature of 1.0, a group size of 16, a batch size of 64, and a maximum sequence length of 16,384 tokens. The four longest scenario families use a frame budget of 12 to stay within the inference server's limit of 16 images per request. The three shorter families do not require frame selection. Training and evaluation use separate seed ranges.

With binary rewards, groups in which every rollout receives the same reward provide no relative advantage for GRPO. These groups consume simulation and inference resources without contributing to policy updates. In our training run, 25% of groups were discarded, with substantial differences across families. The discard rate reached 91% for find_pad and 64% for compound, while four families had no discarded groups. This suggests that some scenario families provide much less useful training signals than others. Tracking the fraction of groups containing both successful and failed rollouts can help identify where training is likely to stall and guide scenario sampling.

Results

We evaluated the base and RL-trained Qwen3.5-4B models on 56 held-out missions across seven scenario families. Each model flew eight scenarios per family using the same evaluation seeds. Training and evaluation used separate seed ranges, with evaluation starting at seed 50. Each mission was scored as a pass or fail based on the complete flight outcome.

Table 4: Missions completed out of eight per family, on held-out scenarios starting at seed 50. Each flight is scored pass or fail on its complete outcome.
Scenario familyQwen3.5-4BQwen3.5-4B (Trained)
land_or_hold2/83/8
find_pad0/80/8
obstacle_bypass1/85/8
alternate_pad4/85/8
patrol_inspection4/84/8
go_around0/81/8
compound0/81/8
Total11/56 (19.6%)19/56 (33.9%)

Training improved mission success from 19.6% to 33.9%. The largest gain came from obstacle_bypass, where successful missions increased from 1 to 5. The remaining families showed changes of at most one successful mission.

Claude Haiku 4.5 completed 41 of the same 56 missions, providing a reference for performance against a stronger general-purpose model.

Training Improved Flight Completion

Training reduced turn-limit failures from 24 to 9 out of 56 missions. Before training, the model frequently exhausted its allowed turns without landing or aborting. After training, it was more likely to reach a terminal decision within the turn limit.

This improvement suggests that RL training helped the model manage flight execution and decide when to end a mission. However, reaching a terminal decision does not necessarily mean the mission was completed successfully.

Table 5: Flights that used every allowed turn without landing or aborting, out of 56. A turn-limit failure means the mission ended with no decision taken.
OutcomeQwen3.5-4BTrainedHaiku 4.5
Ran out of turns24/569/560/56

Counterfactual Scenes Reveal Perception Failures

The land_or_hold family tests whether the model can distinguish between a clear landing pad and an obstructed one. The evaluation includes four scenarios with a clear pad and four with an obstruction.

Table 6: land_or_hold split by scene condition, four scenarios each. Every flight reached the pad, so the difference lies in what happened after the drone arrived.
ModelClear padBlocked pad
Qwen3.5-4B2/40/4
Trained3/40/4
Haiku 4.54/43/4

Both Qwen models succeeded only when the landing pad was clear. Training improved clear-pad performance from 2 to 3 successful missions, but neither model completed a blocked-pad mission. The drone reached the landing pad in every flight in this family. The failures therefore occurred after navigation to the target area. The results suggest that the models struggled to recognize the obstruction or act appropriately on the visual evidence.

Haiku completed three of the four blocked-pad missions, showing that the same scenarios can be solved by a stronger model.

Counterfactual evaluation here is a useful debugging tool. The aggregate success rate shows how often a model completes a mission. Splitting the results by scene condition helps identify which changes in the environment cause the model to fail.

Validating Mission Feasibility with a Scripted Pilot

We also evaluated a scripted pilot across all seven scenario families. It completed every mission it flew.

The scripted pilot uses ground-truth scene information, such as landing-pad locations and obstacles, to select actions without processing camera observations. It was evaluated on scenarios generated from seed 0, while the model evaluation used held-out scenarios starting at seed 50.

The scripted pilot confirms that its evaluated missions are feasible and that the grader recognizes successful outcomes. This helps validate the simulation and evaluation setup, although it does not establish feasibility for every held-out scenario.

Conclusion

RL training improved mission success and reduced turn-limit failures, with the largest gain in obstacle avoidance. However, the land_or_hold results show that improved flight execution does not necessarily lead to better visual decision-making. The models reached the landing area but struggled to respond correctly when the pad was obstructed.

These findings highlight the value of counterfactual evaluation for identifying specific weaknesses in embodied agents. They also motivate further investigation into how models use visual observations to make decisions during flight.

Getting Started

Install the package and build the simulator image:

uv pip install -e ./environments/aerialsim_flight
bash scripts/aerialsim_flight/build_image.sh

Run an evaluation on the Land or Hold scenario family:

uv run eval aerialsim-visual -n 6 -r 1 -c 4 \
  --env.runtime docker --env.image aerialsim-city-park:local \
  --env.pilot.harness.id null --env.gpu device=0 --env.cpu 4 \
  --env.taskset.count 6 --env.taskset.family land_or_hold \
  --env.frame-budget 0 \
  -m <model> --client.base-url <url> \
  --client.api-key-var <key var>

The scenario family can be changed using --env.taskset.family, and the development or evaluation split can be selected using --env.taskset.split.

The simulator requires a GPU for rendering. In our setup, each simulator instance used approximately 2.2 GB of GPU memory and four vCPUs.

What's Next

Our initial results show that RL training improves mission execution and reduces turn-limit failures. However, the land_or_hold results reveal a challenge in visual understanding and decision-making. The next phase of AerialSim development is guided by these findings:

1. Understanding Visual Decision-Making

We plan to investigate how vision-language models interpret their surroundings and make flight decisions. The goal is to better understand the failures revealed by counterfactual scenarios and improve model performance.

2. Evaluating Model Scaling

We will evaluate larger models to test whether the gap to Claude Haiku 4.5 closes with scale, and compare architectures to see whether it is a property of size or of this task.

3. Expanding the Environment

We plan to introduce more challenging flight scenarios with dynamic obstacles, changing conditions, and longer missions.

Links

Acknowledgements

This project was funded by Prime Intellect. Special thanks to Sebastian Müller for the ongoing support and feedback.