RL Octocopter -- Karolina Dubiel

Day 55: (Dis)armed and (not) dangerous, props to everyone!

Jul 23, 2026 Phase IV Project Complete!

The project is complete and successful! I officially have designed, built, and wired up an octocopter, trained an RL policy to sustain flight through single- and dual-motor failures, and deployed it zero-shot to the real drone -- compiled directly into ArduPilot's C++ (at the AP_Motors level), with no companion computer and no fine-tuning after simulation. On hardware, it goes on to survive not only single and dual motor failures, but also out-of-domain triple and quad failures.

The policy surviving an initial single failure (back motor), then a stacked
dual failure (happens during zoom in), for a total of 3 dead motors.

The final controller is a small neural network -- a two-hidden-layer MLP (128 units each, ~43k parameters) -- running at 50 Hz onboard the drone's own flight controller. It reads the last 10 state frames (~0.2 s of history): attitude, angular rates, position, velocity, and one 8-value channel giving each motor's current thrust level, where any failed motor is forced to zero. That zero is a supplied fault signal -- the policy is told which motors are out rather than sensing it (in these tests the failures are commanded, so no ESC/RPM telemetry is needed); its job is to fly given that information. It outputs eight per-motor commands as small adjustments around hover. It was trained entirely in sim (MuJoCo + PPO via PufferLib) with a fixed mixture of zero-, single-, and dual-failure episodes present from the start, under a curriculum that ramps domain randomization from nominal to full. The one architectural trick is a symmetry-averaging head: at every step the network runs its weights twice -- once on the true state, once on that state relabeled by a 180° rotation of the airframe -- and averages the two, which makes it exactly equivariant to that rotation (to floating-point precision, at zero extra parameters).

The policy was only ever trained on 0, 1, or 2 dead motors (the fault mix is (0.1, 0.2, 0.7) for 0/1/2), yet on the real airframe it held station through triple and quad failures too. Zero-shot generalization past the training envelope, on real hardware, is honestly more than I set out to prove -- I only ever asked the policy for dual-failure recovery, and the airframe cleared double that. I assumed the reason it stretched so far was a quirk in how the policy learned -- its habit of collapsing to four motors, described below. Testing that later showed the opposite: the collapse cost it out-of-distribution survival rather than buying it (see the edit note below). It generalized in spite of that quirk, not because of it.

In many cases, my octocopter had taught itself to fly with less motors than it has available. It drives itself down to roughly four motors and holds pose there, doing this even in certain trials of nominal hover, with all eight motors perfectly healthy. When four motors die, the drone is often in the regime it would choose to live in anyway.

A trial in which the drone chose to
fly as a quad, despite only one motor loss.

This isn't one bug in one place, it's a handful of design choices/mistakes composing into a strategy the optimizer was completely free to take:

Nothing in the reward rewards using all eight motors. My reward is pose-only -- altitude, tilt, yaw, rate, position. Holding the target pose on 8 motors and holding it on 4 score identically, so parking a healthy motor is free.
70% of its life was already a reduced-motor drone. With dual failures weighted at 0.7, the policy spent most of training flying six motors, not eight -- and it's told which are dead (I ended up going with explicit fault detection in the final policy). Its behavioral center of mass is a degraded octo. It never learned "fly the octo, patch around faults"; it learned "find a surviving subset and fly that." Nominal 8-motor hover is the rare case, not the default.
Hovering an octo on all 8 motors is over-actuated -- 8 thrusts against 4 constraints (lift + 3 torques), an infinite space of equally-valid allocations, with yaw as the weak, ill-conditioned axis (it makes ~11× less torque per newton than roll/pitch). That floppy null space is hard to learn a crisp policy for, so the optimizer collapses it to a minimal four-motor set. Not a quadcopter, though: the four it keeps are yaw-balanced (two CW, two CCW), but no opposing pair is ever among them. Two sit 45° apart and two ~135° apart, and the adjacent pair -- whose thrust vectors reinforce into a large roll/pitch moment -- runs at roughly half the thrust of the spread pair. That ~2:1 split cuts the residual roll/pitch moment from ~0.56 to ~0.014 N·m. And because my pose reward pays for tightness of hover, that configuration measurably scores higher.

Edit (Jul 31, 2026): In the interest of training the truly most effective policy, I went back and added a reward for using every available motor. Retrained from scratch with everything else held identical -- same dynamics, same pose reward weights, same (0.1, 0.2, 0.7) fault mix, same network, same hyperparameters, same 20M steps -- plus one per-step term:

r_usage = −w_u · (1 − n_active / n_available)

n_available counts only healthy motors, so a motor the sim has killed is never held against the policy; n_active counts healthy motors commanded above 5% of hover throttle, the same threshold I use to measure the collapse. w_u = 0.55, set by measuring what collapsing buys: dropping 8 motors to 4 gains ~0.134 pose reward per step, so the penalty has to beat that. The result holds all eight motors in hover (20/20 episodes) for 3.6% less hover precision (0.9484 vs 0.9838 mean per-step pose reward), and survives out-of-distribution failures substantially better. Both policies flown on every physically survivable failure set -- feasibility decided by a static control-allocation LP, so unrecoverable geometries count against neither -- under full domain randomization, identical sets and seeds:

Motors lost Trained on? Original (deployed on 7-23) policy Revised (trained on 7-31) policy Difference

0 (hover) yes 100.0% 100.0% –

1 yes 100.0% 100.0% –

2 yes 92.9% 97.1% +4.2 pp

3 no 90.0% 99.0% +9.0 pp

4 no 64.0% 85.0% +21.0 pp

Motors lost	Trained on?	Original (deployed on 7-23) policy	Revised (trained on 7-31) policy	Difference
0 (hover)	yes	100.0%	100.0%	–
1	yes	100.0%	100.0%	–
2	yes	92.9%	97.1%	+4.2 pp
3	no	90.0%	99.0%	+9.0 pp
4	no	64.0%	85.0%	+21.0 pp

I consider this project a huge success, because the end goals were fully met, no matter how the final policy got there. The honest caveat is that this is textbook reward-hacking: the agent maximized exactly the objective I specified, which was admittedly faulty. A great lesson for the future, and I'm choosing to not beat myself up over it :D

The core claim is done and confirmed on real hardware: design → build → train → zero-shot deploy, surviving failures it was never trained for. I do plan to revisit this drone and do further projects with it (maybe computer vision!), but the work is done for now. I'm feeling incredibly proud and happy of what's been accomplished. Thank you to everyone who followed along. Props to everyone who helped along the way (pun very much intended). 🚁

Day 54: We have RL liftoff! (and many lessons learned)

Jul 22, 2026 Phase IV

The drone flew. In real life. My RL policy worked!!!!

Proof it's my policy flying it, not me

A lot has happened in the past 2 weeks, and I slacked on regular updates amidst SO much happening in my work/personal life (I do have a full-time job... hence all the drone videos being in the dark haha).

Also, a LOT went wrong before it went right. In short: I had an insane bug in my past training runs that rendered every fault-tolerance number I'd published incorrect, a reward function that was rewarding the drone for giving up, a motor map that was way off of reality, and one flight-controller parameter that quietly defeated the whole fault demo. Long post, but I've decided to keep it all as one story.

First, a brief summary of SOME of the bugs/issues I've fixed over the past 2 weeks:

Three major sysid bugs (these were really bad). I thought I made it a decent way through the hardware / mathy part of this project without making major errors, and I was 100% incorrect:
- The CoM→motor arm length was hardcoded at 35.20 mm. I have no idea how this happened. My best guess is that I thought I overwrote that temp value with the real measurement but never saved it? I have no idea how I didn't notice it or catch it earlier. The real distance is 181.938 mm -- an arm 5.17× too short. (At 35mm my 5" props would overlap by ~100mm; physically impossible, and something I should have caught ages ago.)
- My bifilar-pendulum inertia formula used 8π² where it should have used 4π² (see the correction on Day 25), silently halving every inertia I fed the simulator. The raw timing data was fine; it was purely the constant.
- The motor layout was modeled as a "+"-config when the real airframe is an "X"-config, 22.5° off the body axes (severe lack of drone experience is showing).

The two scalar bugs compound in the same direction -- torque 5.17× too low, compounded with inertia 2× too low, means the simulated drone had roughly 2.6× less control authority than the real one to fly the same maneuver. I'd been training on a strictly harder drone than the one on my desk. Retrained from scratch on the corrected plant (middle column below): hover and single-motor failure came back solved, but the uncompensatable-yaw dual cases (two same-spin motors 90° apart) cratered under domain randomization.

Faults (survival under full DR)	After sysid fix	After reward + frame-averaging
0 / 1 (hover / single)	100.0% / 98.7%	100.0% / 100.0%
2 (dual, all 28 pairs)	71.7%	95.8%
└ uncompensatable-yaw	41.2%	94.6%

^ See above, 41.2% survival for dual motor failures (bad). A static control-allocation check proved all 28 dual combinations have a feasible hover equilibrium with thrust headroom to spare, so it was something I was teaching wrong. I'd zeroed the yaw penalty for the "uncompensatable" pairs, and there was no yaw-rate penalty at all. "Fight to hold heading" and "spin freely" scored the same, or worse for fighting.
→ fix: removed the exemption, added a yaw-rate penalty, oversampled the hard pairs. Uncompensatable-yaw survival under DR jumped 41.2% → 69.9%.

69.9% is still not great, and it was that way because two of the eight pairs stayed stuck at 0% survival -- while their mirror-image twins (the same physical failure, 180° rotated) sat at 100%.
→ fix: frame-averaging -- run the network on the true orientation and on its 180°-rotated twin, then average the two outputs. Exactly rotation-equivariant by construction, zero new parameters, old checkpoints still load. Final retrain had all 28 dual pairs at 100% nominally, none below 70% under full DR, with the two stubborn 0% pairs measuring 98%.

There were also a LOT more bugs (most smaller than the ones above). I'm going to leave them out of this writeup for brevity and relevance, but I have it noted in my private build log and happy to discuss any of them!

And then it flew!! Plain RL hover mode on the physical airframe worked PERFECTLY.

RL hover = basically loiter mode!

The fault modes went less well, for an extremely fixable reason. Both engaged and flew -- single-fault had a bit more jitter than hover, dual-fault drifted more than single -- but no motor ever stopped. All eight kept spinning. The issue was really simple -- I swear I thought I'd set it correctly, but MOT_SPIN_MIN was still at its 0.15 default. The fault path zeroes the thrust request, and ArduPilot maps a request of 0 to the actuator floor, 1150µs of PWM. Per-motor hover is only ~0.16 across 8 motors (~1280µs), so the dead motor slowed by ~130µs instead of stopping.

The mask itself fired. It zeroes the observation channel too, so the policy saw dead motors and compensated, which is why dual-fault drifted more than single-fault -- if it hadn't fired, the two modes would look identical, and there was a clear difference in stability when flying. The ESC just never got a stop command. This is a SUPER easy fix, but it was already dark at this point, so it's a problem for tomorrow. See you very soon for the full test flight!!

Day 40: CPU, I love you

Jul 8, 2026 Phase III

I had a plan to speed up the 20M-step training run on a GPU box with two RTX 5060 Ti cards by replacing MuJoCo's CPU-bound physics with a hand-rolled JAX core (since MuJoCo has no GPU path). Before committing to that plan, I designed and implemented all the prerequisite fixes:

1. I set one of my penalties wayyyy too high. PufferLib unconditionally clamps every reward to [-1, 1] during rollout collection. My crash penalty was -10 and alive steps had unbounded penalty terms, so a spinning-but-alive step and an actual crash looked identical to the learner. Day 30's policy worked despite this, but it was still a bug
→ fix: Alive steps now score continuously in [0, 1], crash is a flat -1.

2. The asymmetric actor-critic was never actually wired up. I was computing privileged critic observations (fault flags, full state) every step and throwing them away -- the critic was training on the same obs as the actor the whole time
→ fix: the critic now has its own trunk reading privileged state; the actor is architecturally incapable of seeing fault flags, not just conventionally so. The combined training-time model (actor + critic) grew from 43.4k to 62.6k params -- the deployed actor alone is unchanged, since the critic gets discarded once training is done.

3. The curriculum would've broken under multiprocessing. train_progress was a shared class attribute -- fine on the Serial backend Day 30 used, but it would've silently frozen at "pure hover forever" the moment training moved to multiprocessing, since worker processes never see the main process's updates
→ fix: threaded the progress value through a shared variable all workers read from. Works identically whether training runs single- or multi-process.

Spoiler: the domain-randomized policy is very good at surviving out-of-distribution triple motor failures!

On top of those three, I also cleaned up a system-identification module so all my physical constants (mass, inertia, motor curves) come from one place instead of being scattered across files, added domain randomization so the policy trains against varying mass, thrust, sensor noise, and latency instead of one fixed airframe, and fixed a bug where the angular velocity in the observation was getting rotated twice -- a no-op near hover, but it corrupted the gyro reading exactly during the large-tilt recovery this whole project is about. All of that landed and was verified on my Mac. I then rewrote the entire physics engine in JAX so it could run on GPU, and checked it against MuJoCo output at every step to make sure it was simulating the exact same physics.

All of that was for nothing! I hit GPU driver/compute-capability issues that weren't worth chasing down -- CPU training was already fast enough that debugging a driver stack for a marginal speedup didn't pencil out. 🙃

I decided to go back to square one (kinda, the bug fixes from earlier still applied) and trained on MuJoCo CPU instead. 20M steps on my M4 MacBook Air took 2.76 hours (~2.0-3.5k SPS), ran through all four curriculum phases cleanly (pure hover → single → dual-heavy), multiprocessing threaded the progress correctly the whole way (first time confirming that fix at full scale), and converged hard: entropy -4.9, clipfrac ≈ 0, value_loss 0.12.

Bar chart comparing mean survival across 16 physically-recoverable triple-motor-failure combinations for the pre-DR policy vs the domain-randomized policy, with per-combo survival shown as dots — 16 physically-recoverable triples, out-of-distribution for both policies

Grouped bar chart comparing survival under full domain randomization for the pre-DR (Jun 28) policy vs the domain-randomized policy, across hover, single, and dual motor failure — Pre-DR policy folds under randomization it never trained against

Correction (added Jul 17, 2026): the eval numbers below (67.7% dual / 48.8% uncomp-yaw survival under DR, and the ±4 mm-CoM retest) were measured against a simulated airframe with a motor arm hardcoded 5.17× too short and an inertia tensor 2× too small -- both unrelated to CoM offset, and both found and fixed in a later sysid audit. A full retrain was required; these specific percentages don't reflect the real airframe.

Results: Two eval surfaces, 300 episodes/fault-class each. The deployed policy is still a 43.4k-parameter MLP -- same actor architecture as Day 30, since the critic (which grew to absorb more privileged state for the asymmetric setup) gets thrown away at deployment anyway. Nominal airframe (no randomization) is perfect:

Faults	Nominal	Under full DR	Transfer gap
0 (hover)	100.0%	98.3%	1.7%
1 (single)	100.0%	89.3%	10.7%
2 (dual)	100.0%	67.7%	32.3%

I had been randomizing center-of-mass offset by ±15 mm. With ±4 mm (closer to what I can actually achieve by hand-balancing the real drone) the same DR eval shows:

Faults	Nominal	Under full DR (±4 mm CoM)	Transfer gap
0 (hover)	100.0%	100.0%	0.0%
1 (single)	100.0%	100.0%	0.0%
2 (dual, all)	100.0%	94.7%	5.3%
└ dual, excl. uncompensatable-yaw	100.0%	100.0%	0.0%
└ uncompensatable-yaw only	100.0%	79.0%	21.0%

We can work with that!!

Day 35: Internet microcelebrity?

Jul 3, 2026

My project got posted on HackerNews, where it stayed on the front page for well over a day!

Y news.ycombinator.com

Building a custom octocopter from scratch with no prior hardware experience

418 points · 91 comments · submitted by noleary

I'm taking the week slow in terms of this project due to a TON of stuff happening in my work/personal life, but I'll be back soon, I promise! I'm so happy that this project has resonated with people, and I truly appreciate every single person who has interacted with it in some way.

Day 30: Learning curve & baby policy

Jun 28, 2026 Phase III

The drone flies through all single, dual, and SOME TRIPLE motor failures on a no-DR trial policy!! But it's sim-only, and the path to get here had significant learning curves.

Surviving dual motor failures (2x speed)

Surviving worst-case 2-motor failure (2x speed)

Before training the huge policy with domain randomization and all of the bells and whistles, I decided to train a sim-only policy, which would be ready in about an hour and half on CPU. I'm really glad I did this, because it definitely didn't work out the first time. Here's a (maybe too transparent) timeline of the process:

#	What I tried	Result
1	Baseline PPO, high exploration, always 2 faults. Ran overnight.	Failed. Entropy climbed 11→22 the whole time, and it crashed at 20M when trying to save (it was fine, I had checkpoints, but still)
2	Lowered exploration, kept always-2-faults and no curriculum (straight-to-dual).	This one seemed to be working. Looked broken at 2M (crashing at step 7, very negative entropy -- Gaussian differential entropy can go negative when variance collapses), but trained through it. I killed it deliberately because I wanted to add single-motor failures first.
3	Added a hover->single->dual curriculum	Broke everything. Turned out the curriculum's 4M steps of pure hover was exposing two latent bugs (failures from step 0 gave enough training signal to bulldoze past them).
4	(not a config -- an operator error)	A zombie training process from the night before was running alongside the new one, both writing the same checkpoint filenames.
5	Residual actions (u = hover + action).	Commands got driven to ±3, saturated and lopsided -- tip-over in 7 steps.
6	Stripped/re-added the curriculum, low exploration.	0% at every checkpoint -- and it crashed at step 7 every single time, even during the pure-hover phase. That's what finally told me that this bug had to be systemic.
7	The two real fixes (below).	Worked. Hover learned by 0.5M steps, 100% survival on hover/single/dual by ~9.5M.

Everything from #3 onward -- the entropy panic, the residual detour, the curriculum back-and-forth -- was me poking at symptoms of two underlying bugs.

1. The actions were getting stuck. The Gaussian policy outputs unbounded means, but the env hard-clips commands to [0,1] and PPO computes gradients on the unclipped value -- so once a motor drifts past the clip edge there's no corrective gradient pulling it back inside, and it stays there. With 8 motors this produces a lopsided tip-over, and no hyperparameter fixes it because it's not a hyperparameter problem. Fix: squash through tanh as a residual around hover throttle, so commands can't saturate and an untrained net already hovers. This alone bumped untrained survival from 7 steps to 205.

2. Staying alive paid nothing. An open-loop hover test printed r = 0.00 every step: at the ~1.9m the drone actually settles, the +0.1 survival bonus was exactly cancelled by the -0.1 altitude penalty. Since the drone is marginally stable, every episode eventually crashes (-10), so "hover 200 steps then crash" and "crash immediately" had the same return. Fix: bump r_survive 0.1 → 1.0, so hovering pays +0.9/step and PPO finally has a reason to stay up.

Correction (added Jul 17, 2026): this policy (and every survival number below, including the uncompensatable-yaw heading-hold claim) was trained against a simulated airframe with a motor arm hardcoded 5.17× too short, an inertia tensor 2× too small, and motors modeled in a "+" layout instead of the real "X" layout. A later sysid audit found and fixed all three, and the fault-recoverable set is layout-dependent, so these results don't transfer even in shape -- this policy was retrained from scratch.

Results

Final policy is a 43.4k-parameter MLP.

Learning curve across 20M training steps — Learning curve across 20M steps -- survival and reward by fault class

It even generalizes to 3-motor failures it was never trained on, as long as recovery is physically possible. Even when I killed 3 adjacent motors (physically unrecoverable) then, it fought for 7.2 seconds and sank instead of tumbling.

Surviving triple motor failures (out-of-distribution)

One nice surprise: the "uncompensatable-yaw" cases (two same-spin motors 90° apart, where in theory the drone should just spin freely) aren't actually uncompensatable. The policy holds heading to within ~13°/s in all of them -- a slow drift, not a free spin. My heuristic for flagging those was too pessimistic.

Next step: the real, sim-to-real-able policy!

Day 25: Being a physicist, aka swinging my drone from the kitchen table

Jun 23, 2026 Phase III

I modeled and printed a GPS mount, which was the final component I needed to lock down my total system mass. Now that the drone is fully complete, I was able to collect the full system identification data on the drone. I haven't been able to dedicate as much time as I would've wanted to this drone for the past ~1.5 weeks, but I hope to lock back in now that it's a software-only task for the next little bit.

Yaw bifilar pendulum setup (I promise the strings are parallel and straight IRL) — Yaw bifilar pendulum setup
(I promise the strings are parallel IRL)

Here are the results and calculations of my system identification. They're not completely perfect or scientific, but I'm hoping domain randomization can make up for any inaccuracies. I'll be using motor/propulsion characterization information from the manufacturer and have also collected CoM info.

Correction (added Jul 17, 2026): the formula below is off by 2× -- the correct bifilar-pendulum constant has 4π² in the denominator (NACA Report 467), not 8π². The raw trial timings/measurements are fine, but every I value in the results table is half what it should be. Corrected: I_xx/I_yy/I_zz = 0.008516 / 0.008324 / 0.012250 kg·m² (roughly double the 0.004258 / 0.004162 / 0.006125 shown below).

              I = (T² · M · g · d²) / (8π² · L) → I = 0.36974 · d² · T²

              M = 1.177 kg  ·  L = 0.39552 m  ·  d =
                half wire sep. (m)  ·  T = time for 20 osc ÷ 20 (s)
            

Measurements

Each trial timed over 20 oscillations (T = total / 20). Wire separations were measured as full width and halved for d.

7 trials each; pink = worst 2 dropped before averaging.

Axis	T1	T2	T3	T4	T5	T6	T7	Avg×20 (s)	T (s)	Wire sep. (mm)	d (m)
Roll	12.97	12.87	13.15	12.92	12.9	12.82	12.92	12.906	0.6453	332.592	0.166296
Pitch	12.77	12.64	12.82	12.63	12.75	12.84	12.82	12.76	0.638	332.592	0.166296
Yaw	13.84	13.52	13.52	13.47	13.52	13.4	13.65	13.536	0.6768	380.314	0.190157

Results

Axis	T (s)	d (m)	I (kg·m²)
Roll	0.6453	0.166296	0.004258
Pitch	0.6380	0.166296	0.004162
Yaw	0.6768	0.190157	0.006125

Day 22: Standing on business and changing of plans

Jun 20, 2026 Phase II

The drone now has 8 legs to land on, which should make test flights significantly less scary. Although I never optimized this drone for weight when designing it (the body plates could have more cutouts and other optimizations could've been made), I've now decided to be more mindful about each component going forward. In total, the 8 legs weigh 7.67g.

I've also made the decision to no longer have a separate microcontroller for now. I'd originally planned to bolt a separate companion computer onto the drone to run the RL policy and feed motor commands to Betaflight over MSP -- first a Raspberry Pi 4, then a Teensy when the Pi looked like a bad fit. The problem: no matter which board I picked, my architecture needs to send 8 direct per-motor commands, and doing that over MSP fights Betaflight's safety model (motors don't reliably stop on disarm or link loss). So I'm scrapping the separate microcontroller -- my flight controller is already an STM32H743 (480MHz M7), so I'm just compiling the policy straight into the Betaflight firmware on the board I already have, which also kills a big chunk of my loop latency. Papers that inspired this: "Learning to Fly in Seconds" (Eschmann et al., RA-L 2024) and Neuroflight (Koch et al., 2019).

I'll try this out and re-evaluate if I run into significant blocks.

Day 17: The eagle has landed (taken off)!

Jun 15, 2026 Phase II

Today, exactly 2.5 weeks after the kickoff of my initial idea, I officially completed the pipeline from concept -> flying octocopter with 0 prior hardware or CAD experience. When I started this project, I had never flown a drone before.

I haven't done anything except hover yet, and there's no microcontroller on board, so this is a completely regular octocopter with no RL abilities at all. I also have yet to make a GPS mount, mount my antenna, or strap anything down nicely, so there's a lot of work to do before this thing can properly fly safely.

YouTube link to flight video ->

We have liftoff!

To be clear: currently, if this drone lost a single motor in flight, it would probably stay up. Octocopters are famously tolerant to single motor failure for two reasons:

1. There's huge thrust overcapacity: 8 motors at ~125 gf load each at hover means losing one drops total capacity from ~11,000 gf to ~9,750 gf, still enough to maintain a healthy 2:1 thrust-to-weight ratio up to nearly 5 kg of drone weight (we're at 1 kg).

2. Betaflight's PID loop runs at several kHz and doesn't need to know why the drone is tilting -- it just sees the gyro reporting a roll, and commands the remaining motors on the low side to push harder. The yaw imbalance gets partially compensated by Betaflight's mixer redistributing throttle between the surviving CW and CCW motors. The drone would stay airborne with maybe a slow yaw drift and degraded responsiveness, but it usually wouldn't fall out of the sky unless other bad things happened.

The problem is that this only really works for one motor. As soon as a second motor dies (especially two same-rotation motors at 90° from each other) the static mixer breaks down. It keeps demanding thrust from two dead motors and the drone would become uncontrollable. That's the failure mode RL is supposed to fix.

P.S. -- while setting everything up in Betaflight, I set the startup chime of the drone to be Mask Off by Future :D

Also: the flat 8-arm frame has a much larger effective disc area than a typical quad, which means strong ground effect -- air pushed down by the rotors compresses against the floor and bounces thrust back up, so the drone feels weirdly buoyant near the ground (you can see it floating in the video below).

My drone sings “Mask Off” by Future upon startup 🎶🎤 @1future https://t.co/GQh8Fyy5wJ #buildinginpublic pic.twitter.com/2uouzK0spv
— Karolina Dubiel (@karolina_dubiel) June 19, 2026

Floating on its own ground-effect cushion

A common question: why not MPC?

A lot of people on X have asked this -- to be honest, the primary reason is that I specifically wanted an RL project and designed this drone around that goal. I came up with the fault-tolerant octocopter concept as a vehicle for learning RL on real hardware, not the other way around.

That said, there are real engineering arguments for RL here:

Inference cost: MPC solves an optimization problem at every timestep, which is a lot of computation for a RPi commanding 8 motors (more than RL, which is a single pass through a ~50k parameter network, which would probably be under or around 1ms)
Unknown failure state: from my understanding, MPC normally needs to know what the system is doing, and without a dedicated fault detector, this would create extra work for me. The RL policy learns to infer failure state implicitly from the gap between commanded and observed behavior.
Model mismatch tolerance: MPC is only as good as its model. My cheap motors probably aren't perfectly identical and the inertia tensor I measure will only be approximate. Heavy domain randomization during RL training explicitly teaches the policy to handle model error. An MPC controller built on the same uncertain model doesn't get that for free.

MPC is probably the more reliable choice for a project like this, but not the most fun option :D If I can't get RL working, MPC is absolutely my fallback -- and at that point I'd probably treat this whole attempt as useful data collection for a model anyway.

Day 13: What's next -- training an RL policy

Jun 11, 2026

Since posting on X, I've gotten many DMs asking exactly how I want to approach the next phase of this project: making the drone fly with RL. Here's the plan I have so far.

Most importantly, the RL policy will directly command all 8 motors at 50 Hz over a serial link to the flight controller with no traditional PID loop in the path. This is the only architecture that gives the policy full authority to reallocate thrust when motors fail.
I'm focusing on six unique failure classes (ignoring rotational equivalence): single motor, adjacent pair (45°, mixed CW/CCW), 90° same-type, 135° mixed, 180° same-type, and full ESC loss (each ESC controls its own quad). The hardest case is the 90° same-type failure, because it's the only one that hits both problems simultaneously: a yaw torque imbalance (the two dead motors were the same spin direction) and a spatial asymmetry in the remaining thrust geometry.

The circuit diagram that I drew for wiring everything up — The single- and dual-motor failures that I want to support, plus ESC loss

Losing two same-spin motors leaves 2 CW and 4 CCW running (or vice versa), yaw-torque imbalanced 2:1 at equal throttle. Balancing them forces the CW motors to run at 2× the per-motor thrust of the CCW motors. At 1393 gf max per motor, the yaw-balanced thrust ceiling works out to 5,572 gf -- enough to maintain a 2:1 thrust-to-weight ratio up to ~2.8 kg of drone weight (we're at 1 kg). The remaining 6 motors span a 270° arc, so roll and pitch authority still exists. The worst case is survivable -- the drone would be spinning, but it could still hover to a soft landing.

	Full 8-motor	90° same-type (6 motors)
Max total thrust	11,144 gf	5,572 gf (yaw balanced)
CW motor load at hover	~9%	~18%
Max drone weight at 2:1 T/W	~5.6 kg	~2.8 kg
Yaw authority	full	near zero

Simulation

I'm building the sim in MuJoCo, because it runs fast on a CPU and I have a Mac, which rules out Isaac Lab and basically everything else NVIDIA-shaped. For a single rigid body with 8 thrust points, MuJoCo is more than enough, and I can run ~128 environments in parallel on my laptop.

The model itself comes from measurements, not the CAD. I'll be gathering data on:

Total mass
Inertia tensor via the bifilar pendulum test
Motor thrust curves
Motor time constant
Hover throttle point

I'm also adding two things to my sim environment that I keep reading are what actually kill sim-to-real transfer for motor-level control:

1. Motor lag: real motors take 20–50 ms to reach a commanded speed. In sim, thrust changes instantly unless you model it. A policy that learns with instant motors learns to twitch.

2. Loop latency: on the real drone, there's ~15–30 ms between the IMU reading and thrust actually changing (serial read, inference, serial write, ESC response). If I train with zero latency, the policy will oscillate the second it touches hardware. This one scares me the most, so it's getting randomized aggressively (the policy trains against a delay that changes every episode and jitters within episodes).

puffer. just puffer. trust me. you will thank me in about a month from now.
— Chris von Csefalvay 🔜 CVPR26 (@epichrisis) June 7, 2026

Everything else physical gets randomized too: mass ±10%, per-motor thrust constants ±15% (cheap motors are not identical, I own eight data points proving this), center of mass, battery sag over a flight, sensor noise^[4].

Training

PPO^[1] via PufferLib. I looked at SAC since it's more sample-efficient, but sample efficiency solves a problem I don't have -- my sim steps are nearly free. PPO with a pile of parallel environments is what almost every sim-to-real flight paper I've read actually shipped, and it scales more simply across parallel envs and is more stable under the variance that heavy randomization adds". (Also: an X reply told me "puffer. just puffer. trust me.")

Two more decisions I stole from sim-to-real literature:

1. The critic gets to cheat. During training, the value network sees ground truth the real drone will never have, like which motors are dead, the exact thrust constants, and true velocity. The actor only sees what real sensors provide. The critic gets thrown away after training, so this costs nothing at deployment. (This is called asymmetric actor-critic^[2], and I've read that it makes a huge difference when the physics are randomized this hard.)

2. No fault detector (for now). The policy sees its last 5 observation/action frames and has to figure out failures on its own, from the gap between what it commanded and what the drone did.

Under a same-type dual failure the drone physically cannot hold its heading -- the torques don't balance at any throttle combination. The right behavior is to give up on yaw, spin slowly about vertical, and stay level. If the reward punishes spinning, the policy sacrifices roll and pitch chasing a heading it can't have. Mueller & D'Andrea showed the same thing for quads losing a motor^[3] -- their recovering quad spins the whole time. Mine will too, on purpose.

Deployment

If the policy shows promising survival rates in sim, it'll get exported to ONNX and run on the RPi 4 (I think. Any opinions on this vs other microcontroller options?) the network is ~45k parameters, which is under a millisecond of inference, so the Pi is not the bottleneck. The 50 Hz loop will read attitude and gyro over serial, run the policy, and write 8 motor commands.

Then, the actual experiment: fly, kill motors from the transmitter, and find out if millions of simulated crashes taught it anything!

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal Policy Optimization Algorithms," arXiv:1707.06347, 2017.
L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel, "Asymmetric Actor Critic for Image-Based Robot Learning," RSS, 2018.
M. W. Mueller and R. D'Andrea, "Stability and control of a quadrocopter despite the complete loss of one, two, or three propellers," IEEE ICRA, 2014.
J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, "Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World," IROS, 2017.

Day 11: Insider traitor-ing

Jun 9, 2026 Phase II

Flashing the firmware didn't go as planned -- the USB-C input port on my H743 AeroSelfie FC is broken. This isn't the biggest deal in the world; everything on that board was pre-soldered and detaching it for return was pretty easy. The annoying part is that a broken FC is pretty critical-path, and nothing can progress until the new one comes in (thankfully soon!)

I attached the standoffs and top body plate to the drone to get an idea of what everything would look like all together and weigh the assembled drone.

The drone with the top plate and standoffs attached — The drone with the top plate
and standoffs attached

The drone weighs exactly 1kg with a mounted battery (this weight includes everything except the flight controller, which is negligible). Each motor produces approximately 950gf of thrust at 70% throttle on a fully charged 6S battery, and up to 1393gf at full throttle. Across all 8 motors, that's 7,600gf -- 7.6kg of thrust -- at 70% throttle alone, against 1kg of weight, which gives a thrust-to-weight ratio of 7.6:1. To hover, I only need 125gf per motor, which is around 15-20% throttle. That means the drone has enormous headroom above hover -- at 70% throttle it's producing nearly 8x what it needs to stay airborne. This is really, really good! An overpowered drone = way more leeway to tolerate (or ideally, fully recover from and maintain normal flight during) motor loss.

I'm kind of blocked until the new FC arrives. I'm not used to blockers like this (given my all-software background), but I'm reminding myself that it's just part of hardware to have stuff like this happen. Annoying ≠ discouraging, and I'm really excited to see this drone hover soon.

Day 9: Wired up!

Jun 7, 2026 Phase II

I soldered the flight controller, two ESCs, GPS, battery wires, and receiver together! The drone is theoretically able to hover now, but I haven't tested that yet :D

I'm pretty inexperienced with soldering, so this part took me longer than any of the CAD/assembly so far. Given that I've never assembled electronics together this way, it was difficult for me to imagine how everything would fit together and to solder everything neatly. I ended up deciding to just have one battery wire, which I sandwiched between the two ESCs, so that it could serve them both.

The ESCs and flight controller sit on top of each other to simplify my center of gravity. Once I fly this drone as a regular octocopter, I'll also have to mount a Raspberry Pi or Jetson Nano (open to feedback here!) onboard to run the inference. I plan on sticking this board to the bottom of the top plate.

Before I test hovering, I'll need to:

Flash Bluejay firmware to both ESCs
Configure Betaflight: set the mixer to Octocopter Flat X, ESC protocol to DSHOT600, enable the accelerometer, and dial in conservative rates and arming parameters
Configure failsafe behavior for if RC signal drops mid-flight
Balance check: find the battery position that centers the CoM and lock it down

Day 6: Superglued, taped down, and ready to solder

Jun 4, 2026 Phase I

As soon as the jig was printed, I used it to align the arms of the drone and filled any gaps in between with superglue.

The phase wires for the motors are now taped down and the frame is perfectly aligned

Filling the space between the arms with superglue while the drone is in the jig — Filling the space between
the arms with superglue while
the drone is in the jig

The arms were fully stable once the superglue set, which means I won't have any vibrational issues due to the imperfect arm alignment that I was worried about earlier. My original plan was to start soldering all the electronics today, but I'm still waiting on a soldering iron shipment. Planning to start wiring everything as soon as materials and my full-time job allow :D

Supergluing the arms in the jig required loosening the screws to get the arms to fully pop into the jig supports. One of the screws got stuck, snapped, and had to be drilled out :( Crisis averted with very minimal damage to the frame, though!

Day 4: Getting jiggy with it

Jun 2, 2026 Phase I

I continued with assembly, screwing the 8 motors to the arms and the 8 propellers to the motors.

Assembled drone frame with motors and propellers — The assembled drone frame (arms, bottom
and middle plates, motors, and propellers)

A screenshot from a timelapse of me attaching the motors + propellers to the body — A screenshot from a video of me
attaching the motors + propellers
to the body

A small problem: if you tug really hard, some of the arms wiggle a bit, even when fully screwed together and tightened. I think this is due to the fact that I set the cut tolerance as 0.1mm in the CAD, not knowing how precise the CNC mill would be. For the future, a better tolerance would be 0.05mm or 0.08mm. Any wiggle room in the arms can cause vibrations when flying, which could mess up my flight dynamics and make RL-based flying impossible.

The solution for this is to 3D print a 0-tolerance assembly jig to hold the arms in perfect position while the center of the drone is superglued together. Here's the design of said jig -- it'll be printed and ready to use soon:

Day 3: Assemble!

June 1, 2026 Phase I

After a weekend away at Pinnacles National Park, I screwed the body of the drone together: the 8 arms, the bottom plate, and the middle plate.

Work in progress ... screwing all of the arms together — Work in progress ... screwing all
of the arms together

Day 1: CAD, CNC milling, and humble beginnings

May 30, 2026 Phase I

While on a recent vacation in Guatemala, I came up with the idea for this project from a hammock on the shores of Lake Atitlán. Immediately, I ordered (most of) the necessary parts on Amazon and started ideating on exactly how to go about building a fully RL-powered, intelligently fault-tolerant octocopter.

I have never done a substantial hardware project before. I have never CADed, I've soldered once, and I have no experience with drone flight controllers, speed controllers, or anything in that domain. I have never flown a drone before. I have never trained an RL policy as complex as the one required for this project.
I got started thanks to hours spent on Google, Reddit, Claude, and talking to Tomas, an AE major who helped with every CAD and machine shop question I had.

The first two steps of this project were both started and completed today:
1. CAD of the drone's body and arms in Fusion360
2. CNC milling forms out of G-10 fiberglass (arms) and 5mm carbon fiber (body)

Fusion 360 CAD render with all eight motors placed on the frame — Fusion360 view of the finished CAD -- full octocopter with third-party motor/propeller .step files imported into my design

Top-down layout of the arm geometry in CAD — Intertwined arm geometry layout

The arms intertwine in the center of the drone for stability. They're sandwiched between a flower-shaped bottom plate and a larger body plate on top. I decided on flower cutouts for the carbon fiber body.

Arms drawing exported for CNC milling — Arms -- prepared for the CNC mill

Body plate drawing exported for CNC milling — Body plates -- prepared for the CNC mill

After that, it was time to CNC cut. This was my first time in a machine shop :D
I ended up having to re-cut the arms because the drill was going too fast and pushed the G-10 plate as it was cutting. I learned that cutting at ~20% speed when using thick materials is a much better idea than having to re-cut due to going too fast.

CNC toolpath simulation for the arm cuts — The CNC mill cutting out the arms

CNC mill cutting the carbon fiber body plate — Me preparing the CNC mill

Freshly CNC-cut parts laid out — Freshly cut G-10 and carbon fiber parts

Parameter	Range	Type
mass_scale	0.90 – 1.10	uniform
I_roll_scale	0.80 – 1.20	uniform
I_pitch_scale	0.80 – 1.20	uniform
I_yaw_scale	0.75 – 1.25	uniform
com_offset_x / y	±15 mm	uniform
thrust_scale	0.85 – 1.176	log-uniform
per_motor_thrust (×8)	0.88 – 1.136	log-uniform
KM_scale	0.60 – 1.40	log-uniform
tau_motor	0.02 – 0.08 s	uniform
loop_latency	0.015 – 0.030 s	uniform
attitude_noise_std	0.001 – 0.02 rad	log-uniform
battery_sag_frac	0.0 – 0.08	uniform

Fault-Tolerant RL Octocopter