Hard Real-Time on ARM: Adopting Xenomai for Mission-Critical Sustainment Workloads

The constraint

A class of sustainment workloads sits in an awkward middle ground: they're complex enough that you want a real operating system underneath them — file systems, networking, package management, security tooling — but they have a deterministic-response requirement that mainline Linux cannot meet.

Examples include:

• A robotic depot system performing repetitive precision pick-and-place where the closed-loop control band is around 1 kHz.
• A safety interlock for a remote refueling rig where the worst-case response from sensor-event to actuator-cutoff must stay under 100 µs.
• A time-sensitive networking endpoint that has to satisfy IEEE 802.1Qbv scheduled-traffic guarantees with sub-millisecond gate precision.

The mainline Linux scheduler, even with the PREEMPT_RT patch applied, will typically guarantee worst-case latency in the low hundreds of microseconds. That's adequate for many workloads, but the three above are not among them.

Two viable adoption paths

There are two approaches that get a system into the tens-of-microseconds latency regime while retaining a full Linux userspace:

Path A: PREEMPT_RT mainline patches. This is the simpler choice. Apply the mainline real-time patch series to the kernel, configure with full-preemption mode, and pin the real-time threads with appropriate scheduling policy and priority. Tooling is mature and the upstream story is improving with each release; PREEMPT_RT is on the path to becoming part of mainline.

For most cases, this is correct and sufficient. Worst-case latency on a quiesced ARM Cortex-A system typically lands in the 80–150 µs range.

Path B: Xenomai co-kernel. When the requirement is harder — single-digit microsecond response, or determinism under cache-thrashing background load — a co-kernel architecture is the right tool. Xenomai installs alongside Linux: a small real-time nucleus has priority over the Linux kernel, and real-time threads run inside the nucleus with their own scheduler. Linux is treated as the lowest-priority thread of the real-time nucleus.

This is more invasive — you can't just install Xenomai from a package manager on most ARM platforms, you need a board-support package that's been adapted — but the latency floor is dramatically lower. Worst-case response of 6–10 µs on a modest Cortex-A is realistic.

The porting effort

A Xenomai port to a new ARM SoC family is a substantial but bounded task. The components that need attention:

• Interrupt pipeline. Xenomai uses Adeos (I-pipe) to interpose between the SoC's interrupt controller and Linux. This requires SoC-specific patches in the architecture-dependent layer.
• Timer subsystem. The real-time nucleus needs a high-resolution timer that the Linux side cannot stall. Most modern ARM SoCs have the ARM generic timer architecture that works well; older SoCs may need a board-specific timer to be carved out.
• SMP affinity. Real-time threads should be pinned to a CPU not shared with Linux housekeeping work. On a quad-core SoC, dedicating one core to the real-time nucleus and three to Linux is a common pattern.
• Driver coverage. Drivers used in the real-time path (GPIO, SPI to motor controller, Ethernet to TSN switch) must be real-time-aware. Vanilla Linux drivers will work, but their latency under Xenomai is no better than under PREEMPT_RT — the real wins come from purpose-written real-time-domain drivers.

For an Cortex-A15-class platform with a typical sustainment workload set, the porting effort is roughly six to twelve weeks of engineering for a competent team, plus a robust regression suite that measures latency under stress.

Validation discipline

A common failure mode is to declare victory after a clean benchmark run. The realistic stress test sequence is:

1. Run the application's real-time threads.
2. Saturate the non-real-time cores with compute load (kernel compile, video transcode).
3. Saturate the file system with bursty I/O.
4. Saturate the network with full-rate non-real-time traffic.
5. Run all of the above for at least 24 hours.
6. Plot the latency CDF, including the upper tail.

Worst-case is what matters; mean and even 99.9th percentile can mask the failure mode that ends up biting in deployment. A Xenomai port that holds 10 µs at the 99.99th percentile but spikes to 800 µs once every six hours is not actually deterministic, and the spike will eventually correspond to the wrong moment in a mission.

When this is overkill

For everything that does not have a firm sub-millisecond requirement, PREEMPT_RT is the better answer. The engineering and maintenance overhead of a co-kernel architecture is real, and most sustainment workloads — even ones with "real-time" in their requirements documents — do not actually need the co-kernel. The discipline is to measure first, choose second.