NVIDIA Jetson AGX Orin GPU and Display Virtualization

How the Jetson AGX Orin GPU and display path are assigned to either a split gpu-vm/disp-vm pair or one accelerated gui-vm. The display owner runs NVIDIA’s guest KMS stack while the DCE R5 and physical display hardware remain host-owned. This builds on GPU Passthrough to gpu-vm and shares its BPMP-virtualisation and ghaf-nvidia-qemu-bpmp infrastructure.

Note

Verified on the NVIDIA Jetson AGX Orin devkit with DisplayPort and passive DP->HDMI output. In split mode, disp-vm starts a persistent kmscube KMS owner. In accelerated mode, gui-vm reaches the COSMIC login and desktop, accepts keyboard and pointer input, uses the hardware cursor keyhole, and presents animated Wayland clients. Long-duration compositor/display stability remains under test.

Supported topologies

Topology	GPU, host1x, media	Display and DCE relay	Desktop
Split	gpu-vm	disp-vm	Display bring-up/KMS owner
Accelerated	gui-vm	gui-vm	COSMIC plus App VMs

The modes are exclusive. disp-vm depends on gpu-vm for the shared host BPMP allowlist, module blacklist, and passthrough overlay. Assertions reject a standalone gpu-vm, a standalone disp-vm, or simultaneous split and combined owners. Exactly one display VM sets GHAF_DCE_GUEST=1 and consumes /dev/dce-host events.

Why not plain vfio passthrough

On Tegra234 the display controller is not an independent device like the GPU. Mode setting, plane programming, and hotplug are owned by the DCE (Display Controller Engine), an R5 coprocessor running NVIDIA firmware. The CPU-side driver (nvdisplay, i.e. nvkms/nvidia-drm) does not program display registers directly for these — it sends DISPRM RPCs to the DCE over IPC (tegra-dce IVC channels), and the DCE touches the hardware.

That breaks both naive approaches:

• Passing the display MMIO through (display@13800000 bound to vfio-platform) resets the unit under the R5’s feet and halts the DCE: host loses display, R5 never recovers without a full reboot.
• Running the DCE stack in the guest would require the guest to own DCE boot, its carveouts, and its IPC — all tightly coupled to host-owned early boot firmware.

So the split: host keeps the display hardware and the DCE; the guest’s display driver stack is plumbed to the host’s DCE over a relay. The guest runs the full nvdisplay OOT stack and believes it owns the display; every DISPRM RPC and asynchronous DCE event crosses the VM boundary instead.

Architecture

   GUEST display owner (disp-vm or gui-vm)                HOST (Jetson L4T kernel 6.6)
  +--------------------------------------------+   +----------------------------------------+
  |  nvidia-drm / nvkms (nvdisplay OOT,        |   |                                        |
  |  patched: address policy, flip completion) |   |                                        |
  |        | DISPRM RPCs (sync)                |   |                                        |
  |        v                                   |   |                                        |
  |  tegra-dce (ipc_send redirect hook)        |   |                                        |
  |        v                                   |   |                                        |
  |  dce-guest-proxy.ko                        |   |                                        |
  |    forward:  MMIO window write ------------+---+--> QEMU DCE bridge (nvidia_dce_guest)  |
  |    reverse:  virtual SPI <-----------------+---+--     |         ^                      |
  |        |                                   |   |       v         | ch3 events           |
  |        v                                   |   |  /dev/dce-host (dce-host-proxy.ko)     |
  |  tegra_dce_client_ipc_inject               |   |       |         ^                      |
  |    -> nvkms RM_EVENT callback              |   |       v         |                      |
  |  (vblank, flip, hotplug delivered          |   |  tegra-dce client API (CPU_RM)         |
  |   as if from the IVC channel)              |   |       |         |                      |
  |                                            |   |       v         |                      |
  |  scanout surfaces in identity carveouts    |   |     DCE R5 firmware  ==> display HW    |
  |  (GPA == HPA), high-IOVA offset            |   |                                        |
  +--------------------------------------------+   |  dce-iso-anchor.ko: SMMU domains for   |
                                                   |  display ISO (SID 1) / NISO (SID 7),   |
                                                   |  high-IOVA -> carveout maps, MC SID    |
                                                   |  retag of scanout readers              |
                                                   +----------------------------------------+

Four cooperating pieces:

1. dce-guest-proxy.ko (guest) installs the tegra_dce_ipc_send_redirect hook exported by the patched guest tegra-dce, so every synchronous DCE client IPC goes into a shared MMIO window instead of a (non-existent) guest IVC channel. A virtual SPI wakes its threaded handler when the bridge publishes a reverse-window event; the handler injects asynchronous events back into tegra-dce’s client callbacks via tegra_dce_client_ipc_inject — identical to native IVC delivery from the driver’s point of view.
2. QEMU DCE bridge (nvidia_dce_guest device in ghaf-nvidia-qemu-bpmp) exposes the existing trapped MMIO window, with a RAM overlay for only the 4 KiB reverse-event payload. Bridge creation is opt-in through GHAF_DCE_GUEST=1; only the VM that owns the display path may set it and open /dev/dce-host. A doorbell write runs the forward round-trip synchronously: it packages the request and write()s it to /dev/dce-host, which returns only when the real DCE has answered. A pump thread poll()s the same fd for asynchronous events, publishes them into the reverse window, and asserts the virtual SPI. It sleeps on the guest acknowledgement before publishing the next event. Keeping the 4 KiB reverse payload RAM-backed is essential: placing it in an IO memory region turns each vblank copy into roughly 1,000 KVM exits.
3. dce-host-proxy.ko (host) registers as a tegra-dce client (CPU_RM interface) and relays each guest request to the real DCE via tegra_dce_client_ipc_send_recv. Unsolicited DCE notifications (channel type 3: hotplug, vblank, RM events) are queued into an event ring exposed over the same char device with peek-then-pop reads, so a failed copy never loses an event.
4. dce-iso-anchor.ko (host) owns the SMMU/memory-controller plumbing that makes the DCE’s and the display engine’s DMA agree with the guest’s view of memory (next section).

The relay window

A single 0x5000-byte MMIO region with a RAM overlay at the reverse-event payload, one transaction at a time (serialized by a guest spinlock — DCE IPC can be issued from atomic context):

Offset	Field	Purpose
0x0000	TX buffer	request payload (up to 4KB, DCE frames are opaque)
0x1000	RX buffer	response payload
0x2000/0x2008	TX/RX size	u64 sizes; host writes actual RX length back
0x2010	return code	s32 from the host send_recv
0x2018	interface	DCE channel type
0x2100	doorbell	write triggers the synchronous forward round-trip
0x3000+	reverse window	EVT_SEQ/EVT_IFACE/EVT_SIZ/EVT_ACK + payload

The reverse window is single-slot with a sequence/ack handshake: the bridge publishes one event and bumps EVT_SEQ; the guest consumes it and writes the sequence back to EVT_ACK, which deasserts the virtual SPI; the bridge holds the next event (buffered in the host ring) until the ack catches up. Slow guest ⇒ delayed events, never lost. Neither endpoint polls the reverse MMIO window.

The host event ring is single-consumer. Do not enable the DCE bridge in a compute-only GPU VM, NetVM, or AdminVM. Multiple QEMU readers race after poll(), produce EAGAIN, and can consume a display event in the wrong VM.

Caution

Every word moved through this window is a trapping MMIO access (~µs each, serviced by QEMU), so the guest proxy copies only the live payload per RPC. An earlier version bulk-copied the full 12KB forward window both ways (~3000 vmexits ≈ 10ms per RPC); because nvkms issues DISPRM RPCs from its flip path while holding the display RM lock, that 10ms serialized async event delivery and capped real present at ~50Hz. Payload-only copies are what make a locked 60Hz possible.

Memory and SMMU architecture

The DCE and display engine DMA into guest-allocated memory. Three address spaces must be reconciled: the guest’s CPU/physical view, the R5’s view, and the display engine’s ISO/NISO SMMU streams.

• Identity carveouts (GPA == HPA). All guest memory the display path touches (instance memory, pushbuffers, notifiers, scanout surfaces) lives in fixed reserved-memory carveouts placed 1:1 (guest PA == host PA), removing one translation layer.
• High-IOVA address policy (guest nvdisplay patch). Natively nvkms hands the R5 display-SMMU IOVAs in a high window (0x7f_xxxx_xxxx). The guest mirrors this: every address handed to the DCE (ctxdma descriptors, pushbuffers, instance memory) is the CPU PA offset by the high-IOVA base, and the absolute ctxdma limit is computed after the offset. The host installs matching translations.
• dce-iso-anchor.ko creates DMA-less anchor devices via a runtime DT overlay that (a) claim the display ISO (smmu_iso, stream ID 1) and NISO (smmu_niso0, stream ID 7) SMMU streams with translating domains carrying the high-IOVA → carveout mappings, and (b) retag the physical scanout readers NVDISPLAYR/NVDISPLAYR1 to ISO SID 1 through stock interconnects entries (memory-controller SID override registers). Boot-lifetime by design: unloading could not restore the MC SID overrides.
• Split display-VM CMA. The two-VM display guest allocates display memory from 0xe2000000..0xe6000000. Both anchor domains therefore include 0x7fe2000000 -> 0xe2000000 for 64 MiB. Without this mapping, SID-7 context faults and host EMEM decode errors leave an active DP modeset black.
• 39-bit ISO limit. The display frontend truncates ISO addresses at bit 39, so scanout surfaces must be allocated below 4GB: the guest’s nvmap generic carveout (384MB at 0x98000000) keeps EGL surface allocations inside the ISO-mapped range. A higher-placed carveout scans out from a bit-truncated address — an smmu_iso fault storm and a white panel.
• Notifiers are plain WRITE. The core-channel completion notifier is forced to WRITE mode: the WRITE_AWAKEN variant makes the R5 abort completions the guest can never receive (the awaken interrupt path does not cross the relay).

Host-Memory Footprint

The identity mappings are removed from the host’s normal RAM before Linux boots. Count them separately from each VM’s ordinary QEMU RAM:

Mapping	Size	Accelerated	Split	Purpose
vm_cma	768 MiB	yes	yes	GPU, NVMAP, and guest CMA heaps
vm_hs	64 MiB	yes	yes	host1x semaphore/syncpoint shim
scanout	128 MiB	yes	yes	1:1 display scanout aperture
dispram_lo	736 MiB	no	yes	disp-vm kernel and working RAM
dispram_hi	416 MiB	no	yes	disp-vm display carveouts
Total		960 MiB	2112 MiB

The former vm_cma_vram mapping reserved another 4 GiB at 0x100000000, but no active guest heap used it after GPU allocations moved below 4 GiB. It is no longer present in the host overlay, guest memory node, QEMU VFIO arguments, or guest reserved-memory. The disp-vm banks are emitted only for the split topology; the combined gui-vm therefore releases a further 1152 MiB.

On an AGX accelerated target, removing the split-only banks increased host MemTotal from 29,303,456 KiB to 30,483,120 KiB: exactly 1,179,664 KiB after kernel accounting, matching the 1152 MiB DT reservation. The host and gui-vm remained healthy, and the live host DT contained vm_cma, vm_hs, and scanout, but no old VRAM or dispram nodes.

QEMU RAM Is a Separate Budget

The fixed mappings above do not replace a graphics VM’s ordinary microvm.mem allocation. QEMU exposes that allocation as a separate high guest memory bank. VFIO locks its address range, so lowering microvm.mem reduces the amount that can become unreclaimable on the host. Residency can be lazy: the complete range is locked even when not all pages are resident yet.

Measurements from the accelerated targets on 2026-08-05 illustrate the difference:

Board	gui-vm QEMU RAM	Host VmLck	Host VmRSS	Guest used	Guest available
AGX	6144 MiB	6144 MiB	6298 MiB	1336 MiB	4712 MiB
NX 16 GB	4096 MiB	4096 MiB	1845 MiB	1255 MiB	2723 MiB

All graphics system VMs already enable LZO-RLE zram at 25 percent of guest RAM with swappiness 10. Neither measured gui-vm was using zram. Zram can provide headroom after reducing QEMU RAM, but enabling or enlarging it alone does not release a host carveout or shrink the VFIO-locked range.

Tune normal VM RAM independently for each topology and workload:

• The AGX accelerated target can first test gui-vm at 4096 MiB, matching the current NX allocation. This lowers the locked ceiling by 2 GiB.
• On NX, test 3072 MiB before considering a smaller gui-vm; idle use does not cover COSMIC login, application, graphics, and first-boot peaks.
• In split mode, AGX defaults of 6000 MiB for gpu-vm and 4000 MiB for disp-vm can be tested against the existing NX allocations of 2048 MiB and 1536 MiB. Validate gpu-vm with the intended CUDA or unified-memory workload.
• Do not reduce the NX net-vm from 2048 MiB based on idle data. A 1024 MiB configuration previously OOM-killed during the logging and networking first-boot burst.

For each step, cold boot the board, compare host VmLck/VmRSS and guest MemAvailable, then exercise modesets, COSMIC login, applications, and the intended GPU workload. Treat OOM events, sustained zram use, memory-pressure stalls, or GPU/DCE faults as a failed size. Guest kernel mem= limits, offlining, ballooning, and huge pages do not by themselves shrink these static VFIO mappings; change microvm.mem to reduce the host ceiling.

Asynchronous events: hotplug and vblank

The DCE pushes unsolicited notifications (channel type 3) for hotplug, vblank and RM events. Two guest-visible consequences needed handling:

• Boot hotplug long-pulse. A panel already connected at boot produces no HPD edge, so the guest would never mode-set. A synthetic long-pulse is injected once at guest display bring-up.
• DP++ dual-mode detection falls through to the TMDS partner using the RM connect state rather than the DP-lib HPD guess, lighting passive DP->HDMI adapters (at TMDS-limited rates).

GPU-accelerated present

With the display path in place, the guest renders on the passed-through GA10B and presents through it:

• Graphics userspace is wired via hardware.graphics (l4t-3d-core + runtime dependencies, egl_external_platform.d), because the guest’s bring-your-own 6.12 kernel cannot enable hardware.nvidia-jetpack. eglinfo reports renderer NVIDIA Tegra Orin (nvgpu), GLES 3.2 / GL 4.6.
• GBM path: nixpkgs egl-gbm 1.1.3 replaces the L4T-bundled 1.1.0 (which heap-corrupts eglInitialize under mesa 26’s libgbm), patched with a single-device fallback — on Tegra the EGL device’s DRM node (tegra-drm on host1x) never path-matches a GBM device opened on the display node (nvidia-drm), so the stock strict device match always failed. Stock kmscube -D /dev/dri/card0 runs via a small LD_PRELOAD shim forcing no-modifier GBM surfaces (the EGLStream producer rejects modifier surfaces on this guest).
• Native flip completion. The window notifier uses WRITE_AWAKEN, and the stock R5 FLIP_OCCURRED callback is the sole DRM completion owner. There is no synthetic vblank completion, notifier polling, or two-vblank fallback.
• Continuous vblank stream. drm.vblankoffdelay=0 avoids a synchronous DCE notification disable/re-enable cycle between flips.
• Interrupt-driven reverse events. QEMU publishes one event, asserts a virtual SPI, and blocks for the guest ACK. The guest threaded IRQ handler injects the event into tegra-dce; neither endpoint polls the MMIO window.
• Aligned event payload. Only the 4 KiB reverse payload is RAM-backed and page-aligned. The control words remain MMIO, avoiding roughly 1,000 KVM exits for every vblank payload copy.

Accelerated COSMIC path

The combined gui-vm uses the NVIDIA GBM node for the compositor and keeps Mesa available for Wayland clients. Orin-specific integration includes:

• a fallback when L4T EGLDevice enumeration returns BAD_ALLOC;
• no-modifier GBM surfaces for the L4T EGL platform;
• timer-backed checks for implicit DMA-BUF fences that become readable without waking epoll; explicit sync remains event-driven;
• nonblocking eglut dispatch when NVIDIA EGL consumes shared Wayland-FD readiness on its buffer-release thread;
• VT-less seatd activation and evdev-only forwarding for the Logitech Unifying receiver; and
• a dedicated cursor PIO keyhole, separate from DCE-owned registers.

Use

The regular AGX targets enable split gpu-vm plus disp-vm. The accelerated desktop target is nvidia-jetson-orin-agx-accelerated-guivm (and its -from-x86_64 cross variant). It disables the split pair and assigns the combined capability to gui-vm.

Verify

On the host:

# proxy + anchors loaded
lsmod | grep dce            # tegra_dce, dce_host_proxy, dce_iso_anchor
# anchor carveout maps installed (ISO SID 1 + NISO SID 7), result "= 0"
dmesg | grep -E 'dce_iso_anchor|dce_niso_anchor'

The scanout readers’ MC SID retag (NVDISPLAYR/NVDISPLAYR1 → ISO SID 1) is confirmed functionally by the health check below: a failed retag shows up as EMEM address decode errors in the host log. (Raw MC registers aren’t read back with devmem — the release build keeps CONFIG_STRICT_DEVMEM=y, blocking /dev/mem access.)

In the display owner:

# list outputs first: in the Connectors table, use the `connected` row's
# id as <connector> and one of its mode names as <mode> (e.g. 32:3440x1440)
modetest -M nvidia-drm

# split mode starts this persistent owner automatically
systemctl status kms-owner

# GPU-accelerated present: spinning cube on the physical panel
eglinfo -B                  # renderer: NVIDIA Tegra Orin (nvgpu)
kmscube -D /dev/dri/card0

# real page-flip rate: steady ~60Hz
modetest -M nvidia-drm -s <connector>:<mode> -v

Health checks: zero Lost display notification in guest dmesg, zero nvdisplayr ... EMEM address decode error in host dmesg. A held mode survives cable replug and relights through the hotplug relay. In accelerated mode, systemctl status greetd and journalctl -u greetd cover COSMIC startup.

Debugging notes

• The R5 accepts one guest display init per host boot. Restarting gpu-vm without rebooting the host wedges the next init: reboot the host to iterate on the guest display stack.
• The host proxy’s event ring uses peek-then-pop reads: a QEMU bridge event read failure retries and sees the same event again — investigate, but nothing was dropped.
• A dark panel with a live mode usually means an address-translation problem (check host smmu_iso/smmu_niso faults and EMEM address decode error), not a mode-set failure: the DCE reports success while scanout reads garbage.

Split ownership details

Owner	Devices
GPU VM	GA10B, physical host1x, syncpoint shim, media engines
Display VM	nvdisplay, DCE proxy, display keyholes, scanout, connector

The split follows hardware measurement: CUDA requires physical host1x, while the display-only guest can run NVKMS without host1x syncpoints. The two VMs use disjoint VFIO devices and reserved-memory mappings. The accelerated topology instead retains host1x syncpoints and the hardware cursor in the same VM as COSMIC.