Managed GPU Partitions on NVIDIA Jetson Orin

Ghaf can divide the CUDA streaming multiprocessor (SM) resource in the split gpu-vm into two equal CUDA Green Contexts. A manager in gpu-vm owns the contexts and runs trusted, Nix-built workload plugins on their streams. Native processes and containers use the same local protocol.

Ghaf owns the platform integration: NixOS options, service confinement, JetPack binding, and the Container Device Interface (CDI) specification. The GPU partition manager repository owns the daemon, client, protocol, plugin SDK, and mock-CUDA tests. Workloads, OCI images, scenarios, and hardware measurements live in the downstream examples repository.

This feature targets NVIDIA Jetson Orin AGX and Orin NX. The complete GA10B GPU remains assigned to one compute-only gpu-vm; disp-vm remains the display owner.

Caution

Green Contexts provide cooperative SM placement, not security isolation, fault containment, guaranteed concurrency, forward progress, or memory-bandwidth isolation. An unrestricted CUDA process can interfere with both managed slots.

Architecture

                         gpu-vm

  native client                         managed container
  gpu-partition-run                     gpu-partition-run
           |                                   |
           +------------- AF_UNIX -------------+
                               |
             /run/gpu-partition-manager/control.sock
                               |
                  gpu-partition-manager
                    /                   \
              slot 0 worker         slot 1 worker
              Green Context         Green Context
              half of SMs           half of SMs
                    \                   /
                     passed-through GA10B

At startup, the daemon queries CUDA device 0, requires two equal SM groups with no remainder, creates one Green Context and worker per group, loads only the configured immutable plugins, and then opens its group-restricted socket. It exits with EX_CONFIG (78) when the queried geometry cannot be divided as required; systemd does not restart that configuration failure.

See the manager’s architecture, protocol, plugin API, and security model for the authoritative contracts.

CDI Devices

The build-generated /etc/cdi/nvidia.json exposes these device names:

CDI device	GPU nodes	Manager socket	Purpose
nvidia.com/gpu=all	Complete JetPack set	No	Direct, unrestricted CUDA
nvidia.com/gpu=managed	None	Yes	Jobs submitted through the manager

The unrestricted device preserves the existing gpu=all behavior. It also lets a process bypass the manager. The managed device injects the client and socket but no /dev/nvgpu, /dev/nvhost-*, /dev/nvmap, DRM render, or host1x-fence nodes.

The CDI spec is immutable and generated at build time. Docker reads CDI specs only from /etc/cdi; Ghaf does not enable Docker’s writable /var/run/cdi directory or install a privileged runtime discovery helper.

Configure Ghaf

Both features are opt-in for general configurations. The existing NX debug target enables them with the manager repository’s mock plugin so CI evaluates and builds the complete integration. A downstream module supplies real trusted plugins:

ghaf.hardware.nvidia.passthroughs.gpu_vm = {
  containerRuntime.enable = true;
  partitionManager = {
    enable = true;
    plugins = [ myWorkloadPlugin ];
  };
};

Each plugin package must:

• expose gpuPartitionPluginName;
• expose requiredPluginAbiVersion matching the manager ABI; and
• install lib/gpu-partition-manager/plugin.so.

Ghaf validates the metadata during evaluation and the shared-object path while building the VM closure. The manager repository exports the ABI-v1 SDK through lib.mkSdk; Ghaf also makes it available as pkgs.gpu-vm-partition-manager-sdk inside configured package sets.

The runtime is rootful Docker. By default only root controls Docker. This optional setting adds ghaf to the docker group:

ghaf.hardware.nvidia.passthroughs.gpu_vm.containerRuntime = {
  enable = true;
  addGhafUserToDockerGroup = true;
};

Danger

Membership in the docker group is root-equivalent inside gpu-vm: a user can start a privileged container and mount the guest filesystem. Enable it only when that matches the image’s trust model.

These options have an effect only when the split ghaf.hardware.nvidia.passthroughs.gpu_vm.enable topology is selected. They do not apply to the combined accelerated gui-vm topology.

Build and Deploy an Example

The Ghaf NX debug target builds the manager, probe, plugin validation, and CDI generator with the manager-owned mock plugin. It is integration coverage, not a GPU workload example:

nix build .#nvidia-jetson-orin-nx-debug-from-x86_64

The AGX target and NX release target leave both options disabled. Use a downstream configuration for real workloads, or build the complete configurations maintained in the examples repository:

AGX

git clone https://github.com/tiiuae/ghaf-gpu-partitioning-examples
cd ghaf-gpu-partitioning-examples
nix build .#nvidia-jetson-orin-agx-gpu-partitioning-example
nix build .#nvidia-jetson-orin-agx-flash-script \
  --max-jobs 8 -o result-flash

NX

git clone https://github.com/tiiuae/ghaf-gpu-partitioning-examples
cd ghaf-gpu-partitioning-examples
nix build .#nvidia-jetson-orin-nx-gpu-partitioning-example
nix build .#nvidia-jetson-orin-nx-flash-script \
  --max-jobs 8 -o result-flash

Follow the examples repository’s user guide for image selection, recovery-mode checks, flashing, and its reproducible scenarios. Flashing is destructive; verify the board and recovery USB identity before running a flash script.

Connect using the configured VM hostname rather than a topology-dependent IP:

ssh ghaf@NET_VM_ADDRESS
ssh ghaf@gpu-vm

Operate the Manager

The client uses /run/gpu-partition-manager/control.sock by default:

gpu-partition-run --help
gpu-partition-run list
gpu-partition-run status
gpu-partition-run WORKLOAD [ARGS...]
gpu-partition-run --slot 1 WORKLOAD [ARGS...]
gpu-partition-run cancel JOB_ID

The scheduler runs one job per slot and queues bounded work per slot. Automatic placement selects the idle or shorter queue. Disconnecting a waiting client or sending Ctrl+C requests cancellation. Refer to the manager’s protocol page for exact limits, status values, and compatibility rules.

The Green Context geometry probe is installed only when partitionManager.enable is true:

gpu-vm-green-context-probe

It queries the real CUDA resource geometry independently of the manager and is useful for board bring-up. It does not prove concurrent execution or workload isolation.

Run Containers

Use the managed device to submit a configured workload without granting the container direct GPU access:

docker run --rm --device nvidia.com/gpu=managed IMAGE \
  /opt/ghaf/bin/gpu-partition-run WORKLOAD [ARGS...]

The v1 interface expects the process in the rootful container to run as root so it can connect to the group-restricted socket. The plugin executes inside the trusted manager process, not inside the container process.

Use the unrestricted device for applications that initialize CUDA directly:

docker run --rm --device nvidia.com/gpu=all IMAGE

Service and Troubleshooting

systemctl status gpu-partition-manager
journalctl -u gpu-partition-manager -b
gpu-partition-run status

The service runs as the dedicated gpu-partition user with video group access. Its systemd sandbox uses a private network namespace, permits only AF_UNIX, removes capabilities, and restricts namespaces, kernel controls, IPC, and writable filesystem access. Do not enable MemoryDenyWriteExecute; CUDA’s PTX JIT needs executable mappings.

• Exit status 78 means the runtime geometry is unsupported. Inspect the total SM count, split, remainder, and Green Context readback.
• For repeated failures, find the first CUDA or plugin error and inspect the guest kernel log for nvgpu faults.
• If a managed container cannot connect, check service readiness, socket group access, and the managed entry in /etc/cdi/nvidia.json.
• If a managed container contains GPU device nodes, stop the test and inspect the CDI spec; the managed entry must contain mounts only.
• Record any simultaneous gpu=all workload as unmanaged interference.

Development and Validation

Keep ownership boundaries explicit:

• Ghaf owns platform modules, service confinement, CDI generation, and target integration.
• The manager repository owns source, wire protocol, plugin ABI, constructors, and mock-CUDA integration tests.
• The examples repository owns plugins, images, scenarios, and measurements.

Validate manager state-machine changes without hardware:

git clone https://github.com/tiiuae/ghaf-gpu-partition-manager
cd ghaf-gpu-partition-manager
nix build -L .#checks.x86_64-linux.tests

The existing NX debug target enables both options with the manager-owned mock plugin. Building it compiles the manager and probe, validates the plugin contract, and generates both CDI devices. AGX, NX release, and other configurations remain opt-in. Downstream configurations replace the default mock plugin with trusted workload plugins. Integration builds are not a replacement for hardware testing.

Before claiming support on a board, verify the queried split, direct and managed CDI paths, simultaneous managed jobs, cancellation, endurance, guest nvgpu errors, daemon restarts, and gpu-vm restarts. Report measurements as observations rather than isolation guarantees.