Build and Run

This tutorial assumes that you already have basic git experience and are working on a Linux system.

The canonical URL for the upstream Ghaf git repository is https://github.com/tiiuae/ghaf. To try Ghaf, you can build it from the source.

Prerequisites

System Requirements

Operating System: Linux (NixOS recommended, but Ubuntu/Debian/Fedora also work)
Architecture: x86_64 (for building all targets including cross-compilation to AArch64)
Disk Space: Minimum 20GB free space
Memory: 8GB RAM minimum, 16GB recommended
Network: Stable internet connection for downloading dependencies and binary cache

Required Software Installation

Clone the Ghaf repository:

git clone https://github.com/tiiuae/ghaf.git
cd ghaf

Install Nix Package Manager (if not already installed):
Terminal window
```
curl -L https://nixos.org/nix/install | sh
```

Enable Nix Flakes by adding to your nix configuration:

mkdir -p ~/.config/nix
echo "experimental-features = nix-command flakes" >> ~/.config/nix/nix.conf

Configure Binary Cache for faster builds:

# Add to ~/.config/nix/nix.conf
echo "substituters = https://ghaf-dev.cachix.org https://cache.nixos.org" >> ~/.config/nix/nix.conf
echo "trusted-public-keys = ghaf-dev.cachix.org-1:S3M8x3no8LFQPBfHw1jl6nmP8A7cVWKntoMKN3IsEQY= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" >> ~/.config/nix/nix.conf

Verify your setup by viewing available targets:
Terminal window
```
nix flake show
```

Quick Start: Primary Build Targets

Lenovo X1 Carbon Gen 11 (Recommended for x86_64)

This is the primary reference device for Ghaf on x86_64 architecture.

Build the image (5-15 minutes with binary cache, 60-120 minutes from source):
Terminal window
```
nix build .#lenovo-x1-carbon-gen11-debug
```
Prepare USB boot media:
Terminal window
```
./packages/pkgs-by-name/flash-script/flash.sh -d /dev/<YOUR_USB_DRIVE> -i result/ghaf-lenovo-x1-carbon-gen11-debug.img
```
⚠️ Warning: Replace <YOUR_USB_DRIVE> with your actual USB device (e.g., /dev/sdb). This will erase all data on the USB drive.
Boot from USB: Insert the USB drive and boot your Lenovo X1 Carbon Gen 11 from the USB device.

NVIDIA Jetson AGX Orin (Primary AArch64 target)

This is the primary reference device for Ghaf on AArch64 architecture, built using cross-compilation.

Build flash script (10-30 minutes with binary cache, 90-180 minutes from source):

# For 32GB AGX version (cross-compiled):
nix build .#nvidia-jetson-orin-agx-debug-from-x86_64-flash-script

# For 64GB AGX version (cross-compiled):
nix build .#nvidia-jetson-orin-agx64-debug-from-x86_64-flash-script

Set up hardware connections:
- Connect power supply via USB-C
- Connect host computer to Jetson via USB-C (for flashing)
- Connect Micro-USB cable for serial console access
Put device in recovery mode: See NVIDIA’s Force Recovery Mode guide
Flash the device:
Terminal window
```
cd result
sudo ./flash.sh
```

Alternative Build Targets

Virtual Machine (For Testing)

Before building for physical hardware, you can test Ghaf in a virtual machine:

Run VM directly (5-15 minutes with binary cache, 45-90 minutes from source):
Terminal window
```
nix run .#packages.x86_64-linux.vm-debug
```

This creates a ghaf-host.qcow2 overlay disk image in your current directory. For a clean shutdown, use the QEMU menu: Machine → Power Down.

Build Time Optimization

Using Binary Cache (Recommended)

Ghaf provides pre-built binaries through a binary cache, dramatically reducing build times:

# Build with binary cache (default configuration)
nix build .#lenovo-x1-carbon-gen11-debug

# Explicitly specify binary cache
nix build .#lenovo-x1-carbon-gen11-debug \
  --option substituters "https://ghaf-dev.cachix.org https://cache.nixos.org"

Expected Build Times

Target Type	With Binary Cache	From Source
x86_64 targets	5-15 minutes	60-120 minutes
AArch64 cross-compiled	10-30 minutes	90-180 minutes
VM targets	3-10 minutes	45-90 minutes
Installer images	8-20 minutes	90-150 minutes

Cross-Compilation Benefits

No ARM64 hardware needed: Build AArch64 targets on x86_64 development machines
Faster than emulation: True cross-compilation is faster than QEMU emulation
CI/CD friendly: Standard x86_64 infrastructure can build ARM64 targets

Running Ghaf Image for Generic x86 Computer

For general x86_64 hardware (not Lenovo-specific):

Build the image:
Terminal window
```
nix build .#generic-x86_64-debug
```

Flash to USB:

./packages/pkgs-by-name/flash-script/flash.sh -d /dev/<YOUR_USB_DRIVE> -i result/ghaf-generic-x86_64-debug.img

Boot from USB.

./packages/flash/flash.sh -d /dev/<YOUR_USB_DRIVE> -i result/<IMAGE_NAME>

Boot the computer from the USB media.

Running Ghaf Image for Lenovo X1

Lenovo X1 is the reference x86_64 device for the Ghaf project.

Do the following:

To build the target image, run the command:

nix build github:tiiuae/ghaf#lenovo-x1-carbon-gen11-debug

After the build is completed, prepare a USB boot media with the target image you built using the flash.sh script:
Terminal window
```
./packages/flash/flash.sh -d /dev/<YOUR_USB_DRIVE> -i result/<IMAGE_NAME>
```
Boot the computer from the USB media.

Ghaf Image for NVIDIA Jetson Orin AGX

Before you begin:

Check device-independent prerequisites.
If you use a new device, flash bootloader firmware first. Then you can build and run a Ghaf image.

Flashing NVIDIA Jetson Orin AGX

Run the command:
Terminal window
```
nix build github:tiiuae/ghaf#nvidia-jetson-orin-agx-debug-flash-script
```
The above command is for AGX 32GB version. For the AGX 64GB RAM version please use:
Terminal window
```
nix build github:tiiuae/ghaf#nvidia-jetson-orin-agx64-debug-flash-script
```
It will build the Ghaf image and bootloader firmware, and prepare the flashing script. Give “yes” answers to all script questions. The building process takes around 1,5 hours.
Set up the following connections:
1. Connect the board to a power supply with a USB-C cable.
2. Connect a Linux laptop to the board with the USB-C cable.
3. Connect the Linux laptop to the board with a Micro-USB cable to use serial interface.
For more information on the board’s connections details, see the Hardware Layout section of the Jetson AGX Orin Developer Kit User Guide.
After the build is completed, put the board in recovery mode. For more information, see the Force Recovery Mode section in the Jetson AGX Orin Developer Kit User Guide.
Run the flashing script:
Terminal window
```
sudo ~/result/bin/flash-ghaf-host
```
There is a time-out for this operation, so run the script within one minute after putting the device in recovery mode. If you got the error message “ERROR: might be timeout in USB write.”:
1. Reboot the device and put it in recovery mode again.
2. Check with the lsusb command if your computer can still recognize the board, and run the flash script again.
Restart the device after flashing is done.

Building and Running Ghaf Image for NVIDIA Jetson Orin AGX

After the latest firmware is flashed, it is possible to use a simplified process by building only the Ghaf disk image and running it from external media:

To build the target image, run the command:
Terminal window
```
nix build github:tiiuae/ghaf#nvidia-jetson-orin-agx-debug-from-x86_64
```
The above command is for AGX 32GB version. For the AGX 64GB RAM version please use:
Terminal window
```
nix build github:tiiuae/ghaf#nvidia-jetson-orin-agx64-debug
```
After the build is completed, prepare a USB boot media with the target image you built using the flash.sh script:
Terminal window
```
./packages/flash/flash.sh -d /dev/<YOUR_USB_DRIVE> -i result/sd-image/<IMAGE_NAME>
```
Boot the hardware from the USB media.

In the current state of Ghaf, it is a bit tricky to make NVIDIA Jetson Orin AGX boot Ghaf from a USB if the same thing has already been flashed on the boards’s eMMC. To succeed, you can change partition labels on eMMC (or optionally wiping everything away by formatting):

Check the current labels:

lsblk -o name,path,fstype,mountpoint,label,size,uuid

Change the ext4 partition label:
Terminal window
```
e2label /dev/mmcblk0p1 nixos_emmc
```
Change the vfat partition label:
Terminal window
```
dosfslabel /dev/mmcblk0p2 ESP_EMMC
```

Verify the labels that were changed:

lsblk -o name,path,fstype,mountpoint,label,size,uuid

After these changes NVIDIA Jetson Orin AGX cannot boot from its internal eMMC. It will boot from the USB device with the correct partition labels.

Building Ghaf Image for NXP i.MX 8MP-EVK

Before you begin, check device-independent prerequisites.

In the case of i.MX8, Ghaf deployment consists of creating a bootable SD card and USB media with the Ghaf image:

To build and flash the Ghaf image:
1. Run the nix build .#packages.aarch64-linux.imx8mp-evk-release command.
2. Prepare the USB boot media with the target HW image you built: ./packages/flash/flash.sh -d /dev/<YOUR_USB_DRIVE> -i result/<IMAGE_NAME>.
Insert an SD card and USB boot media into the board and switch the power on.

Building Ghaf Image for Microchip Icicle Kit

Before you begin:

Check device-independent prerequisites.
Make sure HSS version 0.99.35-v2023.02 is programmed in your board eNVM. The version can be seen in the pre-bootloader log. Check the video guide to build HSS and program the eNVM: How to build HSS and program the eNVM?

In the case of the Icicle Kit, Ghaf deployment consists of creating an SD image with U-Boot and Linux kernel from Microchip, and Ghaf-based NixOS rootfs:

Build a Ghaf SD image:

a. Run the nix build .#packages.riscv64-linux.microchip-icicle-kit-release command to release the image. b. Run the nix build .#packages.riscv64-linux.microchip-icicle-kit-debug command to debug the image.
Flash the Ghaf SD image:
- If you want to use a SD card:
  - Prepare the SD card with the target HW image you built: ./packages/flash/flash.sh -d /dev/<YOUR_SD_CARD> -i result/<IMAGE_NAME>.
  - Insert an SD card into the board and switch the power on.
- If you want to use the onboard MMC:
  - You can directly flash a NixOS image to an onboard MMC card: ./packages/flash/flash.sh -d /dev/<YOUR_MMC_DEVICE> -i result/<IMAGE_NAME>.

For more information on how to access the MMC card as a USB disk, see MPFS Icicle Kit User Guide.