Network booting a Raspberry Pi 4

As highlighted in the previous articles, the flashing process for a Raspberry Pi 4 is far from ideal. It demands manual involvement from the developer and this article aims to fix that. We want as little manual involvement as possible for automation purposes.

Luckily, the firmware on RPi 4 supports network booting which enables it to fetch the boot dependencies from the network, without the need of an SD card. Since the will fetched from a network share, this opens up the possibility of modifying the OS files from the development machine without needing to touch the board. This is exactly what we want, which is why we’ll use network boot to send our OS kernel to the RPi.

Configuring the development machine #

For network boot, the RPi expects two things from the development machine:

• It should be on the same network.
• It should be able to serve files over the Trivial File Transfer Protocol (TFTP).
• Required boot files should be present on the network share.

Doing the networking #

For using network boot on the RPi, Dynamic Host Configuration Protocol (DHCP) is the default as it makes sense for people using multiple of these boards on their home networks. However, for OS development, static IPs are better suited because of the following reasons.

• A DHCP server makes sense in a network having multiple devices with a need for dynamic IP allocation.
• In this setup, RPi will form a point-to-point network with the development machine. There is no possibility of another device joining the network which eliminates the need of dynamic IP allocation.
• Static IP assignment is easier to configure as compared to introducing a DHCP server which requires modifying the host OS’ networking stack.

Considering this, I decided use an IPv4 addressed network in the 192.168.1.x subnet with the following IP address allocations.

Now, we’ll create a new ethernet connection to configure our development machine’s ethernet interface, enp2s0 for me. For NetworkManager, the command is:

$ nmcli con add con-name rpi-dev type ethernet ifname enp2s0 \
    ipv4.method manual \
    ipv6.method ignore \
    ipv4.addresses 192.168.1.1/24

Connection 'rpi-dev' (20f0142f-b920-4341-9b6c-66d409b1ec77) successfully added.

enp2s0 should be replaced with the name of your ethernet interface. Use ip link show to list out interfaces on your machine.

We can verify that it works by enabling the connection and checking the IP address assigned to the interface.

$ nmcli con up rpi-dev
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)

$ ip addr show dev enp2s0
2: enp2s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 02:00:00:00:00:01 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/24 brd 192.168.1.255 scope global noprefixroute enp2s0
       valid_lft forever preferred_lft forever

As we can see, our ethernet interface got the IP we assigned to it and it will be now visible to all devices on the 192.168.1.x subnet as 192.168.1.1.

Preparing the TFTP server #

With the networking taken care of, we can move on to the TFTP server side of things. On Arch Linux, the it can be installed through the tftp-hpa package which contains the binaries for both the client (tftp) and server (in.tftpd).

To start a TFTP server instance, we issue the following commands.

$ mkdir ~/public/rpi-dev/
$ sudo in.tftpd --listen --secure --user $USER ~/public/rpi-dev

The command shown above creates a TFTP server listening on localhost:69 as $USER, hosting files from ~/public/rpi-dev/. Read the manpage for tftpd for more information.

If everything goes well, a tftpd background process will be spawned which can be checked using pgrep. If not, then the command would exit with a non-zero status code like any other Linux application. We can verify that the server works by using the tftp to connect as a client as shown below.

$ touch ~/public/rpi-dev/placebo
$ tftp localhost -c get placebo

This command tries to fetch a file called placebo from the TFTP server and save it in the current working directory. If everything goes well, the command will exit with a zero status code and without any output. For further information, read the tftp manpage.

Preparing required boot files #

The RPi has an unusual boot sequence where the GPU bootstraps the board and starts up the CPU. Therefore, when the board is powered on, it will look for the GPU firmware files first. These typically include start4.elf, fixup4.dat, config.txt and the device-tree blob for the board. Therefore, we must ensure that the TFTP server has these files.

Lucky for us, the Raspberry Pi Foundation has these files on GitHub which we can simply place in our TFTP share.

$ wget https://github.com/raspberrypi/firmware/archive/master.zip
$ unzip master.zip
$ cp -r firmware-master/boot/* ~/public/rpi-dev/

It is recommended to download the zip instead of cloning as the repository as the git history is huge for this repository.

Now, we’ll copy our OS kernel to the TFTP share and make appropriate modifications to config.txt. For testing, we’ll use a custom bare-metal OS kernel that blinks the ACT LED on the board. The project source is on GitHub and we’ll use the blink.img kernel.

$ cp blink.img ~/public/rpi-dev/

The kernel is 64-bit, so we’ll have to configure the firmware accordingly through config.txt and place it in the TFTP share.

[all]
# Enable 64-bit mode
arm_64bit=1

# Use the blink kernel instead of kernel8.img
kernel=blink.img

That sums up about everything we need to do on the development machine. For further information about the configuration paramaters, refer to the config.txt docs .

Setting up the Raspberry Pi #

With the development machine configured, now we need to set up the RPi so that it does network boot in our small network. To do this, we need to modify the bootloader configuration stored in its EEPROM.

This step requires Raspberry Pi OS and an SD card. I went with a headless installation of Raspberry Pi OS Lite , which was trivial to do using the Raspberry Pi Imager utility and its built-in SSH and Wi-Fi configuration.

Once the RPi is up, we can configure the bootloader using the following command.

# rpi-eeprom-config --edit

This will bring up a text editor containing the configuration variables. The ones which we need to add/modify are:

[all]
BOOT_UART=1
BOOT_ORDER=0xf21

TFTP_PREFIX=1
TFTP_IP=192.168.1.1
CLIENT_IP=192.168.1.2
SUBNET=255.255.255.0

The above-stated configuration does the following:

• Enable UART logs for the bootloader
• Set the boot order to - SD card > Network boot > Restart
• Specify TFTP server details and disable DHCP in favour of static IP

For further information, refer to the official docs on configuration properties .

It is recommended to update the bootloader , especially for old boards.

After saving these changes, raspi-eeprom-config will ask for a reboot, which will save our changes to the EEPROM.

Once the board has rebooted, we can hook up a TTL adapter to the UART pins of the RPi. Starting a serial session in at 115200 baud in 8-N-1 mode should yield the bootloader’s output. This is helpful as it gives us insight into the RPi’s boot process.

Here are the important file transactions from the serial logs .

TFTP_GET: 02:00:00:00:00:01 192.168.1.1 config.txt
RX: 14 IP: 0 IPV4: 7 MAC: 1 UDP: 1 UDP RECV: 1 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
TFTP: complete 35
RX: 16 IP: 0 IPV4: 9 MAC: 3 UDP: 3 UDP RECV: 3 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
Read config.txt bytes       35 hnd 0x0

TFTP_GET: 02:00:00:00:00:01 192.168.1.1 start4.elf
RX: 22 IP: 0 IPV4: 15 MAC: 7 UDP: 7 UDP RECV: 7 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
TFTP: complete 2253088
RX: 40 IP: 0 IPV4: 22 MAC: 7 UDP: 7 UDP RECV: 7 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
Read start4.elf bytes  2253088 hnd 0x0

TFTP_GET: 02:00:00:00:00:01 192.168.1.1 fixup4.dat
RX: 40 IP: 0 IPV4: 22 MAC: 7 UDP: 7 UDP RECV: 7 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
TFTP: complete 5397
RX: 45 IP: 0 IPV4: 27 MAC: 12 UDP: 12 UDP RECV: 12 IP_CSUM_ERR: 0 UDP_CSUM_ERR: 0
Read fixup4.dat bytes     5397 hnd 0x0

We can see that the bootloader fetches config.txt, start4.elf and fixup4.dat from the TFTP server and executes them. Then the GPU firmware loads the kernel into memory after which the ACT LED starts blinking.

Conclusion #

With network boot, we have solved the biggest pain point in the RPi flashing process, that is, SD cards. Now, booting the RPi with an updated kernel is a simple task of copying the file to the TFTP shared directory. Building upon the previous article, we can automate this using entr to copy the kernel after building it.

ls include/**/*.h src/**/*.c link.ld Makefile | entr -s 'make clean all && cp build/kernel8.img ~/public/rpi-dev/'

We’ll still need to reset the board manually as of now, but fret not. The next article will automate this bit as well giving us a fully automated flashing process.

Thank you for reading this article, and please feel free to share any suggestions or questions in the comments section.

References #

• Network Booting Raspberry Pi 4B with Upstream Kernel - Stephen Brennan
• Bare Metal Raspberry Pi 3B+: Network Boot - Mete Balci