The mailbox

The Raspberry Pi (RPi) boards differ from conventional embedded systems. Their boot sequence starts with the VideoCore GPU and the ARM CPU boots up only after a couple of stages. This leads to the GPU handling the board’s low-level functionality, such as power and clock management. Since the kernel (running on the CPU) requires access to such functionality, the GPU provides a mechanism to the CPU to allow it to talk to the GPU, called the mailbox peripheral.

To talk to the GPU, the CPU has to send a mail to the mailbox. Upon receiving the request, the GPU places the response in another mailbox which the CPU can read to get the requested data.

The protocol #

The mailboxes act as pipes between the CPU and the GPU. The peripheral exposes two mailboxes, 0 and 1, to allow full-duplex communication. Each mailbox operates as a FIFO buffer with the capacity to store up to eight 32-bit words. Notably, mailbox 0 alone supports triggering interrupts on the CPU side, which makes it well-suited for CPU-initiated read operations. As a result, CPI-initiated write operations naturally get channelled through mailbox 1.

At the time of writing, no documentation or usage exists for the “opp is empty” IRQ. If you know what it does, please leave a comment.

Little documentation exists on the mailbox registers due to the closed-source nature of the GPU firmware. The bits of information presented here come from the Linux vcio driver , Broadcom’s ICRS graphics drivers , and the Raspberry Pi Foundation’s wiki page . Still, the device-tree sources of RPi’s Linux kernel reveals the memory placement of the mailboxes. They exist at 0x7e00b880 in the memory, occupying 0x40 bytes in total (0x20 bytes for each mailbox).

As shown in Fig. 1 of BCM2711 ARM peripherals datasheet , the SoC has various memory maps to choose from. Since peripheral offsets relative to the peripheral base remain unchanged for all the memory maps, it makes sense to express peripheral addresses relative to the peripheral base address.

#define PERIPHERAL_BASE 0x7c000000
#define MBOX_BASE       (PERIPHERAL_BASE + 0x200b880)
#define MBOX_SIZE       0x40

#define MBOX0_BASE (MBOX_BASE + 0x00)
#define MBOX1_BASE (MBOX_BASE + 0x20)

Mails #

A mail encapsulates the data and channel information within a 32-bit word. It serves as the fundamental quantum for the information exchange over the mailboxes. The diagram below describes the memory layout for a mail.

The C snippet below shows how to compose a mail and also defines macros for accessing the encoded fields.

#define MBOX_MAIL_CHANNEL_MASK_WIDTH 4
#define MBOX_MAIL_CHANNEL_MASK       ((1 << MBOX_MAIL_CHANNEL_WIDTH) - 1)
#define MBOX_MAIL_DATA_MASK          ~MBOX_MAIL_CHANNEL_MASK

uint32_t mbox_mail_compose(uint8_t channel, uint32_t data) {
    // Fill in the data bits
    uint32_t mail = data << MBOX_MAIL_CHANNEL_WIDTH;

    // Set the channel field
    mail |= channel & MBOX_MAIL_CHANNEL_MASK;

    return mail;
}

Channels #

A channel acts as a descriptor for the associated data within a mail. Given that the RPi boards support a diverse range of transactions via the mailbox interface, the channel serves as a means to categorize the transaction type. It helps the GPU discern how to process the incoming mail.

At the time of writing, the wiki documents two channels, out of which one has entered deprecation in favour of the other. This makes the property tags channel the sole choice for new applications.

Talking to the mailbox #

Communicating with the mailboxes requires the ability to exchange mails with it. Owing to the FIFOs’ finite capacities, doing this requires performing some safety-checks before interacting with the peripheral. These checks rely on the status register which provides feedback to the CPU by conveying the FIFO states.

Though the mailbox supports interrupts, this article will only touch upon the busy-wait implementations for simplicity’s sake.

Also, the C snippets discussed below will rely on the following macros to keep things concise.

#define REG(addr) *(volatile uint32_t*)(addr)

#define MBOX_RW(base)     REG(base + 0x00)
#define MBOX_STATUS(base) REG(base + 0x18)

Sending mails #

After sending the mail, a response arrives at the CPU’s read end of the pipe. Whether the FIFO can accommodate this response depends on its availability at the time of sending the mail. To ensure proper retrieval of the GPU’s response, the mail dispatch should not happen unless the receiving FIFO has space availability.

To achieve this, the status register’s bit 31 proves useful as it indicates whether the mailbox is full or not.

#define MBOX_STATUS_MBOX_FULL (1 << 31)

void mbox_write(uint32_t mail) {
    // Wait until the read end can accommodate new mails
    while (MBOX_STATUS(MBOX0_BASE) & MBOX_STATUS_MBOX_FULL)
        ;

    // Send the mail to the write end
    MBOX_RW(MBOX1_BASE) = mail;
}

Receiving mails #

The retrieve a mail, the CPU just has to busy wait until a mail arrives its read end of the pipe, and then read the value off of it.

In this case, bit 30 of the status register helps discern whether the mailbox is empty at a point of time.

#define MBOX_STATUS_MBOX_EMPTY (1 << 30)

uint32_t mbox_read(void) {
    // Wait until the read end receives a mail
    while (MBOX_STATUS(MBOX0_BASE) & MBOX_STATUS_MBOX_EMPTY)
        ;

    // Fetch the received mail from the read end
    uint32_t mail = MBOX_RW(MBOX0_BASE);

    return mail;
}

The property interface #

The property interface serves as Raspberry Pi Foundation’s unified solution for mailbox peripheral communication. While centred around the property tags channel, it offers extensive functionality, a result of the foundation’s efforts to create a cohesive mailbox interface. This interface employs two abstraction layers:

• Property tags
• Property messages

Property tags #

A property tag embodies an instruction to the GPU, and it encapsulates all the metadata associated with the instruction. The property interface supports three kinds of operations:

• Get
• Set
• Test

The wiki covers all the supported tags. In general, any property tag follows the memory layout shown below.

struct mbox_property_tag {
    uint32_t id;
    uint32_t buffer_size;
    volatile uint32_t status;
    volatile uint32_t buffer[];
};

volatile fields specify regions overwritten by the GPU.

The status field encodes values differently depending on whether the tag is a part of a request or response, as shown in the diagram below.

Note that the tags need 4-byte alignment, hence the uint32_t[] type for the tag buffer. Certain tags, such as the “get board MAC address tag” , contain byte-aligned buffers, with the mentioned example having a 6-byte buffer. To meet the ARM instruction set’s enforced 4-byte alignment condition , at least as per ARM v8-A, these byte-aligned tags require padding. Hence, the procedure responsible for populating tags within the message should address this.

In case of (unpacked) structs, the compiler automatically adds padding to ensure proper alignment.

Preparing a tag #

Defining a tag boils down to doing the following:

• Setting the id field.
• Setting the size field.
• Clearing bit 31 of the status field.
• Populating the buffer with required tag-specific arguments.

Property messages #

A property message consists of contiguously placed 32-bit words. It has a defined header including the associated metadata and a list of tags. The structure definition shown below describes the memory layout for a property message.

 struct mbox_property_message {
     uint32_t size;
     volatile uint32_t status;
     struct mbox_property_tag tags[];
 } __attribute__((aligned(16)));

The volatile fields specify regions overwritten by the GPU.

The __attribute__((aligned(16))) token hints the compiler to place the message structure at a 16-byte aligned memory address. The reason behind this is covered in the mail composition section below.

Preparing a message #

A property message preparation involves the following steps:

• Setting the size field.
• Setting the status to “process request” value.
• Populating the tag list .
• Marking end of the tag list with a null byte.

Composing a mail #

Composing a mail for the property tags channel entails the following:

• Specifying the channel as 8.
• Passing the upper 28-bits of the message pointer as data.

As an example, consider the C snippet shown below. It composes a mail for the property tags channel.

#define MBOX_PROPERTY_TAGS_ARM_TO_VC 8

// Prepared message
struct mbox_property_message message = {...};

// Extract the upper 28-bits of the message pointer
uint32_t data = (uintptr_t)&message >> 4;

// Compose the mail
uint32_t mail = mbox_mail_compose(MBOX_PROPERTY_TAGS_ARM_TO_VC, data);

Looking at this process, one question arises naturally. The GPU receives a 28-bit address to the message, so how does it resolve it to a valid physical address in the memory? The wiki has got us covered here.

Apparently, the GPU firmware expects a 16-byte aligned structure for the message. Such a structure, always has its least-significant nibble as 0x0. Leveraging this, the GPU can precisely reconstruct the 32-bit pointer to the message clearing out the channel bits in the request. The bitwise expression shown below summarizes this operation.

// Clear the channel bits and typecast to a message pointer
struct mbox_property_message* message = (void*)(mail & ~MBOX_MAIL_CHANNEL_MASK);

Parsing the response #

On receiving the mail, the GPU parses the message, and fills in the response by overwriting the message contents. The response mail sent by the GPU contains the same data and channel values which prove useful for validation on the CPU side.

As an example, the C snippet below performs basic validation checks on the response. This example intentionally omits tag-parsing as it highly depends on the used tags.

#define MBOX_PROPERTY_MESSAGE_STATUS_RESPONSE_ERROR 0x80000000

// Assumptions:
// `request` mail has been sent and `response` is the retrieved response
// `message` is the previously prepared message

if (response != request)
    ;  // Error: Received response for a different request

if (message.status == MBOX_PROPERTY_MESSAGE_STATUS_RESPONSE_ERROR)
    ;  // Error: Request was ill-formed

In case of ill-formed requests, investigating status fields of the tags can help isolate the erroneous tag.

Usage #

The preceding sections of this article shed light on various aspects of the mailbox peripheral. Now, we can use those bits of information to talk to the GPU to get what we want.

Interacting with the mailbox using the property interface comes down to:

1. Preparing the message
2. Composing the mail
3. Sending the request
4. Receiving the response
5. Parsing the response

An example #

To get a better grasp of the process, here is a step-by-step breakdown of the different transactions involved in obtaining the board’s MAC address using the “get board MAC address” tag.

Suppose that the property message structure get allocated at 0x10110030 in the memory. The mbox_compose_mail() call would pack this address with the channel as shown below.

For brevity, the table shown below shows message values for both, before and after the transaction. The “During Request” column contains the values filled during message preparation. The “After Request” column contains the message values for a successful transaction, read after mail dispatch, retrieval, and validation.

Note that the message places the null tag at 0x1c instead of 0x1a even when the buffer contents require 6 bytes. Doing this ensures the 4-byte alignment for the property tag buffer, as discussed in the mail composition section.

The RPi boards follow the little-endian notation. Hence, the MAC address gets stored in tags[0].buffer[1:0] as shown below:

Reading the buffer value from the response buffer yields the MAC address as dc:a6:32:32:6a:f2. A quick MAC address lookup indicates that the dc:a6:32 prefix belongs to “Raspberry Pi Trading Ltd”, thereby validating the response.

Final remarks #

This article took a considerable amount of effort due to the lack of documentation on the topic. I hope it helps you in your low-level adventures on the RPi.

Thank you for reading this article, and please feel free to share any suggestions or questions in the comments section.

References #

• https://github.com/raspberrypi/firmware/wiki/Mailboxes
• https://jsandler18.github.io/extra/prop-channel.html
• https://jsandler18.github.io/extra/mailbox.html
• https://fossies.org/linux/qemu/hw/misc/bcm2835_mbox.c
• https://www.cl.cam.ac.uk/projects/raspberrypi/tutorials/os/index.html
• https://github.com/BrianSidebotham/arm-tutorial-rpi/blob/master/part-5/readme.md