Writing Subroutines

As illustrated before, calling of subroutines heavily hinges on a number of conventions. Thanks to those conventions, however, knowing how to call one subroutine essentially means knowing how to call any subroutine. When writing subroutines yourself it is essential to respect those conventions yourself.

Theoretically, you could make up any conventions as long as you are the only caller of your subroutines. However, following the given conventions not only helps you stay consistent but also makes your subroutines generally usable. The assignments will furthermore test whether you are following the calling conventions through the automated tests.

Callee-Saved Registers

As mentioned before, some registers are callee-saved by convention and need to retain their value during a subroutine call. However, this does not mean that you may not use them at all.

You can push the values of all callee-saved registers that you want to use to the stack at the start of the subroutine. If you pop the same values to the corresponding registers before returning it will look like the registers never changed their value to the caller of your subroutine.

Stack Frame

The stack is an area in memory that is freely useable by your subroutine. However, as by the calling convention, the stack should remain seemingly untouched for the caller of your subroutine. The area of the stack that is used by your subroutine is typically called its stack frame.

Prologue

The rbp ("base pointer") register typically denotes the bottom (→ start) of the current stack frame and does not change during the main execution of the subroutine (while the stack pointer moves with every push or pop instruction). To set up the stack frame of a subroutine, a so-called prologue is commonly used:

foo:
    pushq   %rbp        # save previous base pointer
    movq    %rsp, %rbp  # set base pointer for new stack frame

As rbp is a callee-saved register, clearly its value at the entry to the subroutine needs to be saved somewhere such that it can be restored before returning. As explained on the page on Memory, the stack is the go-to location for temporary storage. Thereby, the previous base pointer is pushed onto the stack (line 2).

Now all that's left is to modify the base pointer (rbp) such that it correctly points to the "base" of the new stack frame. This is done by copying the value of the stack pointer (rsp) to the base pointer. As the stack pointer always points at the topmost element of the stack (so the element last pushed), after executing line 3 both the base pointer and the stack pointer point at the bottom of the current stack frame

Example: Stack Frame after Prologue

Let's consider a slightly longer snippet of the start of a subroutine:

foo:
    pushq   %rbp        # save previous base pointer
    movq    %rsp, %rbp  # set base pointer for new stack frame 
    
    pushq   $'A'        # push some values
    pushq   $'B'

After line 6, the stack will look like this:

(considering the call of the foo subroutine as given earlier - with 7 as the seventh argument)

You can see the seventh argument as the topmost element of the previous stack frame.
After that is the return address, so the address that should be loaded into the program counter when foo returns.
Next is the previous base pointer, so the value that was pushed in line 2. This is also the memory location that rbp points to, denoting the bottom of the current stack frame.
Lastly the two values (ASCII representations of A and B) pushed in lines 5 and 6 are on the stack, with rsp pointing at the one pushed last.

The nice side-effect of this sort of prologue is concerning the stack alignment. As mentioned before, the stack should be 16-byte aligned before calling a subroutine. Calling a subroutine, however, will push the 8-byte return address. So at the start of the called subroutine, the stack will be misaligned by 8 bytes. By pushing the previous base pointer, the alignment is again restored to 16 bytes and other subroutines may be called without further action to align the stack.

Epilogue

Before a subroutine can return, the stack frame that was built in the prologue needs to be properly removed, and the previous stack frame restored. That action is called the epilogue of the function (as opposed to the prologue):

    # ...
    movq    %rbp, %rsp  # reset stack pointer to bottom
    popq    %rbp        # restore previous base pointer
    ret

The previous base pointer is, considering a proper prologue, stored at the bottom of the current stack frame, so at the location pointed to by rbp. Therefore, the value of the base pointer is copied to the stack pointer - the top of the stack is now again the beginning of the stack frame and any pushed values are no longer part of the stack. Next, the topmost value of the stack is popped and saved as the base pointer - the caller's value of rbp is properly restored. Lastly, the ret instruction will pop the return address from the stack and move it into the program counter.

Example: Stack Frame during Epilogue

Let's consider the same foo subroutine as used in the earlier example, but with the added epilogue:

foo:
    pushq   %rbp        # save previous base pointer
    movq    %rsp, %rbp  # set base pointer for new stack frame 
    
    pushq   $'A'        # push some values
    pushq   $'B'
    movq    $0, %rax    # return value 0
    
    movq    %rbp, %rsp  # reset stack pointer to bottom
    popq    %rbp        # restore previous base pointer
    ret

Arguably, this subroutine has very little functionality (essentially none). However, it should be sufficient for this example.

The earlier example showed the stack after line 6, with this in mind, the following 2 figures show the stack after lines 9 and 10 respectively. Note that the dashed lines represent data that is still present in memory but no longer considered to be part of the stack (as the stack pointer has moved below).

Line 9 "resets" the stack pointer, so the two values are no longer part of the stack. You may note that two distinct popq instructions would have had the same effect.

Now that the stack pointer again points at the location where the previous base pointer is stored, the pop instruction from line 10 will restore the caller's base pointer value.

Furthermore, popping that value will decrement the stack pointer, such that it points at the return address. Thereby, line 11 will have the program correctly return to the caller's code.

Local Variables

When writing a subroutine, you will likely need some local variables. You could, of course, simply rely on registers for those variables. However, there are multiple drawbacks to that:

there is only a limited number of registers, some of which you may even need for temporary storage during computations
if your subroutine calls another subroutine, some of the registers may not keep their value

So a different place to store variables is needed. As you may have guessed the answer is the stack.

Reserving Space

To make space for local variables, you can push their initial values right after the prologue of your subroutine. If they are uninitialized, you can, even more simply, subtract the needed space from the stack pointer (remember: the stack grows towards the lower memory addresses, so if you want to reserve space for a quadword you need to subtract 8 from the stack pointer).

Accessing Local Variables

As you want to keep the variables on the stack during the execution of your subroutine, it is not a valid option to pop each variable from the stack when you need it. Rather, you want to access them somehow without modifying your stack. For that, you can use a mov instruction with the address of the needed variable as the first operand.

As you should know, there are various Addressing Modes in x86-64 Assembly. For accessing items from the stack, it is useful to refer to them as either an offset from the stack pointer or an offset from the base pointer. The stack pointer changes its value with every push or pop, making it mostly unusable for referring to local variables for longer subroutines. The base pointer, however, remains unchanged during the execution of a subroutine (except for during the pro- and epilogue). Thereby, local variables are commonly referred to as an offset from the base pointer.

Example: Accessing Local Variables

Consider the same foo example as used before:

foo:
    pushq   %rbp        # save previous base pointer
    movq    %rsp, %rbp  # set base pointer for new stack frame 
    
    pushq   $'A'        # var1 at -8(%rbp)
    pushq   $'B'        # var2 at -16(%rbp)
    subq    $8, %rsp    # (uninitialized) var3 at -24(%rbp)
    # ...

This Assembly program could represent a C function somewhat similar to the following:

int foo() {
    int64_t var1 = 'A';
    int64_t var2 = 'B';
    int64_t var3;
    // ...
}

As (partly) already shown earlier, the stack of the subroutine after line 7 would look as follows:

As the base pointer points to the bottom of the stack frame and each quadword is 8 bytes long, the addresses of var1, var2, and var3 can be referred to as:

-8(%rbp)    # var1
-16(%rbp)   # var2
-24(%rbp)   # var3

Note that the value of var3 is denoted with 0x? in the graphic. This isn't any kind of special value but simply a representation of an unknown value. As the stack pointer was moved without pushing a value, the contents of this memory location are unknown. If the stack was larger at some point prior to this function call, then some old value will still be present there.

PreviousCalling Subroutines NextSection Exercises

Last updated 3 months ago