Assembly with Linux and C

Recently I've been learning x86 assembly using the wonderful tutorials by a guy who goes by Creel. He is using Windows and Visual Studio in the tutorials. Here are some notes on getting an environment setup on Linux using GNU Assembler (gas). It shouldn't be too difficult to use NASM or some other assembler instead.

The code below is all based on the shift, rotate, and bit manipluation tutorial.

A Simple Build Script

I prefer using GNU make for this, as that is what I will eventually use. But you could use a simple bash alias at this level.

all:
    $(CC) -o app -no-pie -Wall main.c bits.S

When you type make the all recipe will run and build an app executable. The -Wall flag turns on all warnings in GCC. The -no-pie option requires a bit of explanation. Many Linux distros today turn on Position Independent Executable (PIE) by default. This is a security feature that allows the Linux kernel to randomize the memory location that your segments are loaded. In order to do assembly with this setting you need to use a special addressing mode called RIP-relative addressing. That's quite a bit to worry about if you're just wanting to test out a few things, so we are disabling that here. You can read all the gory details here.

Some C Code

Below is the main.c file. Do not take this as a good example of C code. It's been awhile since I've done any C programming. It's mostly translated from the carry flag tutorial Creel has in his playlist.

Note the PrintBits call with either 32 or 64 for the bitCount parameter. If you're using EAX, EBX, etc. then you want to set this to 32. If using RAX, RBX, etc. then set to 64. Or set to 16 or 8. Whatever registers you are using in assembly and want to dump out, match it to the correct bit size.

#include <stdio.h>

extern int ShiftTest(unsigned long long int *p);

void PrintBits(int carry, unsigned long long int p, int bitCount) {
    printf("C: %d ", carry);

    for (int j = bitCount - 1; j >= 0; j--) {
        printf("%lld", (p>>j) & 1);
    }

    printf("\n");
}

int main() {
    int carry;
    char s[10];

    unsigned long long int v =
        0b00001111000011110000111100001111;
    //    0  4   8   12  16  20  24  28  32

    while (1) {
        carry = ShiftTest(&v);
        // PrintBits(carry, v, 32);   // Testing 32-bit registers
        PrintBits(carry, v, 64);      // Testing 64-bit registers

        // wait for newline/enter to continue
        fgets(s, sizeof(s), stdin);
    }
}

The Assembly Code

Finally we have the bits.S file. Note that it's important to use an uppercase .S extension if you plan to use a preprocessor (i.e. using #define or #include). If you use a lowercase .s and try to use the preprocessor commands you will be in for a frustrating time trying to figure out why it's not working.

.globl ShiftTest

.type ShiftTest, @function

.section .data
lastFlags: .quad 0

.section .text
ShiftTest:
    // Copy the unsigned long long ptr to %rcx
    movq (%rdi), %rcx

    // Restore flags from the previous call. This is necessary for rcl/lcl and
    // other operations that depend on the prior eflags values.
    call RestoreFlags

    //
    //   The operation to test
    //
    // 64-bit rotate left
    // rol $1, %rcx

    // 32-bit rotate left
    // rol $1, %ecx

    // 64-bit rotate through carry
    rcl $1, %rcx

    //
    //  Returning the carry flag + result
    //
    setc %al          // put carry flag in %al
    movzx %al, %rax   // move %al to %rax and zero-extend the remaining bits

    // Update the input pointer
    movq %rcx, (%rdi)

    // Save flags so that we can return back to what we were doing with previous
    // flags, if necessary (e.g. rcl operation)
    call SaveFlags

    ret

RestoreFlags:
    pushq (lastFlags)
    popfq
    ret

SaveFlags:
    pushfq
    popq (lastFlags)
    ret

Testing It Out

Type make and you should see GCC build the app executable. Running app should result in:

[assembly_demo]$ ./app
C: 0 0000000000000000000000000000000000011110000111100001111000011110

Press ENTER a few times and you will see the bits shift left and eventually the carry flag (C: digit) will get set.