When we compile our program, the compiler turns our code to binary executables, that can be executed directly by CPU.
Have you ever wondered how these binary executables actually structured? Would you love to be able to understand them and even modify them directly?
Well, after this series, you’ll be able to understand every single byte of a simple Hello World executable. The binary executables would no longer be a mystery to you, you’ll be able to open them up and read them just like plain text.
First of all, when people say binary executables can run directly on CPU, it is not the whole truth. Obviously, to run an executable, it must be loaded into memory first. And there is some preparation work needs to be done as well.
This loading and preparing procedure is done by the Operating System, via a program called loader. As you’d expected, each OS has its own version of loader. When the loader completed its task, the OS would then hands over the control of CPU to the program. And only since then, the binary executables is running directly on CPU.
So, only having binary machine code in the binary executables is not good enough, it also needs to contain meta data describing different kinds of info that loader needs to know.
PE, ELF and Macho-O
Since each OS has its own loader, so does the standard of structuring meta data and machine code. We call these standards executable file formats, and all executables must conform to the formats of the targeting OS in order to run on it. This explains why you can not run a Windows .exe on Linux, or a Linux binary on Windows either, because they’re of different executable file formats.
Today, the most popular executable file formats are PE for Windows, ELF for Linux and some other Unix-like OS, and Mach-O for macOS and iOS. To keep this series simple, we’ll be focusing on Mach-O format, while you can easily use the knowledge learned here to understand the other 2 formats by referring to their specifications. Because under the hood, they’re all binary machine code + meta data.
Assembly and Binary Machine Code
We know the actual code executed on CPU is binary code consists solely of
0. For example, for x86-64 instruction set,
0101 0101 means pushing the content in the base pointer register to the stack, and
0100 1000 1000 1001 1110 0101 means moving the content in the stack pointer register to the base pointer register.
Despite how cool it is to code in
1, it’s very hard to write code in this format. You can easily introduce errors to your code while spotting them could be super difficult. That’s why we normally use Assembly instead.
Assembly is an exact one on one mapping of machine code. For example, instead of
0101 0101, we write
pushq %rbp; and instead of
0100 1000 1000 1001 1110 0101, we write
movq %rsp, %rbp. We then use a tool called Assembler to convert the text to those cool
In fact, when you’re inspecting the binary machine code of an executable or a library, there are many tools that can display the content in Assembly directly. On macOS, one of the tools you can use is
otool. Run command
otool -tv your_binary_file would show the machine code inside the binary in Assembly to you.
The following is the binary machine code of a simple Hello World program, represented in Assembly:
➜ otool -tv hello_world hello_world: (__TEXT,__text) section _main: 0000000100000f50 pushq %rbp 0000000100000f51 movq %rsp, %rbp 0000000100000f54 subq $0x20, %rsp 0000000100000f58 leaq 0x47(%rip), %rax 0000000100000f5f movl $0x0, -0x4(%rbp) 0000000100000f66 movl %edi, -0x8(%rbp) 0000000100000f69 movq %rsi, -0x10(%rbp) 0000000100000f6d movq %rax, %rdi 0000000100000f70 movb $0x0, %al 0000000100000f72 callq 0x100000f84 0000000100000f77 xorl %ecx, %ecx 0000000100000f79 movl %eax, -0x14(%rbp) 0000000100000f7c movl %ecx, %eax 0000000100000f7e addq $0x20, %rsp 0000000100000f82 popq %rbp 0000000100000f83 retq
Don’t worry if you don’t understand it yet. We’ll go through all the details in the following series.