Basic Reverse Engineering Techniques

Reverse engineering (RE) is the process of analyzing software to understand its inner workings without having access to its original source code. Think of it as being a detective for programs!

It involves taking a compiled program (a binary) and working backward to figure out what it does, how it does it, and sometimes, why.

To reverse engineer, you'll primarily use two types of tools:

• Disassemblers: These tools convert machine code (the raw bytes of a program) back into human-readable assembly language. Popular examples include objdump, IDA Pro, and Ghidra. They are your 'eyes' into the program's instructions.
• Debuggers: Tools like GDB (GNU Debugger) allow you to run a program step-by-step, pause its execution, and inspect the contents of registers and memory at any point. They are your 'hands' for interacting with the live program.

For this lesson, we'll analyze a simple x86 assembly program. Imagine you only have its compiled version and need to figure out its logic!

This program simulates a basic 'password check' by comparing two hardcoded values and printing a message based on the result.

First, we'd compile our assembly program into an executable. On Linux, this typically involves an assembler (like NASM) and a linker (like LD).

nasm -f elf32 program.asm -o program.o
ld -m elf_i386 program.o -o program

Then, we use a disassembler like objdump to see the machine code converted back into assembly:

objdump -d program

Here's a snippet of what you might see:

08048060 <_start>:
8048060: b8 d2 04 00 00 mov $0x4d2,%eax
8048065: bb 36 16 00 00 mov $0x1636,%ebx
804806a: 39 d8 cmp %ebx,%eax
804806c: 74 1c je 804808a <.access_granted>

When reverse engineering, one of the first things you look for is the program's entry point. This is where execution begins.

For Linux executables compiled from assembly, the entry point is often labeled _start. In our disassembled output, you can see the <_start> label at address 08048060.

This tells you exactly where the CPU starts executing instructions when the program is loaded.

To understand a program's logic, you need to trace its flow of execution. Conditional jump instructions are key to understanding decision-making (like if/else statements).

In our example, after comparing eax and ebx with cmp %ebx,%eax, we see je 804808a <.access_granted>.

• cmp: Compares two values and sets CPU flags.
• je (Jump if Equal): If the comparison result was equal, execution jumps to the address 0804808a (our .access_granted block).
• If not equal, execution continues to the next instruction in sequence (the .access_denied block).

Programs interact with the operating system through system calls. On Linux x86 (32-bit), these are typically invoked using the int 0x80 instruction.

Before int 0x80, specific registers are loaded with values:

• eax: Contains the system call number (e.g., 4 for sys_write, 1 for sys_exit).
• ebx, ecx, edx: Hold arguments for the system call (e.g., file descriptor, buffer address, length for sys_write).

By observing these patterns, you can identify actions like writing to the console or exiting the program.

Messages and other static data are stored in data sections of the binary. You can often view these using objdump -s -j .data program or objdump -s -j .rodata program.

In the assembly, you'll see instructions that load the address of these strings into a register (e.g., mov ecx, 0x8049080 where 0x8049080 points to a string).

For our example, the messages "Access granted!" and "Access denied." would be found in the .data section, and their addresses are passed to sys_write.

By combining all these observations, we can reconstruct the program's original logic:

• It starts at _start.
• It loads two specific integer values into eax and ebx.
• It compares these two values.
• If they are equal, it jumps to a section that prints "Access granted!" to the console.
• If they are not equal, it falls through to a section that prints "Access denied." to the console.
• After printing, the program exits gracefully.

This is the essence of reverse engineering: understanding the program's intent and behavior from its compiled form.

Consider the following disassembled x86 snippet. Assume 0x402000 holds the string "Yes\n" and 0x402008 holds "No\n".

0x401000: mov eax, 0x5
0x401005: mov ebx, 0x5
0x40100a: cmp eax, ebx
0x40100c: jne 0x401018
0x40100e: mov edi, 0x402000 ; "Yes\n"
0x401013: call 0x401040 <puts@plt>
0x401018: mov edi, 0x402008 ; "No\n"
0x40101d: call 0x401040 <puts@plt>

In this lesson, you've learned the fundamental techniques of basic reverse engineering:

• Understanding what RE is and its importance.
• Identifying key tools like disassemblers (objdump) and debuggers (GDB).
• Locating the program's entry point (_start).
• Tracing program flow using conditional jumps (cmp, je).
• Recognizing system calls (int 0x80) and their parameters.
• Extracting meaningful strings and data from the binary.

By applying these techniques, you can begin to reconstruct the logic and behavior of programs even without their original source code!