Buffer Overflow dan Shellcode

Welcome to a critical topic in low-level security: Buffer Overflows. These are a type of software vulnerability that can allow attackers to gain control over a program.

Essentially, a buffer overflow occurs when a program tries to write more data into a fixed-size memory buffer than it was designed to hold. This excess data 'overflows' into adjacent memory regions.

In programming, especially in languages like C or assembly, a buffer is simply a block of memory reserved for storing data. Think of it like a container with a specific capacity.

• Buffers are often used for temporary storage, like holding user input or network data.
• They can be declared as arrays of characters (strings) or other data types.
• For example, char username[32]; declares a buffer that can hold up to 31 characters plus a null terminator.

Let's look at a simple C program with an intentional buffer overflow vulnerability. Pay close attention to the strcpy function.

strcpy copies a string from source to destination, but it does not check the destination buffer's size. This is a common source of overflows.

To understand how overflows can be exploited, we need to revisit the call stack. When a function is called, a new stack frame is created.

This stack frame typically contains:

• Local variables for the function.
• The saved Base Pointer (EBP/RBP).
• Most importantly: the Return Address, which tells the CPU where to resume execution after the function finishes.

When a buffer overflow occurs on the stack, the excess data doesn't just corrupt adjacent local variables. If enough data is supplied, it can overwrite the saved EBP and then the return address itself.

By changing the return address, an attacker can trick the program into jumping to an arbitrary memory location instead of returning to the legitimate caller. This is the core mechanism for many buffer overflow exploits.

So, where does the attacker want the program to jump? Usually, to a piece of code they control, known as shellcode.

• Shellcode is a small, self-contained piece of machine code.
• Its primary purpose is often to execute a 'shell' (a command prompt) on the target system.
• It can also perform other malicious actions, like creating new users, downloading files, or connecting to a remote server.

Shellcode is typically written in assembly language to be as compact and efficient as possible. It avoids null bytes (\x00) because many string functions stop copying at the first null byte.

A common goal is to call the execve system call (on Linux) to launch /bin/sh.

How does the shellcode get into the program? The attacker includes it as part of the malicious input that causes the buffer overflow.

When the buffer overflows, the shellcode is written into the program's memory, usually on the stack, alongside the overwritten return address.

Attackers often use a NOP sled (No Operation sled) to increase the reliability of their exploit.

• A NOP sled is a sequence of 'No Operation' (NOP) instructions (e.g., \x90 in x86).
• If the attacker overwrites the return address to point anywhere within the NOP sled, the CPU will simply execute NOPs until it 'slides' into the actual shellcode.
• This compensates for slight inaccuracies in guessing the exact memory address of the shellcode.

Operating systems and compilers have developed several defenses against buffer overflows:

• ASLR (Address Space Layout Randomization): Randomizes memory addresses to make guessing the return address or shellcode location harder.
• DEP/NX Bit (Data Execution Prevention/No-Execute): Marks memory regions (like the stack) as non-executable, preventing shellcode from running there.
• Stack Canaries: Compiler-generated random values placed on the stack; if overwritten, the program detects tampering and aborts.
• Safe Functions: Using functions like strncpy, snprintf, fgets, or C++ strings that perform bounds checking.

In this lesson, we explored the dangerous world of buffer overflows. We learned that they occur when too much data is written into a fixed-size buffer, corrupting adjacent memory.

Crucially, if this overflow reaches the return address on the stack, an attacker can hijack program control. They achieve this by injecting shellcode—small, malicious machine code—and redirecting execution to it, often aided by a NOP sled. We also touched upon essential mitigation techniques like ASLR, DEP, and stack canaries.