Memory and the Stack

When a program runs, the operating system gives it a memory layout with distinct regions: the text (code), data/BSS (globals), the heap (dynamic allocation), and the stack.

Buffer overflows most often abuse the stack, so understanding it is essential.

The stack is a region of memory used for function calls. It stores local variables, function arguments, and return addresses.

It grows downward on x86: pushing data moves the stack pointer toward lower addresses. This direction matters when an overflow happens.

Each function call creates a stack frame. A frame typically contains local variables, the saved base pointer (EBP), and the return address (saved EIP).

The return address tells the CPU where to continue once the function finishes. Controlling it is the heart of stack overflow exploitation.

On 32-bit x86, the important registers are:

• ESP: stack pointer, top of the stack.
• EBP: base pointer, anchor for the current frame.
• EIP: instruction pointer, address of the next instruction.

If you control EIP, you control execution flow.

A buffer is a fixed-size block of memory, such as a local character array. The program expects data to fit within it.

If the program copies more data than the buffer can hold without checking the length, the excess spills into adjacent memory.

A buffer overflow occurs when input exceeds the buffer's size and overwrites neighboring data on the stack.

Because the saved EBP and return address sit just past local buffers, a large enough input can overwrite the return address, hijacking control flow.

Overflows usually stem from unsafe C functions that do not check length:

• strcpy, strcat, sprintf, gets.

Safer alternatives like strncpy or snprintf take a size limit, but legacy code is full of the dangerous ones.

When the vulnerable function returns, the CPU pops the (now attacker-controlled) return address into EIP and jumps there.

If you point EIP at your own injected machine code (shellcode), the CPU executes it with the privileges of the program.

x86 is little-endian: multi-byte values are stored least-significant byte first. To overwrite the return address with 0x080414C3, you write the bytes in reverse order.

Getting endianness wrong is a common beginner mistake that makes exploits silently fail.

Modern systems add defenses that complicate exploitation:

• DEP/NX: marks the stack non-executable.
• ASLR: randomizes memory addresses.
• Stack canaries: detect overwrites before return.

This course focuses on the classic case (protections off) to teach fundamentals.

Although modern defenses make classic stack overflows harder, the fundamentals remain essential. Embedded systems, IoT, and legacy software often lack these protections entirely.

Mastering the basic case is also the prerequisite for advanced techniques like ROP that bypass modern mitigations.

Recall which value the attacker ultimately wants to control.

You now understand the foundations:

• The stack stores locals, saved EBP, and the return address, and grows downward.
• ESP, EBP, EIP are the key x86 registers; controlling EIP controls execution.
• Unsafe functions like strcpy and gets cause overflows.
• x86 is little-endian; modern defenses are DEP, ASLR, and canaries.

Next you will fuzz a program to find the crash offset.