Shellcode and Exploitation

Shellcode is a small piece of machine code that performs an action when executed, classically spawning a shell. It is the payload you redirect execution to after controlling EIP.

It is written in raw bytes (opcodes) so it can be injected directly into memory.

Common payload goals include:

• Bind shell: opens a listening port on the victim.
• Reverse shell: connects back to the attacker (better through firewalls/NAT).
• Execute command: runs a single command like adding a user.

Reverse shells are most common in real engagements.

msfvenom generates payloads. You specify the payload type, the listener address and port, the output format, and any bad characters to avoid.

The -b flag excludes bad chars; -f sets the output format.

Raw shellcode often contains bad characters (like null bytes). An encoder transforms the payload to avoid those bytes, then prepends a decoder stub that restores the original code at runtime.

The classic encoder is x86/shikata_ga_nai.

Encoded shellcode decodes itself in memory before running. This self-modification needs a little headroom on the stack, which is one reason a NOP sled and adequate buffer space help.

Sometimes you must adjust ESP first so the decoder does not corrupt its own code.

The complete exploit buffer is:

• Padding to the offset.
• The JMP ESP return address.
• A NOP sled.
• The encoded shellcode.

Send this to the vulnerable input and execution flows into your payload.

For a reverse shell you must listen on your machine before triggering the exploit. A simple Netcat listener catches the incoming connection.

For staged payloads use Metasploit's multi/handler instead.

With the listener ready, send the crafted buffer to the target. If everything aligns, the shellcode runs and your listener receives a shell on the victim.

You now have command execution with the privileges of the vulnerable process.

If it fails, check common culprits:

• A bad character slipped into the shellcode or return address.
• The NOP sled is too short.
• The decoder lacked stack room (adjust ESP).
• Wrong offset or endianness.

Methodical debugging in the debugger isolates the issue.

With DEP and ASLR enabled, plain shellcode on the stack will not execute. Modern exploitation uses Return-Oriented Programming (ROP) to chain existing code gadgets, and info leaks to defeat ASLR.

This course taught the foundation those advanced techniques build on.

Payloads come in two flavors. A stageless payload contains the full shellcode in one blob. A staged payload sends a tiny stub that downloads the rest from your handler.

Staged payloads fit in smaller buffers but need a matching handler like Metasploit's multi/handler to deliver the second stage.

Recall why shellcode is often encoded.

You completed the exploitation chain:

• Shellcode is the payload; reverse shells are most practical.
• Generate and encode it with msfvenom to dodge bad characters.
• Assemble padding + JMP ESP + NOP sled + shellcode, start a listener, and fire.
• Modern targets need ROP and info leaks to beat DEP/ASLR.

You have completed the Buffer Overflow Exploitation course.