Automação da Recuperação de Estruturas de Dados
Desenvolva scripts para identificar e reconstruir automaticamente estruturas de dados complexas em binários ofuscados.
Automação da Recuperação de Estruturas de Dados é uma aula grátis de Reverse Engineering & Binary Analysis Basics no CoddyKit. Esta é a aula 2 de 4. Você pode ler a aula completa abaixo gratuitamente — depois pratica ao vivo no navegador com um editor de código integrado e um tutor de IA 24/7. Faz parte do caminho de aprendizado de Reverse Engineering & Binary Analysis Basics, e seu progresso é sincronizado entre a web e o app CoddyKit. O curso de Reverse Engineering & Binary Analysis Basics inclui 4 aulas no total.
Partes desta aula ainda não foram traduzidas e aparecem em inglês.
What are Data Structures?
In programming, a data structure is a way to organize and store data efficiently. Think of it like a neatly arranged filing cabinet for related information.
In reverse engineering, we often deal with compiled programs, which means the original source code is gone. Our goal is to "see" these hidden filing cabinets in the raw binary data.
The 'Why' of Data Recovery
Recovering data structures is crucial for understanding a program's logic. If you know how an object is laid out in memory, you can:
- Understand how different pieces of data relate.
- Identify important program variables.
- Pinpoint potential vulnerabilities more easily.
It turns a jumble of bytes into meaningful information!
Finding Hidden Structures
When a program is compiled, compilers often remove debugging information and optimize code. This makes it hard to automatically identify structures because:
- Original names are lost.
- Fields might be reordered or padded.
- Complex structures can be spread out.
It's like trying to rebuild a puzzle without the picture or edge pieces!
Simple Types in Binary
Before tackling complex structures, let's remember how basic data types look in raw bytes. A structure is just a collection of these simpler types.
For example, an integer might be 4 bytes, a character 1 byte. Their order and size matter!
import struct
# Simulate a small piece of binary data
binary_data = b'\x01\x00\x00\x00' + b'\x41' + b'\x02\x00\x00\x00'
print("Raw bytes:", binary_data)
# Interpret bytes 0-3 as a 32-bit integer (little-endian)
# '<I' means little-endian unsigned int
int_val = struct.unpack('<I', binary_data[0:4])[0]
print(f"Integer (offset 0): {int_val}")
# Interpret byte 4 as a character
char_val = chr(binary_data[4])
print(f"Character (offset 4): {char_val}")
# Interpret bytes 5-8 as another 32-bit integer
int_val2 = struct.unpack('<I', binary_data[5:9])[0]
print(f"Integer (offset 5): {int_val2}")What is a 'Struct'?
In languages like C, a struct is a user-defined data type that groups related variables into one single unit. Imagine a "User" struct that holds a user's ID (integer), name (string), and age (integer).
When compiled, this struct occupies a contiguous block of memory, with each field at a specific offset from the start of the block.
Spotting Structures Manually
When reverse engineering manually, you'd look for clues like:
- Repeated Access Patterns: Code that always reads/writes at
[reg + 0],[reg + 4],[reg + 8]. - Function Arguments: A large block of memory passed as a single argument to a function.
- Pointers: A field that points to another known structure or data type.
These patterns suggest a structured block of data.
Automating Pattern Search
Manually finding structures is tedious! This is where scripting shines. We can write scripts to automatically scan binary data for common patterns that might indicate a structure.
For example, a script could look for two integers followed by a null-terminated string, a very common pattern for simple objects.
def find_pattern(data_bytes: bytes, pattern_bytes: bytes):
"""Searches for a byte pattern within a larger byte string."""
indices = []
for i in range(len(data_bytes) - len(pattern_bytes) + 1):
if data_bytes[i:i+len(pattern_bytes)] == pattern_bytes:
indices.append(i)
return indices
# Simulate a binary's data section
simulated_binary_data = (
b'\xDE\xAD\xBE\xEF' + # random bytes
b'\x01\x00\x00\x00' + # int 1 (little-endian)
b'\x0A\x00\x00\x00' + # int 10
b'NAME\x00' + # string "NAME"
b'\x00\x00\x00\x00' + # padding
b'\x02\x00\x00\x00' + # int 2
b'\x0B\x00\x00\x00' + # int 11
b'ITEM\x00' # string "ITEM"
)
# Define a pattern to search for: int(1), int(10), string("NAME")
pattern_to_find = (
b'\x01\x00\x00\x00' +
b'\x0A\x00\x00\x00' +
b'NAME\x00'
)
found_at_offsets = find_pattern(simulated_binary_data, pattern_to_find)
if found_at_offsets:
print(f"Pattern found at offsets: {found_at_offsets}")
else:
print("Pattern not found.")XRefs for Structure Clues
In reverse engineering tools, a cross-reference (xref) shows you where a specific address or data is used in the code.
If many functions consistently access data starting at a particular address, and then at +0x4, +0x8, +0xC, these xrefs strongly suggest a data structure is being manipulated at that memory location.
Scripts can automate the analysis of these xrefs!
Scripting Structure Definitions
Once you've identified a potential data structure layout (e.g., through patterns or xrefs), your script can then define this structure within the reverse engineering tool itself.
This means telling the tool: "At this address, there's a structure named 'MyObject' with an integer 'ID' at offset 0, and a string 'Name' at offset 4."
This makes the disassembled code much more readable, replacing raw memory accesses with meaningful field names!
import struct
# Reusing simulated_binary_data from previous example
simulated_binary_data = (
b'\xDE\xAD\xBE\xEF' + # random bytes
b'\x01\x00\x00\x00' + # int 1 (little-endian)
b'\x0A\x00\x00\x00' + # int 10
b'NAME\x00' + # string "NAME"
b'\x00\x00\x00\x00' + # padding
b'\x02\x00\x00\x00' + # int 2
b'\x0B\x00\x00\x00' + # int 11
b'ITEM\x00' # string "ITEM"
)
class ItemStruct:
def __init__(self, data_bytes, start_offset):
# We assume the struct starts at start_offset in data_bytes
# Field 1: 4-byte integer (ID) at offset 0 from struct start
self.item_id = struct.unpack('<I', data_bytes[start_offset:start_offset+4])[0]
# Field 2: 4-byte integer (Quantity) at offset 4 from struct start
self.quantity = struct.unpack('<I', data_bytes[start_offset+4:start_offset+8])[0]
# Field 3: Null-terminated string (Name) at offset 8 from struct start
name_start = start_offset + 8
name_end = data_bytes.find(b'\x00', name_start)
if name_end == -1: # No null terminator, read until end
self.name = data_bytes[name_start:].decode('ascii', errors='ignore')
else:
self.name = data_bytes[name_start:name_end].decode('ascii', errors='ignore')
def __str__(self):
return (f"ItemStruct:\n"
f" ID: {self.item_id}\n"
f" Quantity: {self.quantity}\n"
f" Name: '{self.name}'")
# The "ITEM" pattern starts at offset 24 in simulated_binary_data
second_struct_offset = 24
found_item = ItemStruct(simulated_binary_data, second_struct_offset)
print(found_item)
# Let's also parse the first one to show it works
first_struct_offset = 4
found_name = ItemStruct(simulated_binary_data, first_struct_offset)
print("\n--- Another instance ---")
print(found_name)Other Structure Clues
Beyond simple patterns and xrefs, scripts can look for:
- Alignment: Data types often align to certain byte boundaries (e.g., 4-byte integers align to addresses divisible by 4).
- Vtables: In C++, objects often start with a pointer to a "virtual method table" (vtable), a strong indicator of an object.
- Common Function Args: If a library function expects a specific structure as input, scripts can identify calls to that function and infer the structure of its arguments.
Data Structure Challenge
Automating data structure recovery is about finding meaningful patterns in raw binary data.
Which of the following is NOT a primary reason why scripting is essential for identifying data structures in obfuscated binaries?
Recap: Automated Structure Recovery
We've learned that recovering data structures is vital for understanding compiled programs. Manual identification is hard due to lost source info and optimizations.
Scripting helps by:
- Scanning for byte patterns that indicate data types.
- Analyzing cross-references to memory locations.
- Programmatically defining structures within RE tools.
This transforms raw bytes into understandable program logic, making complex analysis much easier!
Perguntas Frequentes
A aula “Automação da Recuperação de Estruturas de Dados” é grátis?
Sim — o texto completo de “Automação da Recuperação de Estruturas de Dados” é grátis para ler aqui na web. Para praticá-la interativamente (um editor de código integrado e um tutor de IA 24/7) e desbloquear o restante do curso de Reverse Engineering & Binary Analysis Basics, atualize para CoddyKit PRO. O curso de Reverse Engineering & Binary Analysis Basics inclui 4 aulas no total.
O que vou aprender em “Automação da Recuperação de Estruturas de Dados”?
Desenvolva scripts para identificar e reconstruir automaticamente estruturas de dados complexas em binários ofuscados. Você pratica Reverse Engineering & Binary Analysis Basics com código prático que executa diretamente no navegador, e um tutor de IA 24/7 responde suas dúvidas enquanto trabalha na aula.
Preciso ter experiência prévia para começar Reverse Engineering & Binary Analysis Basics?
Nenhuma experiência prévia é necessária. Reverse Engineering & Binary Analysis Basics no CoddyKit é estruturado para alunos iniciantes até avançados, então você pode começar aqui ou desde o início e aprender no seu ritmo. Esta é a aula 2 de 4.
Quanto tempo leva a aula “Automação da Recuperação de Estruturas de Dados”?
A maioria das aulas CoddyKit leva cerca de 5–10 minutos. Cada uma é compacta e interativa, então você faz progresso constante e retoma exatamente de onde parou entre web e app.
Posso escrever e executar código nesta aula de Reverse Engineering & Binary Analysis Basics?
Sim. Cada aula de Reverse Engineering & Binary Analysis Basics inclui um editor de código integrado, então você escreve e executa código real direto no navegador e recebe feedback de IA instantaneamente — nenhuma configuração local necessária.
Todas as aulas deste curso
- Scripts com IDAPython e Ghidra
- Automação da Recuperação de Estruturas de Dados
- Técnicas de Aplicação de Patches em Binários
- Assinaturas FLIRT e Identificação de Funções de Bibliotecas