自动化数据结构恢复
编写脚本，自动识别并重构混淆二进制文件中的复杂数据结构。
自动化数据结构恢复 是 CoddyKit 上的免费 Reverse Engineering & Binary Analysis Basics 课时。 这是第 2 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Reverse Engineering & Binary Analysis Basics 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Reverse Engineering & Binary Analysis Basics 课程共包含 4 节课。
本课时的部分内容尚未翻译，以英文显示。
What are Data Structures?
In programming, a data structure is a way to organize and store data efficiently. Think of it like a neatly arranged filing cabinet for related information.
In reverse engineering, we often deal with compiled programs, which means the original source code is gone. Our goal is to "see" these hidden filing cabinets in the raw binary data.
The 'Why' of Data Recovery
Recovering data structures is crucial for understanding a program's logic. If you know how an object is laid out in memory, you can:
- Understand how different pieces of data relate.
- Identify important program variables.
- Pinpoint potential vulnerabilities more easily.
It turns a jumble of bytes into meaningful information!
Finding Hidden Structures
When a program is compiled, compilers often remove debugging information and optimize code. This makes it hard to automatically identify structures because:
- Original names are lost.
- Fields might be reordered or padded.
- Complex structures can be spread out.
It's like trying to rebuild a puzzle without the picture or edge pieces!
Simple Types in Binary
Before tackling complex structures, let's remember how basic data types look in raw bytes. A structure is just a collection of these simpler types.
For example, an integer might be 4 bytes, a character 1 byte. Their order and size matter!
import struct
# Simulate a small piece of binary data
binary_data = b'\x01\x00\x00\x00' + b'\x41' + b'\x02\x00\x00\x00'
print("Raw bytes:", binary_data)
# Interpret bytes 0-3 as a 32-bit integer (little-endian)
# '<I' means little-endian unsigned int
int_val = struct.unpack('<I', binary_data[0:4])[0]
print(f"Integer (offset 0): {int_val}")
# Interpret byte 4 as a character
char_val = chr(binary_data[4])
print(f"Character (offset 4): {char_val}")
# Interpret bytes 5-8 as another 32-bit integer
int_val2 = struct.unpack('<I', binary_data[5:9])[0]
print(f"Integer (offset 5): {int_val2}")What is a 'Struct'?
In languages like C, a struct is a user-defined data type that groups related variables into one single unit. Imagine a "User" struct that holds a user's ID (integer), name (string), and age (integer).
When compiled, this struct occupies a contiguous block of memory, with each field at a specific offset from the start of the block.
Spotting Structures Manually
When reverse engineering manually, you'd look for clues like:
- Repeated Access Patterns: Code that always reads/writes at
[reg + 0],[reg + 4],[reg + 8]. - Function Arguments: A large block of memory passed as a single argument to a function.
- Pointers: A field that points to another known structure or data type.
These patterns suggest a structured block of data.
Automating Pattern Search
Manually finding structures is tedious! This is where scripting shines. We can write scripts to automatically scan binary data for common patterns that might indicate a structure.
For example, a script could look for two integers followed by a null-terminated string, a very common pattern for simple objects.
def find_pattern(data_bytes: bytes, pattern_bytes: bytes):
"""Searches for a byte pattern within a larger byte string."""
indices = []
for i in range(len(data_bytes) - len(pattern_bytes) + 1):
if data_bytes[i:i+len(pattern_bytes)] == pattern_bytes:
indices.append(i)
return indices
# Simulate a binary's data section
simulated_binary_data = (
b'\xDE\xAD\xBE\xEF' + # random bytes
b'\x01\x00\x00\x00' + # int 1 (little-endian)
b'\x0A\x00\x00\x00' + # int 10
b'NAME\x00' + # string "NAME"
b'\x00\x00\x00\x00' + # padding
b'\x02\x00\x00\x00' + # int 2
b'\x0B\x00\x00\x00' + # int 11
b'ITEM\x00' # string "ITEM"
)
# Define a pattern to search for: int(1), int(10), string("NAME")
pattern_to_find = (
b'\x01\x00\x00\x00' +
b'\x0A\x00\x00\x00' +
b'NAME\x00'
)
found_at_offsets = find_pattern(simulated_binary_data, pattern_to_find)
if found_at_offsets:
print(f"Pattern found at offsets: {found_at_offsets}")
else:
print("Pattern not found.")XRefs for Structure Clues
In reverse engineering tools, a cross-reference (xref) shows you where a specific address or data is used in the code.
If many functions consistently access data starting at a particular address, and then at +0x4, +0x8, +0xC, these xrefs strongly suggest a data structure is being manipulated at that memory location.
Scripts can automate the analysis of these xrefs!
Scripting Structure Definitions
Once you've identified a potential data structure layout (e.g., through patterns or xrefs), your script can then define this structure within the reverse engineering tool itself.
This means telling the tool: "At this address, there's a structure named 'MyObject' with an integer 'ID' at offset 0, and a string 'Name' at offset 4."
This makes the disassembled code much more readable, replacing raw memory accesses with meaningful field names!
import struct
# Reusing simulated_binary_data from previous example
simulated_binary_data = (
b'\xDE\xAD\xBE\xEF' + # random bytes
b'\x01\x00\x00\x00' + # int 1 (little-endian)
b'\x0A\x00\x00\x00' + # int 10
b'NAME\x00' + # string "NAME"
b'\x00\x00\x00\x00' + # padding
b'\x02\x00\x00\x00' + # int 2
b'\x0B\x00\x00\x00' + # int 11
b'ITEM\x00' # string "ITEM"
)
class ItemStruct:
def __init__(self, data_bytes, start_offset):
# We assume the struct starts at start_offset in data_bytes
# Field 1: 4-byte integer (ID) at offset 0 from struct start
self.item_id = struct.unpack('<I', data_bytes[start_offset:start_offset+4])[0]
# Field 2: 4-byte integer (Quantity) at offset 4 from struct start
self.quantity = struct.unpack('<I', data_bytes[start_offset+4:start_offset+8])[0]
# Field 3: Null-terminated string (Name) at offset 8 from struct start
name_start = start_offset + 8
name_end = data_bytes.find(b'\x00', name_start)
if name_end == -1: # No null terminator, read until end
self.name = data_bytes[name_start:].decode('ascii', errors='ignore')
else:
self.name = data_bytes[name_start:name_end].decode('ascii', errors='ignore')
def __str__(self):
return (f"ItemStruct:\n"
f" ID: {self.item_id}\n"
f" Quantity: {self.quantity}\n"
f" Name: '{self.name}'")
# The "ITEM" pattern starts at offset 24 in simulated_binary_data
second_struct_offset = 24
found_item = ItemStruct(simulated_binary_data, second_struct_offset)
print(found_item)
# Let's also parse the first one to show it works
first_struct_offset = 4
found_name = ItemStruct(simulated_binary_data, first_struct_offset)
print("\n--- Another instance ---")
print(found_name)Other Structure Clues
Beyond simple patterns and xrefs, scripts can look for:
- Alignment: Data types often align to certain byte boundaries (e.g., 4-byte integers align to addresses divisible by 4).
- Vtables: In C++, objects often start with a pointer to a "virtual method table" (vtable), a strong indicator of an object.
- Common Function Args: If a library function expects a specific structure as input, scripts can identify calls to that function and infer the structure of its arguments.
Data Structure Challenge
Automating data structure recovery is about finding meaningful patterns in raw binary data.
Which of the following is NOT a primary reason why scripting is essential for identifying data structures in obfuscated binaries?
Recap: Automated Structure Recovery
We've learned that recovering data structures is vital for understanding compiled programs. Manual identification is hard due to lost source info and optimizations.
Scripting helps by:
- Scanning for byte patterns that indicate data types.
- Analyzing cross-references to memory locations.
- Programmatically defining structures within RE tools.
This transforms raw bytes into understandable program logic, making complex analysis much easier!
常见问题解答
「自动化数据结构恢复」课时是免费的吗？
是的 — 「自动化数据结构恢复」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Reverse Engineering & Binary Analysis Basics 课程的其余内容，请升级到 CoddyKit PRO。 Reverse Engineering & Binary Analysis Basics 课程共包含 4 节课。
「自动化数据结构恢复」这节课中我会学到什么？
编写脚本，自动识别并重构混淆二进制文件中的复杂数据结构。 你通过在浏览器中直接运行的动手代码来练习 Reverse Engineering & Binary Analysis Basics，全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 Reverse Engineering & Binary Analysis Basics 需要有经验吗？
无需任何先前经验。CoddyKit 上的 Reverse Engineering & Binary Analysis Basics 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 2 节课，共 4 节。
「自动化数据结构恢复」课时需要多长时间？
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。
我能在这节 Reverse Engineering & Binary Analysis Basics 课中编写并运行代码吗？
能。每节 Reverse Engineering & Binary Analysis Basics 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。