Rootkits, Spyware, and Keyloggers

While many malware types announce their presence through visible damage, a class of malicious software is specifically designed to remain hidden from users, administrators, and security tools while silently accomplishing attacker goals. Rootkits, spyware, and keyloggers represent the stealth end of the malware spectrum. Their effectiveness depends on persistence without detection — they may remain active for months or years, quietly exfiltrating credentials and data. Understanding their hiding techniques is essential for forensic investigation and for selecting detection tools that can find what standard scans miss.

A rootkit modifies the operating system or low-level software to hide the attacker's presence. The term comes from Unix 'root' (highest privilege) + 'kit' (set of tools). Rootkits intercept OS functions to hide files, registry keys, processes, and network connections from standard system inspection tools. A process running a rootkit might not appear in Task Manager, its network connections won't show in netstat, and its files won't appear in directory listings — because all these tools query the OS through the same APIs the rootkit has compromised. Rootkits are extremely difficult to detect from within the infected OS.

Rootkits are classified by where they operate in the system. User-mode (application-level) rootkits run in user space and hook API calls; they are the easiest to detect and remove. Kernel-mode rootkits operate with OS-level privileges, modifying kernel data structures — they are highly stealthy and require a driver or kernel exploit to install. Bootkit (MBR rootkit) infects the Master Boot Record or Volume Boot Record, loading before the OS — invisible to the OS during operation. Hypervisor rootkits (VM-based) sit below the OS, running the original OS as a virtual machine — theoretically undetectable from within the guest.

Because rootkits compromise the OS's own reporting mechanisms, detection requires looking from outside the infected OS. Techniques include: cross-view analysis (comparing kernel data structures directly in memory against what APIs report — discrepancies reveal hidden items), integrity verification (comparing hashes of critical system files against known-good baselines from a trusted source), offline scanning (booting from a clean USB and scanning the drive), and Secure Boot with TPM attestation (cryptographically verifying the boot chain has not been modified). Tools include GMER, Malwarebytes Anti-Rootkit, and chkrootkit/rkhunter for Linux.

Spyware secretly monitors user activity and transmits information to the attacker without the user's knowledge or consent. It may capture browsing history, search queries, screenshots, application usage, and location data. Some spyware is commercially marketed as 'monitoring software' for parental control or employee monitoring (though legality varies by jurisdiction). Malicious spyware is used by criminals and nation-states to silently exfiltrate sensitive business data, personal information, and intellectual property. Spyware often bundles with free software (PUPs — Potentially Unwanted Programs), making the source hard to identify.

Stalkerware is spyware designed to covertly monitor intimate partners, marketed to abusers as partner surveillance tools. It captures location, calls, texts, and social media. Commercial spyware (like NSO Group's Pegasus, Candiru, and FinFisher) is sold to governments and law enforcement agencies — often targeting journalists, activists, and dissidents. Pegasus exploited zero-click iPhone vulnerabilities to install with no user interaction. The Coalition Against Stalkerware and security researchers have worked to add stalkerware detection to mainstream AV products. Detection often requires forensic analysis of the device with tools like Amnesty International's Mobile Verification Toolkit (MVT).

A keylogger records every keystroke the user types, capturing passwords, messages, credit card numbers, and any other typed data. Software keyloggers operate at different levels: user-mode hooks (SetWindowsHookEx API), kernel-mode drivers (more stealthy), or form-grabbing hooks that intercept browser form submissions before encryption. Hardware keyloggers are physical devices plugged between the keyboard and computer — completely invisible to software-based detection. They store keystrokes in internal memory for later retrieval. Organizations protecting high-security areas should inspect hardware ports regularly for unauthorized devices.

Form grabbers intercept data submitted in browser forms before it is encrypted and sent over the network — capturing usernames, passwords, and credit card numbers at the point of entry. Zeus (Zbot), a notorious banking Trojan, used form-grabbing hooks injected into browser processes. Unlike network-level sniffers that are blocked by HTTPS, form grabbers operate at the application level, making encryption irrelevant. They typically hook browser functions like InternetReadFile or inject into the browser process memory. Modern EDR tools detect process injection and browser API hooking patterns indicative of form-grabber activity.

Adware displays unwanted advertisements and may redirect browser traffic to generate revenue for its operators. While less destructive than other malware, adware degrades system performance, compromises privacy, and creates security risks by redirecting to malicious sites or bundling with more dangerous software. PUPs (Potentially Unwanted Programs) are software that users may not have intentionally installed, often bundled with free software through deceptive installation dialogs. PUPs occupy a gray area — they may not be strictly malicious, but they consume resources, compromise privacy, and create a pathway for more dangerous infections.

Once malware is installed with sufficient privileges, credential dumping extracts stored credentials from memory or disk for lateral movement. The Windows LSASS process stores credentials in memory for single sign-on. Tools like Mimikatz dump plaintext passwords or NTLM hashes from LSASS memory. Defenders counter this with Credential Guard (Windows 10+, uses Virtualization-Based Security to isolate LSASS in a secure container), Protected Users group membership (prevents credential caching), and EDR rules that alert on LSASS memory access from non-system processes.

Attempting to remove a rootkit from a running system is risky — the rootkit actively hides itself and may detect removal attempts. Best practice for rootkit remediation is to reimage the system from a trusted baseline after forensic evidence preservation. If full reimaging is not immediately possible: boot from a trusted external medium (USB with offline scanner), run rootkit detection tools before the OS loads, document all artifacts, and plan for reimaging at the earliest opportunity. For bootkits, use Secure Boot with UEFI to prevent unsigned bootloaders, and consider writing zeros to the MBR before OS reinstallation.

Test your understanding of CompTIA Security+ (SY0-701) concepts from this lesson.

In this lesson you learned: rootkits subvert OS APIs to hide processes, files, and network connections, making detection from within the infected OS unreliable — requiring offline scanning or cross-view analysis, spyware and keyloggers silently exfiltrate credentials and data with behavioral indicators detectable through network monitoring and EDR process inspection, and credential dumping tools like Mimikatz targeting LSASS are mitigated by Windows Credential Guard and Protected Users group policies. Next up we explore fileless malware and living-off-the-land attacks.