Replay Attacks and Nonce Reuse Vulnerabilities

A nonce (number used once) ensures that the same plaintext message encrypted twice produces different ciphertexts. This is essential for semantic security: an attacker observing multiple ciphertexts gains no information about whether they encrypt the same or different plaintexts. In stream ciphers and CTR-mode block ciphers, the nonce seeds the keystream generation, making the nonce critically important for security.

In AES-GCM, the 96-bit nonce is used to initialize a counter for CTR mode encryption and also contributes to the GHASH authentication computation. The counter value for each AES block is derived by incrementing from the nonce. If two encryptions use the same nonce and key, they generate the same keystream, making the XOR of their ciphertexts equal to the XOR of their plaintexts.

Two GCM ciphertexts encrypted with the same (key, nonce) pair: their XOR equals the XOR of their plaintexts. If an attacker knows one plaintext (common in chosen-plaintext scenarios), they immediately recover the other. Even without a known plaintext, XOR of ciphertexts reveals language patterns and can be attacked with crib-dragging techniques. This is one of cryptography's most catastrophic misuse patterns.

GCM nonce reuse has an additional catastrophic consequence beyond plaintext recovery. The GHASH authentication key H is derived solely from the encryption key. If two messages share a (key, nonce) pair, the attacker can solve polynomial equations to recover H. With H known, the attacker can forge valid GCM authentication tags for arbitrary ciphertexts, completely breaking message integrity.

TLS 1.3 learned from nonce reuse risks in TLS 1.2 implementations. TLS 1.3 generates GCM nonces by XORing the 12-byte write IV (derived from the handshake) with the 8-byte sequence number (zero-extended to 12 bytes). Since sequence numbers increment for every record, each record uses a unique nonce. This eliminates random nonce generation risk while ensuring nonce uniqueness through the sequence number mechanism.

A replay attack captures a valid message or authentication token and retransmits it later to achieve a fraudulent effect. Examples: replaying a banking transaction to double-charge, replaying a valid authentication response to impersonate a user, or replaying a signed command to a device. The attacker does not need to understand or modify the message; simply retransmitting the captured data achieves the attack.

Including a timestamp in authenticated messages provides replay protection. The receiver accepts messages only within a short time window (e.g., 5 minutes of the claimed timestamp). A replayed message from hours ago falls outside the window and is rejected. The timestamp must be authenticated (inside the HMAC or AEAD additional data) to prevent an attacker from modifying it to appear fresh.

Sequence numbers provide stronger replay protection than timestamps by detecting not just old replays but also duplicate and out-of-order deliveries. The receiver maintains the expected next sequence number and a sliding window of recently seen numbers. Each message must have a sequence number within the acceptable window and not previously seen. TLS uses sequence numbers to prevent record replay within a session.

For AES-GCM, the recommended nonce generation strategy for random nonces is 96 bits from a cryptographically secure pseudorandom number generator (CSPRNG). With 96-bit random nonces, the birthday bound collision probability reaches 2^-32 after 2^32 encryptions (about 4 billion). For high-volume applications, switch to a counter-based nonce or use a different AEAD (like XChaCha20-Poly1305 with its 192-bit nonce) to extend the safe limit.

Deterministic counter-based nonces avoid birthday bound issues but require careful state management. The counter must be persisted across application restarts; a counter that resets to zero on restart immediately creates nonce reuse with the same key. The counter must be updated atomically before the nonce is used (not after). For distributed systems, counter-based nonces require coordination to prevent two nodes from using the same counter value.

XChaCha20-Poly1305 extends ChaCha20-Poly1305's nonce from 96 to 192 bits. With 192-bit random nonces, birthday-bound collision probability is negligible even after 2^96 encryptions (an astronomically large number of messages). This makes random nonce generation safe without practical birthday concerns. libsodium's crypto_secretbox (XSalsa20-Poly1305) uses 192-bit nonces for the same reason.

If two GCM ciphertexts are produced using the same key and nonce, what can an attacker recover?

Nonce reuse recap: nonces ensure ciphertext uniqueness, GCM nonce reuse reveals plaintext XOR and breaks authentication via H recovery, TLS 1.3 uses sequence number XOR IV to guarantee nonce uniqueness, replay attacks retransmit valid messages without modification, timestamps and sequence numbers in authenticated messages prevent replay, use 96-bit random nonces for GCM (watching birthday bounds) or XChaCha20 for larger random nonce space.