Generating Tests by Prompt

When AI writes most of your code, tests become the contract that keeps it honest. Each assertion is a promise the assistant cannot silently break.

The goal is not 100 percent coverage for its own sake. It is to pin the behaviors you care about so that future generations cannot regress them.

This lesson is about prompting your way to a test suite worth trusting.

Weak prompts ask the model to "write tests for this function." The result mirrors the implementation and passes trivially, even when the code is wrong.

Strong prompts describe the expected behavior in your own words. The test then encodes intent, not the model's reading of the existing code.

Specify inputs, outputs, and invariants the way a product spec would.

Tests that only confirm the happy path give false comfort. Ask for the cases most likely to break first: nulls, empties, limits, and malformed input.

A suite that exercises failure modes is the one that catches real regressions. The boring success case rarely surprises anyone.

Front-load the edges and you front-load the bugs.

If you ask a model to write tests against existing code, it may assert the current buggy behavior as if it were correct. The suite goes green while the bug stays.

This is the single biggest trap in AI test generation. The test documents the defect instead of catching it.

Always review whether each assertion reflects what you want, not merely what the code currently does.

Readable tests follow a clear shape: set up state, perform one action, assert one outcome. Models drift into multi-assertion mega-tests that are hard to debug when they fail.

Ask for focused tests with descriptive names that read like specifications.

A failing test should tell you exactly which behavior broke, in its name alone.

AI often mocks too much, stubbing out the very logic under test until the test asserts nothing real. Or it mocks too little and hits a live database.

Mock at the system boundary: network, clock, filesystem, payment gateway. Keep your own business logic real.

State the boundary explicitly so the model does not guess.

Example tests check specific inputs; property tests check invariants across many generated inputs. "Reversing a list twice returns the original" holds for every list.

Asking the model for property-based tests surfaces bugs that hand-picked examples miss, because the framework hunts for counterexamples.

Use them where invariants are clear: encoders, sorters, serializers, math.

Coverage percentage is a map, not the territory. You can hit every line while asserting nothing meaningful. Models love padding suites to chase a number.

Use coverage to find untested branches, then judge each by hand: is this path worth pinning?

Aim for tests that would fail if the behavior were wrong, not tests that merely execute the code.

A green suite that survives any code change is worthless. Mutation testing introduces small bugs and checks whether your tests notice.

If a mutant survives, your tests do not actually constrain that behavior. The model can propose mutations and you verify your suite kills them.

This is how you measure test quality, not just quantity.

Flaky tests erode trust until people ignore failures entirely. The usual culprits are real time, random seeds, network calls, and test ordering.

Demand deterministic tests: inject the clock, fix the seed, isolate state between runs. A suite you cannot trust is worse than no suite.

Reliability is a feature of the tests themselves.

A good suite doubles as documentation. New contributors, human or AI, learn the system's rules by reading the assertions.

Keep test names and structure clear enough that the suite explains intended behavior on its own. The model reads them too, and writes better code as a result.

Your tests become the durable memory the model lacks.

Test your grasp of prompt-driven test generation.

Prompt tests from behavior and requirements, not from the implementation, so they catch bugs instead of pinning them. Front-load failure and boundary cases, mock only at boundaries, and use property and mutation testing to measure real strength.

Keep the suite deterministic and readable so it serves as a living spec. Next, you will hunt for security gaps the model left behind.