Building a Reliable Form-Filling Agent
A practical case study: log into a portal, fill a multi-page form, and verify the submission.
Building a Reliable Form-Filling Agent is a free AI Agents lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
A Realistic Project
A "fill this multi-page form" agent. We'll combine Playwright + LLM planning + retry + vision fallback.
Architecture
- LLM reads the field requirements from a JSON spec
- Playwright opens the form
- For each field, find selector + value, fill, validate
- If a field is hard to locate via DOM, fall back to vision
- Submit the form, verify success
Step 1: Define the Task
TASK = {
'url': 'https://example.com/apply',
'login': {'user': 'alice', 'pass': '...'},
'fields': {
'first_name': 'Alice',
'last_name': 'Smith',
'email': 'alice@example.com',
'role': 'Senior Engineer'
}
}Step 2: Login
page.goto(TASK['url'])
if page.locator('input[name="username"]').is_visible():
page.fill('input[name="username"]', TASK['login']['user'])
page.fill('input[name="password"]', TASK['login']['pass'])
page.click('button[type="submit"]')
page.wait_for_load_state('networkidle')Step 3: Field Filler
def fill_field(page, field_name, value):
for selector in [
f'input[name="{field_name}"]',
f'input[aria-label="{field_name}"]',
f'input[placeholder*="{field_name}"]',
f'label:has-text("{field_name}") + input'
]:
if page.locator(selector).count() > 0:
page.fill(selector, value)
return True
return False
# --- demo ---
class _FakeLocator:
def __init__(self, found):
self._found = found
def count(self):
return 1 if self._found else 0
class _FakePage:
"""Only the 3rd selector (placeholder) matches, to show the fallback loop."""
def locator(self, selector):
return _FakeLocator(found='placeholder*=' in selector)
def fill(self, selector, value):
print(f'Filled via selector: {selector} -> {value!r}')
page = _FakePage()
ok = fill_field(page, 'email', 'alice@example.com')
print(f'fill_field succeeded: {ok}')
Step 4: LLM-Assisted Locator
If standard selectors fail, ask the LLM to identify the field:
def ask_llm_for_selector(html, field_name):
prompt = f'Find the input element for "{field_name}" in this HTML and return a CSS selector. HTML:\n{html[:5000]}'
return llm.invoke(prompt).content.strip()Step 5: Vision Fallback
If even the LLM can't find a selector, take a screenshot and use vision:
if not filled:
screenshot = page.screenshot()
coordinate = vision_llm_locate(screenshot, field_name)
page.mouse.click(coordinate.x, coordinate.y)
page.keyboard.type(value)Step 6: Validate Each Step
After filling, read back the value:
actual = page.input_value(selector)
assert actual == expected_value, f'Expected {expected_value}, got {actual}'Step 7: Submit and Verify
page.click('button[type="submit"]')
page.wait_for_load_state('networkidle')
if page.locator('text=Thank you').count() == 0:
raise SubmissionFailed('Success page not detected')Step 8: Retry Logic
Forms sometimes glitch. Retry with idempotency checks:
@retry(stop=stop_after_attempt(3), wait=wait_fixed(2))
def fill_and_submit(page, task):
...Step 9: Captcha Handling
If the form has reCAPTCHA, use a solver service (2Captcha, Anti-Captcha) — or fail gracefully and ask a human.
Step 10: Logging and Replay
Save every action + screenshot. When something fails, you can replay the trace to debug:
context.tracing.stop(path=f'trace-{task_id}.zip')
# View with: playwright show-trace trace-abc.zipConcurrency Considerations
To handle many users' forms in parallel, spawn one browser context per task. Cap concurrency to avoid overwhelming the target site.
Respecting the Target
- Throttle between actions (humans don't type 200 cps)
- Honour robots.txt and ToS
- Use rotating IPs only when allowed
- Avoid abuse — agent traffic ≠ scraping spree
Robustness Strategy
What's the most reliable strategy for filling forms with unpredictable HTML?
Recap
Playwright + LLM-assisted selectors + vision fallback + per-step validation + retries + trace logs. The recipe for reliable form-filling.
Frequently asked questions
Is the “Building a Reliable Form-Filling Agent” lesson free?
Yes — the full text of “Building a Reliable Form-Filling Agent” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.
What will I learn in “Building a Reliable Form-Filling Agent”?
A practical case study: log into a portal, fill a multi-page form, and verify the submission. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Agents?
No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Building a Reliable Form-Filling Agent” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Agents lesson?
Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Browser Automation with Playwright
- Vision Models for Screen Understanding
- Computer-Use Patterns (Anthropic Computer-Use)
- Building a Reliable Form-Filling Agent