신뢰할 수 있는 양식 작성 에이전트 구축

현실적인 프로젝트

아키텍처

Step 1: Define the Task

TASK = {
    'url': 'https://example.com/apply',
    'login': {'user': 'alice', 'pass': '...'},
    'fields': {
        'first_name': 'Alice',
        'last_name': 'Smith',
        'email': 'alice@example.com',
        'role': 'Senior Engineer'
    }
}

Step 2: Login

page.goto(TASK['url'])
if page.locator('input[name="username"]').is_visible():
    page.fill('input[name="username"]', TASK['login']['user'])
    page.fill('input[name="password"]', TASK['login']['pass'])
    page.click('button[type="submit"]')
    page.wait_for_load_state('networkidle')

Step 3: Field Filler

def fill_field(page, field_name, value):
    for selector in [
        f'input[name="{field_name}"]',
        f'input[aria-label="{field_name}"]',
        f'input[placeholder*="{field_name}"]',
        f'label:has-text("{field_name}") + input'
    ]:
        if page.locator(selector).count() > 0:
            page.fill(selector, value)
            return True
    return False

# --- demo ---
class _FakeLocator:
    def __init__(self, found):
        self._found = found
    def count(self):
        return 1 if self._found else 0

class _FakePage:
    """Only the 3rd selector (placeholder) matches, to show the fallback loop."""
    def locator(self, selector):
        return _FakeLocator(found='placeholder*=' in selector)
    def fill(self, selector, value):
        print(f'Filled via selector: {selector} -> {value!r}')

page = _FakePage()
ok = fill_field(page, 'email', 'alice@example.com')
print(f'fill_field succeeded: {ok}')

Step 4: LLM 지원 로케이터

def ask_llm_for_selector(html, field_name):
    prompt = f'Find the input element for "{field_name}" in this HTML and return a CSS selector. HTML:\n{html[:5000]}'
    return llm.invoke(prompt).content.strip()

Step 5: 비전 대체 수단

if not filled:
    screenshot = page.screenshot()
    coordinate = vision_llm_locate(screenshot, field_name)
    page.mouse.click(coordinate.x, coordinate.y)
    page.keyboard.type(value)

Step 6: 각 단계 검증

actual = page.input_value(selector)
assert actual == expected_value, f'Expected {expected_value}, got {actual}'

Step 7: Submit and Verify

page.click('button[type="submit"]')
page.wait_for_load_state('networkidle')
if page.locator('text=Thank you').count() == 0:
    raise SubmissionFailed('Success page not detected')

Step 8: 재시도 로직

@retry(stop=stop_after_attempt(3), wait=wait_fixed(2))
def fill_and_submit(page, task):
    ...

Step 9: 캡차 처리

Step 10: 기록 및 재생

context.tracing.stop(path=f'trace-{task_id}.zip')
# View with: playwright show-trace trace-abc.zip

동시성 고려 사항

대상 사이트 존중

견고성 전략

복습