0Pricing
AI Agents · Lesson

Automated PR Review Comments

Reading PR diffs and posting inline review comments via API.

Automated PR Review Comments is a free AI Agents lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

What Is Automated PR Review?

An automated PR review agent reads changed files, analyzes diffs, and posts targeted review comments — just like a human reviewer, but faster and consistent. Use cases: security scanning, style enforcement, detecting common bugs, checking test coverage, or summarizing what a PR does.

from github import Github
import os

g = Github(token=os.environ['GITHUB_TOKEN'])
repo = g.get_repo('myorg/myrepo')

# Get a specific pull request by number
pr = repo.get_pull(42)
print(f'PR #{pr.number}: {pr.title}')
print(f'Author: {pr.user.login}')
print(f'Base branch: {pr.base.ref} <- Head: {pr.head.ref}')
print(f'State: {pr.state}')
print(f'Changed files: {pr.changed_files}')
print(f'Additions: +{pr.additions}')
print(f'Deletions: -{pr.deletions}')

Listing Open Pull Requests

Use repo.get_pulls() to list PRs. Filter by state (open, closed), base branch, or sort order. For a review bot, you typically process open PRs that haven't been reviewed yet.

from github import Github
import os

g = Github(token=os.environ['GITHUB_TOKEN'])
repo = g.get_repo('myorg/myrepo')

# All open PRs targeting main
open_prs = repo.get_pulls(
    state='open',
    base='main',
    sort='created',
    direction='desc'
)

for pr in open_prs:
    # Check if agent has already reviewed it
    reviews = list(pr.get_reviews())
    agent_reviewed = any(
        r.user.login == 'my-agent-bot[bot]'
        for r in reviews
    )
    if not agent_reviewed:
        print(f'Needs review: PR #{pr.number} - {pr.title}')

Getting Changed Files

pr.get_files() returns a list of File objects — each representing a file changed in the PR. Key properties: filename, status (added/modified/removed), additions, deletions, patch (the unified diff text).

from github import Github
import os

g = Github(token=os.environ['GITHUB_TOKEN'])
repo = g.get_repo('myorg/myrepo')
pr = repo.get_pull(42)

changed_files = list(pr.get_files())
print(f'Changed files: {len(changed_files)}')

for file in changed_files:
    print(f'\n{file.status.upper()}: {file.filename}')
    print(f'  +{file.additions} -{file.deletions}')

    # The patch is the unified diff
    if file.patch:
        print(f'  Patch preview: {file.patch[:200]}')

# Filter to only Python files
python_files = [
    f for f in changed_files
    if f.filename.endswith('.py') and f.status != 'removed'
]

Reading the File Patch (Diff)

The file.patch is a unified diff string. Lines starting with + were added, lines starting with - were removed. The @@ header shows which line numbers the hunk starts at. Parse this to identify exactly where issues exist in the new code.

def parse_patch_hunks(patch):
    if not patch:
        return []

    hunks = []
    current_hunk = None
    new_line_num = 0

    for line in patch.split('\n'):
        if line.startswith('@@'):
            # Parse: @@ -old_start,old_count +new_start,new_count @@
            import re
            match = re.search(r'\+([0-9]+)', line)
            if match:
                new_line_num = int(match.group(1))
            current_hunk = {'header': line, 'lines': [], 'start_line': new_line_num}
            hunks.append(current_hunk)
        elif current_hunk is not None:
            if line.startswith('+'):
                current_hunk['lines'].append({'type': 'add', 'content': line[1:], 'line': new_line_num})
                new_line_num += 1
            elif line.startswith('-'):
                current_hunk['lines'].append({'type': 'remove', 'content': line[1:], 'line': None})
            else:
                new_line_num += 1

    return hunks

# --- demo ---
patch = (
    '@@ -10,3 +10,4 @@\n'
    ' def add(a, b):\n'
    '-    return a - b\n'
    '+    return a + b\n'
    '+    # fixed subtraction bug\n'
)
for hunk in parse_patch_hunks(patch):
    print(hunk['header'])
    for line in hunk['lines']:
        print(' ', line)

Getting PR Commits

pr.get_commits() returns the commits in the PR. Each commit has a message, author, and list of changed files. Analyzing commit messages helps the review bot summarize what the PR does and check for good commit hygiene.

from github import Github
import os

g = Github(token=os.environ['GITHUB_TOKEN'])
repo = g.get_repo('myorg/myrepo')
pr = repo.get_pull(42)

commits = list(pr.get_commits())
print(f'Commits in PR: {len(commits)}')

for commit in commits:
    print(f'\nCommit: {commit.sha[:8]}')
    print(f'Author: {commit.commit.author.name}')
    print(f'Message: {commit.commit.message[:80]}')
    print(f'Changes: +{commit.stats.additions} -{commit.stats.deletions}')

# Check commit message quality
poor_messages = [
    c for c in commits
    if len(c.commit.message.split()[0] if c.commit.message else '') < 3
    or c.commit.message.lower().startswith(('wip', 'fix', 'test'))
]
if poor_messages:
    print(f'{len(poor_messages)} commits have poor message quality')

Posting a PR Review

Use pr.create_review() to submit a formal review. The event parameter controls the review type: 'APPROVE', 'REQUEST_CHANGES', or 'COMMENT'. A review can include a general body and a list of inline comments on specific lines.

from github import Github
import os

g = Github(token=os.environ['GITHUB_TOKEN'])
repo = g.get_repo('myorg/myrepo')
pr = repo.get_pull(42)

# Submit a review with a general comment (no inline comments)
pr.create_review(
    body=(
        '**Automated Code Review**\n\n'
        'I analyzed this PR and found the following:\n'
        '- No security issues detected\n'
        '- Style conforms to project guidelines\n'
        '- Test coverage looks adequate\n\n'
        '_This review was generated automatically._'
    ),
    event='COMMENT'  # 'APPROVE', 'REQUEST_CHANGES', or 'COMMENT'
)

Posting Inline Review Comments

Inline comments appear directly on specific lines in the diff. Each comment specifies path (file path), line (line number in the new file), and body. Pass a list of comments to pr.create_review(comments=[...]).

from github import Github
import os

g = Github(token=os.environ['GITHUB_TOKEN'])
repo = g.get_repo('myorg/myrepo')
pr = repo.get_pull(42)

# Build inline comments from analysis results
inline_comments = [
    {
        'path': 'src/auth.py',
        'line': 47,
        'body': (
            ':warning: **Security Issue:** Using `md5` for password hashing.\n'
            'Use `bcrypt` or `argon2` instead.\n'
            'Example: `import bcrypt; bcrypt.hashpw(password.encode(), bcrypt.gensalt())`'
        )
    },
    {
        'path': 'src/utils.py',
        'line': 23,
        'body': (
            ':bulb: Consider using `pathlib.Path` instead of `os.path.join` '
            'for better cross-platform compatibility.'
        )
    }
]

pr.create_review(
    body='Security and style review complete. See inline comments.',
    event='REQUEST_CHANGES',
    comments=inline_comments
)
print(f'Review submitted with {len(inline_comments)} inline comments')

LLM-Powered Code Analysis

Feed the file diff to an LLM with a security/quality review prompt. Ask for a structured JSON response listing issues with file path, line number, severity, and description. Use this to build the inline comments list.

import anthropic
import json
import os

client = anthropic.Anthropic(api_key=os.environ['ANTHROPIC_API_KEY'])

def analyze_file_with_llm(filename, patch):
    prompt = (
        f'Review this Python code diff for issues.\n'
        f'File: {filename}\n\n'
        f'Return JSON array: [{{"line": N, "severity": "error|warning|info", '
        f'"message": "..."}}]\n'
        f'Focus on: security vulnerabilities, bugs, and anti-patterns.\n'
        f'Diff:\n{patch[:3000]}'
    )

    response = client.messages.create(
        model='claude-opus-4-5',
        max_tokens=1000,
        messages=[{'role': 'user', 'content': prompt}]
    )

    try:
        issues = json.loads(response.content[0].text)
        return [
            {'path': filename, 'line': i.get('line', 1),
             'body': f'[{i.get("severity", "info").upper()}] {i.get("message", "")}'}
            for i in issues if isinstance(i, dict)
        ]
    except json.JSONDecodeError:
        return []

Full PR Review Agent Flow

Connect all pieces into a complete PR review flow: fetch files, filter to reviewable ones, analyze each with the LLM, collect comments, and submit the review in a single API call. Limit to the first 10 changed files to control LLM costs.

def run_pr_review(g, repo_name, pr_number):
    repo = g.get_repo(repo_name)
    pr = repo.get_pull(pr_number)

    print(f'Reviewing PR #{pr_number}: {pr.title}')

    # Get changed files (limit to 10)
    files = [
        f for f in list(pr.get_files())[:10]
        if f.filename.endswith(('.py', '.js', '.ts'))
        and f.patch
    ]

    all_comments = []
    for file in files:
        print(f'  Analyzing: {file.filename}')
        comments = analyze_file_with_llm(file.filename, file.patch)
        all_comments.extend(comments)

    # Determine review event based on findings
    critical = [c for c in all_comments if '[ERROR]' in c['body']]
    event = 'REQUEST_CHANGES' if critical else 'COMMENT'

    summary = (
        f'Analyzed {len(files)} files. '
        f'Found {len(all_comments)} issues '
        f'({len(critical)} critical).'
    )

    pr.create_review(body=summary, event=event, comments=all_comments[:25])
    print(f'Review submitted: {event} with {len(all_comments)} comments')

Checking for Existing Agent Reviews

Before posting a review, check if your agent bot already reviewed this PR in the current state. Avoid duplicate reviews by checking the existing reviews list for your bot's username.

def should_review_pr(pr, bot_username):
    reviews = list(pr.get_reviews())

    # Check if bot already reviewed this PR
    bot_reviews = [
        r for r in reviews
        if r.user and r.user.login == bot_username
    ]

    if not bot_reviews:
        return True  # No previous review, proceed

    # Check if new commits were pushed since last review
    last_review_time = max(r.submitted_at for r in bot_reviews)
    commits = list(pr.get_commits())
    latest_commit_time = commits[-1].commit.author.date if commits else None

    if latest_commit_time and latest_commit_time > last_review_time:
        print(f'New commits since last review, re-reviewing...')
        return True

    print(f'PR #{pr.number} already reviewed by {bot_username}')
    return False

# --- demo: minimal stand-ins for PyGithub-style PR/review/commit objects ---
import datetime
from types import SimpleNamespace

class _FakeCommit:
    def __init__(self, date):
        self.commit = SimpleNamespace(author=SimpleNamespace(date=date))

class _FakeReview:
    def __init__(self, login, submitted_at):
        self.user = SimpleNamespace(login=login)
        self.submitted_at = submitted_at

class _FakePR:
    def __init__(self, number, reviews, commits):
        self.number = number
        self._reviews = reviews
        self._commits = commits
    def get_reviews(self):
        return self._reviews
    def get_commits(self):
        return self._commits

pr_never_reviewed = _FakePR(101, [], [_FakeCommit(datetime.datetime(2026, 8, 1))])
print('PR #101 needs review:', should_review_pr(pr_never_reviewed, 'coddy-bot'))

review_time = datetime.datetime(2026, 8, 1, 10, 0)
pr_up_to_date = _FakePR(102, [_FakeReview('coddy-bot', review_time)],
                        [_FakeCommit(review_time - datetime.timedelta(hours=1))])
print('PR #102 needs review:', should_review_pr(pr_up_to_date, 'coddy-bot'))

PR Review Security Scan Pattern

A common automated review pattern is scanning for security anti-patterns in added lines: hardcoded secrets, use of deprecated functions, SQL string concatenation, or eval() calls. Check each added line against a set of patterns and flag findings as inline comments.

import re

SECURITY_PATTERNS = [
    (r'eval\s*\(', 'Dangerous eval() call — code injection risk'),
    (r'os\.system\s*\(', 'os.system() is unsafe; use subprocess with a list'),
    (r'pickle\.loads?\s*\(', 'pickle.load is unsafe with untrusted data'),
    (r'execute\(.*%\s*', 'Possible SQL injection — use parameterized queries'),
    (r'md5\s*\(|hashlib\.md5', 'MD5 is cryptographically broken for passwords; use bcrypt'),
]

def scan_patch_for_security(filename, patch):
    if not patch:
        return []
    comments = []
    new_line = 0
    for line in patch.split('\n'):
        if line.startswith('@@'):
            import re as _re
            m = _re.search(r'\+([0-9]+)', line)
            if m:
                new_line = int(m.group(1))
        elif line.startswith('+') and not line.startswith('+++'):
            content = line[1:]
            for pattern, message in SECURITY_PATTERNS:
                if re.search(pattern, content):
                    comments.append({'path': filename, 'line': new_line,
                                     'body': f':rotating_light: **Security:** {message}'})
            new_line += 1
        elif not line.startswith('-'):
            new_line += 1
    return comments

# --- demo ---
patch = (
    '@@ -1,2 +1,3 @@\n'
    ' def run(cmd):\n'
    '+    os.system(cmd)\n'
    '+    hashlib.md5(cmd.encode())\n'
)
for c in scan_patch_for_security('run.py', patch):
    print(c)

Quick Check: Review Events

Test your understanding of PR review options.

Automated PR Review Recap

Your agent can now review pull requests programmatically:

  • repo.get_pull(number) — get a PR object with metadata
  • pr.get_files() — list changed files with filename, patch, additions
  • pr.get_commits() — inspect commit messages and stats
  • pr.create_review(body, event, comments) — submit review with optional inline comments
  • Inline comment format: {'path': filename, 'line': N, 'body': markdown}
  • Use 'COMMENT' for observations, 'REQUEST_CHANGES' for blocking issues
  • Check existing reviews before re-reviewing to avoid duplicates
  • Limit file analysis to 10 files and 25 inline comments to stay within API limits

Frequently asked questions

Is the “Automated PR Review Comments” lesson free?

Yes — the full text of “Automated PR Review Comments” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Automated PR Review Comments”?

Reading PR diffs and posting inline review comments via API. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Automated PR Review Comments” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. GitHub REST API Overview
  2. Listing and Managing Issues
  3. Automated PR Review Comments
  4. Commit History and Diff Analysis
← Back to AI Agents