AI Agents · Lesson

File Format Handling: CSV, JSON, and TXT

csv module, json.load/dump, and safe text encoding for agent tools.

File Format Handling: CSV, JSON, and TXT is a free AI Agents lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

The Three Most Common Agent File Formats

Agents read and write three file formats constantly: CSV for tabular data, JSON for structured objects, and plain text for logs, prompts, and reports. Each has different parsing requirements, edge cases, and best practices. Python has excellent built-in support for all three.

import csv
import json
from pathlib import Path

# Detect format from extension
def read_data_file(file_path):
    path = Path(file_path)
    if path.suffix == '.csv':
        return read_csv(path)
    elif path.suffix == '.json':
        return read_json(path)
    elif path.suffix == '.txt':
        return path.read_text(encoding='utf-8')
    else:
        raise ValueError(f'Unsupported format: {path.suffix}')

csv.reader — Basic CSV Parsing

csv.reader parses a CSV file row by row, returning each row as a list of strings. It handles quoted fields, embedded commas, and newlines inside quoted values correctly — unlike splitting on commas manually, which breaks on edge cases.

import csv

with open('sales.csv', 'w', newline='') as f:
    f.write('product,qty,price\nWidget,3,9.99\nGadget,1,19.99\n')

with open('sales.csv', 'r', encoding='utf-8', newline='') as f:
    reader = csv.reader(f)
    header = next(reader)
    print('Columns:', header)

    for row in reader:
        product = row[0]
        quantity = int(row[1])
        price = float(row[2])
        print(f'{product}: {quantity} units at ${price}')

csv.DictReader — Row as Dictionary

csv.DictReader reads each row as an OrderedDict (or regular dict in Python 3.8+) with column headers as keys. This is much easier to work with than positional indexing — your code stays readable even if columns are reordered.

import csv

with open('employees.csv', 'w', newline='') as f:
    f.write('name,department,salary\nAlice,Eng,95000\nBob,Sales,70000\n')

with open('employees.csv', 'r', encoding='utf-8', newline='') as f:
    reader = csv.DictReader(f)
    print('Fields:', reader.fieldnames)

    total_salary = 0
    for row in reader:
        name = row['name']
        department = row['department']
        salary = float(row['salary'])
        total_salary += salary
        print(f'{name} ({department}): ${salary:,.2f}')

    print(f'Total payroll: ${total_salary:,.2f}')

csv.writer and DictWriter — Writing CSV

Use csv.writer to write rows as lists, or csv.DictWriter to write rows as dicts. Always pass newline='' when opening the file — the csv module handles line endings itself to avoid double newlines on Windows.

import csv

results = [
    {'task_id': 'T001', 'status': 'completed', 'duration_s': 12.5},
    {'task_id': 'T002', 'status': 'failed', 'duration_s': 3.1},
    {'task_id': 'T003', 'status': 'completed', 'duration_s': 45.8},
]

fieldnames = ['task_id', 'status', 'duration_s']

with open('task_results.csv', 'w', newline='', encoding='utf-8') as f:
    writer = csv.DictWriter(f, fieldnames=fieldnames)
    writer.writeheader()  # write column names
    writer.writerows(results)

print('Wrote task_results.csv')

json.load() and json.dump() — File I/O

Use json.load(file) to parse a JSON file and json.dump(obj, file) to write one. These work with file objects. Use json.loads(string) and json.dumps(obj) for strings. Always use indent=2 for readable output.

import json

with open('config.json', 'w', encoding='utf-8') as f:
    json.dump({'api_url': 'https://api.example.com', 'timeout': 15}, f)

with open('config.json', 'r', encoding='utf-8') as f:
    config = json.load(f)

print('API URL:', config.get('api_url'))
print('Timeout:', config.get('timeout', 30))

output_data = {
    'run_id': 'abc123',
    'items': [1, 2, 3],
    'meta': {'agent': 'v2', 'model': 'gpt-4o'}
}

with open('output.json', 'w', encoding='utf-8') as f:
    json.dump(
        output_data, f,
        indent=2,
        ensure_ascii=False
    )
print('Written output.json')

Handling JSON Decode Errors

Malformed JSON files are common in agent pipelines — incomplete writes, truncated downloads, or encoding issues. Always wrap json.load() in a try/except and provide clear error messages that include the file path for debugging.

import json
from pathlib import Path

def safe_load_json(file_path):
    path = Path(file_path)
    try:
        with open(path, 'r', encoding='utf-8') as f:
            return json.load(f)
    except json.JSONDecodeError as e:
        print(f'Invalid JSON in {path}: line {e.lineno}, col {e.colno}')
        print(f'  Error: {e.msg}')
        # Show the problem area
        content = path.read_text(encoding='utf-8')
        lines = content.split('\n')
        if e.lineno <= len(lines):
            print(f'  Content: {lines[e.lineno-1][:80]}')
        return None
    except FileNotFoundError:
        print(f'File not found: {path}')
        return None

# --- demo ---
Path('good.json').write_text('{"model": "gpt-4o"}', encoding='utf-8')
Path('bad.json').write_text('{"model": "gpt-4o", }', encoding='utf-8')

print('Loading good.json:', safe_load_json('good.json'))
print('Loading bad.json:', safe_load_json('bad.json'))

Reading JSONL (JSON Lines) Format

Many AI APIs and data pipelines use JSONL (JSON Lines) — one JSON object per line. This format supports streaming and is easy to process line by line without loading the entire file into memory. Each line is a complete, self-contained JSON object.

import json

with open('events.jsonl', 'w', encoding='utf-8') as f:
    f.write('{"event": "start"}\n{"event": "stop"}\nnot json\n')

results = []
with open('events.jsonl', 'r', encoding='utf-8') as f:
    for line_num, line in enumerate(f, 1):
        line = line.strip()
        if not line:
            continue
        try:
            event = json.loads(line)
            results.append(event)
        except json.JSONDecodeError as e:
            print(f'Bad JSON on line {line_num}: {e}')

print(f'Loaded {len(results)} events')

with open('output.jsonl', 'w', encoding='utf-8') as f:
    for record in results:
        f.write(json.dumps(record, ensure_ascii=False) + '\n')

Reading Plain Text Files

Plain text is the simplest format — logs, prompts, reports, and configuration files. Read the entire file with .read(), or process line by line. For large files, always use the line-by-line approach to keep memory usage constant.

from pathlib import Path

Path('system_prompt.txt').write_text('You are a helpful agent.', encoding='utf-8')
with open('agent.log', 'w', encoding='utf-8') as f:
    f.write('INFO: boot\nERROR: disk full\nCRITICAL: crash\nINFO: recovered\n')

prompt = Path('system_prompt.txt').read_text(encoding='utf-8')
print(f'Prompt length: {len(prompt)} characters')

error_lines = []
with open('agent.log', 'r', encoding='utf-8') as f:
    for line in f:
        line = line.rstrip()
        if not line:
            continue
        if 'ERROR' in line or 'CRITICAL' in line:
            error_lines.append(line)

print(f'Found {len(error_lines)} error lines')

with open('summary.txt', 'w', encoding='utf-8') as f:
    f.write('Agent Run Summary\n')
    f.write('=' * 40 + '\n')
    for error in error_lines[:10]:
        f.write(f'  {error}\n')
print('Summary written')

Handling BOM (Byte Order Mark)

Files exported from Excel or Windows tools often start with a BOM (Byte Order Mark) — an invisible \ufeff character. If not handled, it corrupts the first field name in CSV parsing. Use encoding='utf-8-sig' to strip it automatically.

import csv

with open('windows_export.csv', 'wb') as f:
    f.write('name,email,age\nAlice,alice@x.com,30\n'.encode('utf-8'))
with open('file.txt', 'wb') as f:
    f.write(b'\xef\xbb\xbfhello')

with open('windows_export.csv', 'r', encoding='utf-8') as f:
    reader = csv.DictReader(f)
    first = next(reader)
    print(list(first.keys()))

with open('windows_export.csv', 'r', encoding='utf-8-sig') as f:
    reader = csv.DictReader(f)
    first = next(reader)
    print(list(first.keys()))

content = open('file.txt', 'rb').read()
if content.startswith(b'\xef\xbb\xbf'):
    content = content[3:]
text = content.decode('utf-8')
print('Decoded:', text)

Handling Encoding Errors

When reading files from unknown sources, encoding errors are common. The errors parameter of open() controls what happens: 'replace' substitutes bad characters with ?, 'ignore' drops them, and 'backslashreplace' escapes them. For strict validation, use 'strict' (the default).

from pathlib import Path

def read_with_fallback(file_path):
    path = Path(file_path)

    # Try UTF-8 first
    try:
        return path.read_text(encoding='utf-8')
    except UnicodeDecodeError:
        pass

    # Try Latin-1 (handles most European files)
    try:
        return path.read_text(encoding='latin-1')
    except UnicodeDecodeError:
        pass

    # Last resort: replace bad characters
    text = path.read_text(encoding='utf-8', errors='replace')
    print(f'Warning: {path.name} had encoding errors (chars replaced)')
    return text

# --- demo ---
Path('notes_utf8.txt').write_text('Notes: café, naïve, résumé', encoding='utf-8')
text = read_with_fallback('notes_utf8.txt')
print(f'Read {len(text)} chars: {text!r}')

Handling Malformed CSV Files

Real-world CSV files have issues: extra commas, missing fields, inconsistent quoting, or mixed delimiters. Use the quoting and error_bad_lines options, and wrap row parsing in try/except to skip bad rows gracefully.

import csv

with open('messy_data.csv', 'w', newline='') as f:
    f.write('name,email,score\nAlice,alice@x.com,88\nBob,,90\nCarol,carol@x.com,notanumber\n')

valid_rows = []
error_count = 0

with open('messy_data.csv', 'r', encoding='utf-8', newline='') as f:
    reader = csv.DictReader(f)
    expected_fields = {'name', 'email', 'score'}

    for line_num, row in enumerate(reader, start=2):
        try:
            if not all(row.get(f, '').strip() for f in expected_fields):
                raise ValueError(f'Missing required field in row {line_num}')
            score = float(row['score'])
            valid_rows.append({
                'name': row['name'].strip(),
                'email': row['email'].strip().lower(),
                'score': score
            })
        except (ValueError, KeyError) as e:
            error_count += 1
            print(f'Skipping row {line_num}: {e}')

print(f'Valid: {len(valid_rows)}, Errors: {error_count}')

Quick Check: CSV DictReader vs reader

Test your understanding of CSV parsing options.

File Format Handling Recap

You can now parse and write all major agent file formats:

CSV: use csv.DictReader for dict rows, csv.DictWriter for output; always newline='' on open
JSON: json.load(f) to parse, json.dump(obj, f, indent=2) to write; catch JSONDecodeError
JSONL: read and parse each line with json.loads(line); great for streaming data
Plain text: .read() for small files, line iteration for large ones
BOM: use encoding='utf-8-sig' for Windows-exported files
Encoding errors: try UTF-8, fall back to latin-1, or use errors='replace'

Frequently asked questions

Is the “File Format Handling: CSV, JSON, and TXT” lesson free?

Yes — the full text of “File Format Handling: CSV, JSON, and TXT” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “File Format Handling: CSV, JSON, and TXT”?

csv module, json.load/dump, and safe text encoding for agent tools. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “File Format Handling: CSV, JSON, and TXT” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

Reading and Writing Files in Agent Context
Directory Traversal and File Discovery
File Format Handling: CSV, JSON, and TXT
Safe File Operations with Error Handling

← Back to AI Agents