0Pricing
Python Academy · Lesson

HTTP Requests with requests and httpx

Fetch web pages, handle redirects, sessions, and headers.

HTTP Requests with requests and httpx is a free Python Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Python Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

requests Library

requests is the standard synchronous HTTP library. Simple, battle-tested, and perfect for scripts and one-off API calls.

# pip install requests
import requests

r = requests.get("https://api.github.com/users/octocat")
print(r.status_code)   # 200
print(r.json()["login"])   # octocat

Query Parameters and Headers

Pass query parameters with params= and custom headers with headers=.

import requests

r = requests.get(
    "https://api.example.com/search",
    params={"q": "python", "page": 1},
    headers={"Authorization": "Bearer my-token"}
)
print(r.url)    # full URL with query string

POST with JSON Body

Send JSON data with json= — requests sets Content-Type: application/json automatically.

import requests

r = requests.post(
    "https://api.example.com/users",
    json={"name": "Alice", "email": "alice@example.com"}
)
print(r.status_code)   # 201
print(r.json())

Session Objects

Use a requests.Session to persist headers, cookies, and connection pooling across multiple requests.

import requests

with requests.Session() as s:
    s.headers.update({"Authorization": "Bearer token"})
    users = s.get("https://api.example.com/users").json()
    profile = s.get("https://api.example.com/me").json()

Timeout and Retries

Always set a timeout. Use urllib3.util.retry.Retry with a HTTPAdapter for automatic retries.

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

s = requests.Session()
retry = Retry(total=3, backoff_factor=1)
s.mount("https://", HTTPAdapter(max_retries=retry))

r = s.get("https://api.example.com/data", timeout=5)

Handling Errors

Call r.raise_for_status() to raise an HTTPError for 4xx/5xx responses.

import requests

try:
    r = requests.get("https://api.example.com/data")
    r.raise_for_status()
    data = r.json()
except requests.HTTPError as e:
    print(f"HTTP error: {e.response.status_code}")
except requests.ConnectionError:
    print("Network unreachable")

Downloading Files

Stream large file downloads with stream=True to avoid loading everything into memory.

import requests

with requests.get("https://example.com/large.zip", stream=True) as r:
    r.raise_for_status()
    with open("large.zip", "wb") as f:
        for chunk in r.iter_content(chunk_size=8192):
            f.write(chunk)

httpx — Modern Async HTTP

httpx has the same API as requests but supports async. Use it for FastAPI apps and async workflows.

# pip install httpx
import httpx

# Synchronous:
r = httpx.get("https://httpbin.org/get")
print(r.json())

# Asynchronous:
import asyncio
async def main():
    async with httpx.AsyncClient() as c:
        r = await c.get("https://httpbin.org/get")
        print(r.json())
asyncio.run(main())

httpx AsyncClient

Share a single AsyncClient across requests for connection pooling. Create it on startup, close on shutdown.

import httpx, asyncio

CLIENT: httpx.AsyncClient | None = None

async def startup():
    global CLIENT
    CLIENT = httpx.AsyncClient(timeout=10)

async def shutdown():
    await CLIENT.aclose()

Authentication Helpers

Both requests and httpx support auth helpers: auth=(user, pass) for Basic auth, and custom Auth classes for token auth.

import httpx

class BearerAuth(httpx.Auth):
    def __init__(self, token): self.token = token
    def auth_flow(self, request):
        request.headers["Authorization"] = f"Bearer {self.token}"
        yield request

async with httpx.AsyncClient(auth=BearerAuth("my-token")) as c:
    r = await c.get("https://api.example.com/me")

Mocking HTTP in Tests

Use respx (for httpx) or responses (for requests) to mock HTTP calls in tests.

# pip install respx
import httpx, respx, asyncio

@respx.mock
async def test_api():
    respx.get("https://api.example.com/users").mock(
        return_value=httpx.Response(200, json=[{"id":1}])
    )
    async with httpx.AsyncClient() as c:
        r = await c.get("https://api.example.com/users")
    assert r.json() == [{"id": 1}]

Quick Check

What does response.raise_for_status() do?

Recap

Use requests for synchronous HTTP. Use httpx when async is needed. Always set timeouts, use sessions for connection reuse, call raise_for_status(), and stream large downloads. Mock HTTP in tests with respx or responses.

Frequently asked questions

Is the “HTTP Requests with requests and httpx” lesson free?

Yes — the full text of “HTTP Requests with requests and httpx” is free to read here on the web, and the Python Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Python Academy course, upgrade to CoddyKit PRO.

What will I learn in “HTTP Requests with requests and httpx”?

Fetch web pages, handle redirects, sessions, and headers. You practise Python Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Python Academy?

No prior experience is required. Python Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “HTTP Requests with requests and httpx” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Python Academy lesson?

Yes. Every Python Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. HTTP Requests with requests and httpx
  2. Parsing HTML with BeautifulSoup
  3. Building a Scrapy Spider
  4. Handling JavaScript and Anti-scraping Measures
← Back to Python Academy