0Pricing
Learn AI with Python · Lesson

REST API Fundamentals for Data Collection

HTTP GET/POST, headers, authentication (API key, Bearer), JSON parsing with requests.

REST API Fundamentals for Data Collection is a free Learn AI with Python lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why APIs Matter for AI

Models are only as good as their data. Web APIs let you collect fresh, structured data on demand — weather, prices, social posts, public datasets — instead of relying on a single static file.

This course uses Python and the popular requests library to fetch and store that data.

What Is a REST API?

A REST API exposes resources over HTTP. You send a request to a URL (endpoint) and get back data, usually as JSON.

The main HTTP methods:

  • GET — read data
  • POST — create data
  • PUT/PATCH — update
  • DELETE — remove

For data collection you mostly use GET.

A First GET Request

requests.get(url) sends a GET request and returns a Response object. The response holds the status, headers, and body.

import requests

resp = requests.get("https://api.example.com/users")
print(resp.status_code)   # 200
print(resp.text[:200])    # raw body as a string

Checking response.status_code

The status code tells you whether the request succeeded:

  • 200 OK
  • 301/302 redirect
  • 400 bad request
  • 401/403 auth problem
  • 404 not found
  • 429 rate limited
  • 500 server error
resp = requests.get(url)
if resp.status_code == 200:
    print("Success")
else:
    print("Failed:", resp.status_code)

Parsing JSON with response.json()

Most APIs return JSON. response.json() parses the body into Python dicts and lists automatically.

Do not parse JSON by hand — .json() handles it and raises a clear error if the body is not valid JSON.

resp = requests.get("https://api.example.com/users/1")
data = resp.json()
print(data["name"])
print(data["email"])

Sending Query Parameters with params

Filter and configure requests using query parameters. Pass a dict to params and requests builds the URL for you, handling encoding.

The example becomes ?city=London&units=metric on the URL.

params = {"city": "London", "units": "metric"}
resp = requests.get("https://api.example.com/weather", params=params)
print(resp.url)
print(resp.json())

Sending Headers

Headers carry metadata such as the content type, a user agent, or an API key/token. Pass them as a dict to headers.

Many APIs require an Authorization header for access.

headers = {
    "Authorization": "Bearer YOUR_TOKEN",
    "User-Agent": "my-data-collector/1.0",
}
resp = requests.get("https://api.example.com/data", headers=headers)
print(resp.json())

raise_for_status() for Robust Code

Instead of checking codes by hand, call resp.raise_for_status(). It raises an HTTPError for any 4xx or 5xx response, so failures fail loudly.

Wrap it in try/except to handle errors gracefully.

import requests

try:
    resp = requests.get(url, timeout=10)
    resp.raise_for_status()
    data = resp.json()
except requests.HTTPError as e:
    print("HTTP error:", e)
except requests.RequestException as e:
    print("Network error:", e)

Always Set a Timeout

Without a timeout, a stalled server can hang your script forever. Always pass one (in seconds).

For data-collection loops that hit thousands of endpoints, a missing timeout is a common cause of frozen jobs.

resp = requests.get(url, timeout=10)   # give up after 10 seconds

Reusing Connections with Session

A requests.Session reuses the underlying TCP connection across requests and lets you set headers once. For many calls to the same host this is much faster.

import requests

session = requests.Session()
session.headers.update({"Authorization": "Bearer YOUR_TOKEN"})

for uid in [1, 2, 3]:
    resp = session.get(f"https://api.example.com/users/{uid}")
    print(resp.json()["name"])

session.close()

A Clean Fetch Helper

Wrap the pattern into a reusable function: set a session, apply timeout, raise on errors, and return parsed JSON.

You will build on this helper in the next lessons for pagination and storage.

def fetch_json(session, url, params=None):
    resp = session.get(url, params=params, timeout=10)
    resp.raise_for_status()
    return resp.json()

Quick Check: Robust Requests

You want your code to raise an exception automatically on a 404 or 500 response.

Recap: REST API Fundamentals

You can now collect data over HTTP:

  • requests.get(url, params=..., headers=...) to make requests
  • response.status_code and raise_for_status() to detect failures
  • response.json() to parse the body
  • timeout to avoid hangs
  • requests.Session to reuse connections and headers

Next: collecting datasets that span many pages.

Frequently asked questions

Is the “REST API Fundamentals for Data Collection” lesson free?

Yes — the full text of “REST API Fundamentals for Data Collection” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.

What will I learn in “REST API Fundamentals for Data Collection”?

HTTP GET/POST, headers, authentication (API key, Bearer), JSON parsing with requests. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Learn AI with Python?

No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “REST API Fundamentals for Data Collection” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Learn AI with Python lesson?

Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. REST API Fundamentals for Data Collection
  2. Paginating and Collecting Large Datasets
  3. Storing Collected Data Efficiently
  4. Working with Public Data APIs
← Back to Learn AI with Python