HTTP Requests with requests and httpx
Fetch web pages, handle redirects, sessions, and headers.
requests Library
requests is the standard synchronous HTTP library. Simple, battle-tested, and perfect for scripts and one-off API calls.
# pip install requests
import requests
r = requests.get("https://api.github.com/users/octocat")
print(r.status_code) # 200
print(r.json()["login"]) # octocatQuery Parameters and Headers
Pass query parameters with params= and custom headers with headers=.
import requests
r = requests.get(
"https://api.example.com/search",
params={"q": "python", "page": 1},
headers={"Authorization": "Bearer my-token"}
)
print(r.url) # full URL with query stringAll lessons in this course
- HTTP Requests with requests and httpx
- Parsing HTML with BeautifulSoup
- Building a Scrapy Spider
- Handling JavaScript and Anti-scraping Measures