0Pricing
Python Academy · Lesson

Parsing HTML with BeautifulSoup

Navigate the parse tree and extract data with selectors.

Installing BeautifulSoup

beautifulsoup4 parses HTML and XML. Use the lxml parser for speed or html.parser for zero-dependency use.

# pip install beautifulsoup4 lxml
from bs4 import BeautifulSoup

html = "<h1>Hello</h1><p class='intro'>World</p>"
soup = BeautifulSoup(html, "lxml")
print(soup.h1.text)   # Hello

Finding Elements

find(tag) returns the first matching element. find_all(tag) returns a list of all matching elements.

from bs4 import BeautifulSoup

html = "<ul><li>A</li><li>B</li><li>C</li></ul>"
soup = BeautifulSoup(html, "lxml")
print(soup.find("li").text)       # A
print([li.text for li in soup.find_all("li")])  # [A,B,C]

All lessons in this course

  1. HTTP Requests with requests and httpx
  2. Parsing HTML with BeautifulSoup
  3. Building a Scrapy Spider
  4. Handling JavaScript and Anti-scraping Measures
← Back to Python Academy