0Pricing
R Academy · Lesson

HTML Structure and CSS Selectors

Understand DOM trees and write CSS selectors to target elements.

HTML Structure and CSS Selectors is a free R Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the R Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

The HTML DOM Tree

Every web page is structured as a Document Object Model (DOM) tree. HTML elements nest inside each other forming a parent-child hierarchy. Web scraping navigates this tree to extract data.

# HTML structure conceptually:
# <html>
#   <body>
#     <div class='container'>
#       <h1 id='title'>Hello</h1>
#       <p class='text'>World</p>
#     </div>
#   </body>
# </html>

# rvest lets us query this tree with CSS selectors
library(rvest)
html <- read_html('<div><h1>Title</h1><p class="info">Text</p></div>')
html_text(html_element(html, 'h1'))

CSS Selector: Tag Selector

The simplest CSS selector targets elements by their tag name. Writing p selects all paragraph elements. Writing h1 selects all level-1 headings.

library(rvest)
html <- read_html('
  <div>
    <p>First paragraph</p>
    <p>Second paragraph</p>
    <h2>A heading</h2>
  </div>
')
# Tag selector: selects all <p> elements
nodes <- html_elements(html, 'p')
html_text2(nodes)
# Returns: c('First paragraph', 'Second paragraph')

CSS Selector: Class and ID

Use .classname to select elements by class and #idname to select a unique element by ID. Classes can appear many times; IDs should be unique on the page.

library(rvest)
html <- read_html('
  <div>
    <p class="highlight">Important text</p>
    <p class="normal">Regular text</p>
    <span id="price">$9.99</span>
  </div>
')
# Class selector (prefix with .)
html_text2(html_element(html, '.highlight'))
# ID selector (prefix with #)
html_text2(html_element(html, '#price'))

Descendant and Child Selectors

div p selects any p inside a div (any depth). div > p selects only direct children. h1 + p selects a p immediately following an h1 (adjacent sibling).

library(rvest)
html <- read_html('
  <div class="outer">
    <p>Direct child</p>
    <section>
      <p>Nested deeper</p>
    </section>
  </div>
')
# Descendant: both paragraphs
all_p <- html_elements(html, 'div p')
length(all_p)  # 2

# Direct child only
direct_p <- html_elements(html, 'div.outer > p')
length(direct_p)  # 1

Attribute Selectors

CSS attribute selectors let you filter by HTML attributes: [attr] checks existence, [attr='val'] checks exact value, [attr*='val'] checks if value contains substring.

library(rvest)
html <- read_html('
  <div>
    <a href="https://example.com">External</a>
    <a href="/about">Internal</a>
    <a>No href</a>
  </div>
')
# Select only anchors that have an href attribute
with_href <- html_elements(html, 'a[href]')
length(with_href)  # 2

# Select anchors with href starting with https
external <- html_elements(html, 'a[href^="https"]')
html_text2(external)  # 'External'

XPath Basics

XPath is an alternative to CSS selectors, more powerful for complex queries. In rvest use html_element(html, xpath='//tag'). //p means any p anywhere; /html/body/p is an absolute path.

library(rvest)
html <- read_html('
  <html><body>
    <table>
      <tr><td class="price">10.99</td></tr>
      <tr><td class="price">5.50</td></tr>
    </table>
  </body></html>
')
# XPath: select all td with class price
nodes <- html_elements(html, xpath = '//td[@class="price"]')
html_text2(nodes)
# c('10.99', '5.50')

# XPath text() function
nodes2 <- html_elements(html, xpath = '//td[contains(@class,"price")]')
html_text2(nodes2)

Combining Selectors

Selectors can be combined for precision. div.card h2 selects h2 inside a div with class card. A comma separates multiple independent selectors: h1, h2, h3 selects all three heading levels.

library(rvest)
html <- read_html('
  <div class="card">
    <h2>Card Title</h2>
    <p class="desc">Description here</p>
    <span class="price">$19</span>
  </div>
  <div class="footer">
    <h2>Footer Heading</h2>
  </div>
')
# Only h2 inside .card
card_h2 <- html_element(html, 'div.card h2')
html_text2(card_h2)  # 'Card Title'

# Multiple selectors with comma
price_desc <- html_elements(html, '.price, .desc')
html_text2(price_desc)

SelectorGadget Tool

SelectorGadget is a browser bookmarklet that helps you find CSS selectors interactively. Click on elements you want (highlighted green) and elements you don't want (highlighted red) — it generates the minimal CSS selector automatically.

# SelectorGadget workflow:
# 1. Open target page in Chrome/Firefox
# 2. Activate SelectorGadget bookmarklet
# 3. Click element you want -> turns green, selector appears
# 4. Click elements you DON'T want -> turns red, selector narrows
# 5. Copy the selector shown at the bottom
# 6. Use in rvest:

library(rvest)
# Example: SelectorGadget found '.product-title' for us
# page <- read_html('https://books.toscrape.com')
# titles <- html_elements(page, '.product_pod h3 a')
# html_text2(titles)
cat('SelectorGadget is available at selectorgadget.com')

Inspecting Pages in DevTools

Browser DevTools (F12) let you inspect the DOM directly. Right-click any element, choose Inspect, and the Elements panel shows the HTML. Hovering highlights elements on the page, helping you identify the exact tag, class, and ID to target.

# DevTools workflow for finding selectors:
# 1. F12 -> Elements tab
# 2. Click the cursor icon (Inspector)
# 3. Click the element on the page
# 4. Right-click highlighted HTML -> Copy -> Copy selector
# 5. Paste selector into rvest

# The copied selector might look like:
# '#main > div.results > article:nth-child(1) > h3'
# Simplify it: usually '.results h3' works just as well

# Validate your selector in the Console with:
# document.querySelectorAll('.results h3')
library(rvest)
cat('Always verify selectors return the elements you expect')

Pseudo-classes and nth-child

CSS pseudo-classes like :first-child, :last-child, and :nth-child(n) let you select elements based on their position within their parent — useful when elements lack distinguishing classes.

library(rvest)
html <- read_html('
  <ul>
    <li>First</li>
    <li>Second</li>
    <li>Third</li>
    <li>Fourth</li>
  </ul>
')
# First item
first <- html_element(html, 'li:first-child')
html_text2(first)  # 'First'

# Third item
third <- html_element(html, 'li:nth-child(3)')
html_text2(third)  # 'Third'

# Even items
evens <- html_elements(html, 'li:nth-child(even)')
html_text2(evens)  # c('Second', 'Fourth')

Selector Specificity Rules

When multiple selectors could match, specificity determines which wins. IDs beat classes; classes beat tags. In scraping, this matters less — but knowing specificity helps you write precise selectors that avoid false matches.

library(rvest)
html <- read_html('
  <div id="header" class="top">
    <p class="title">Main Title</p>
  </div>
  <div class="content">
    <p class="title">Content Title</p>
  </div>
')
# Overly broad: gets BOTH titles
broad <- html_elements(html, 'p.title')
html_text2(broad)

# Specific: only header title
specific <- html_element(html, '#header p.title')
html_text2(specific)  # 'Main Title'

# Be as specific as needed but not more
content_title <- html_element(html, '.content .title')
html_text2(content_title)

Quick Check

Test your understanding of CSS selectors used in web scraping with rvest.

Recap: HTML & CSS Selectors

Key takeaways: The DOM is a tree; CSS selectors navigate it. Use tag for element type, .class for class, #id for unique elements, > for direct children, [attr] for attributes. SelectorGadget and DevTools help you discover selectors interactively. XPath (//tag[@attr]) handles cases where CSS falls short.

# Summary of key CSS selectors for web scraping:
# 'p'            -> all <p> elements
# '.price'       -> elements with class='price'
# '#main'        -> element with id='main'
# 'div > p'      -> direct child p of div
# 'a[href]'      -> a elements that have href attr
# 'a[href*=http] -> a elements where href contains 'http'
# 'li:nth-child(2)' -> second li in its parent
# 'h1, h2'       -> both h1 and h2 elements
# xpath='//td[@class="price"]' -> XPath alternative
library(rvest)
cat('Selectors are the foundation of reliable web scraping')

Frequently asked questions

Is the “HTML Structure and CSS Selectors” lesson free?

Yes — the full text of “HTML Structure and CSS Selectors” is free to read here on the web, and the R Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the R Academy course, upgrade to CoddyKit PRO.

What will I learn in “HTML Structure and CSS Selectors”?

Understand DOM trees and write CSS selectors to target elements. You practise R Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start R Academy?

No prior experience is required. R Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “HTML Structure and CSS Selectors” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this R Academy lesson?

Yes. Every R Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. HTML Structure and CSS Selectors
  2. html_element() and html_text() Basics
  3. Scraping Tables and Links
  4. Handling Pagination and Multiple Pages
← Back to R Academy