R Academy · Lesson

Scraping Tables and Links

Parse HTML tables into data frames and collect all hyperlinks on a page.

Scraping Tables and Links is a free R Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the R Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

HTML Tables in Web Pages

HTML tables (<table>) are the most convenient targets for scraping because they already have structure. rvest's html_table() converts them directly into R data frames, handling headers automatically.

library(rvest)
html <- read_html('
  <table>
    <thead><tr><th>Country</th><th>GDP</th><th>Pop</th></tr></thead>
    <tbody>
      <tr><td>USA</td><td>25T</td><td>330M</td></tr>
      <tr><td>China</td><td>18T</td><td>1400M</td></tr>
    </tbody>
  </table>
')
# html_table converts to data frame
tbl <- html_table(html_element(html, 'table'))
class(tbl)      # 'data.frame'
names(tbl)      # c('Country', 'GDP', 'Pop')
nrow(tbl)       # 2
print(tbl)

html_table() on a Full Page

When a page has multiple tables, use html_elements() (plural) to get all tables, then html_table() on the resulting nodeset to return a list of data frames. Select the one you need by index.

library(rvest)
html <- read_html('
  <table id="t1"><tr><th>A</th></tr><tr><td>1</td></tr></table>
  <table id="t2"><tr><th>X</th><th>Y</th></tr>
    <tr><td>10</td><td>20</td></tr></table>
')
# Get ALL tables as a list of data frames
tables <- html_table(html_elements(html, 'table'))
length(tables)  # 2

# First table
tables[[1]]

# Second table
tables[[2]]

# Select a specific table by its id
table2 <- html_table(html_element(html, '#t2'))
print(table2)

Cleaning Table Data

Tables scraped from websites often have messy headers, merged cells, or extra whitespace. Use janitor::clean_names() for headers, and standard dplyr/stringr operations to clean cell values.

library(rvest)
html <- read_html('
  <table>
    <tr><th>Product Name</th><th>Price (USD)</th><th>In Stock?</th></tr>
    <tr><td>  Widget A  </td><td>$12.50</td><td>Yes</td></tr>
    <tr><td>Gadget B</td><td>$7.99</td><td>No</td></tr>
  </table>
')
df <- html_table(html_element(html, 'table'))

# Clean column names
names(df) <- c('product', 'price', 'in_stock')

# Strip whitespace and dollar signs
df$product  <- trimws(df$product)
df$price    <- as.numeric(gsub('[$]', '', df$price))
df$in_stock <- df$in_stock == 'Yes'
print(df)

Scraping All Links on a Page

Links are anchor (<a>) tags with href attributes. Use html_elements(page, 'a') to get all anchors, then html_attr('href') for URLs and html_text2() for link text.

library(rvest)
html <- read_html('
  <div>
    <a href="/about">About Us</a>
    <a href="/products">Products</a>
    <a href="https://partner.com" rel="external">Partner</a>
    <a>No href link</a>
  </div>
')
# Get all anchor elements
anchors <- html_elements(html, 'a')

# Extract text and href
link_text <- html_text2(anchors)
link_href <- html_attr(anchors, 'href')

data.frame(text = link_text, href = link_href)
# Note: last row has NA href

Filtering Meaningful Links

Real pages have navigation links, footer links, and internal anchors mixed with the links you actually want. Filter by removing NAs, anchors (#), javascript: links, and applying pattern matching to keep only relevant URLs.

library(rvest)
html <- read_html('
  <div>
    <a href="/article/1">Article One</a>
    <a href="/article/2">Article Two</a>
    <a href="#top">Back to top</a>
    <a href="javascript:void(0)">JS link</a>
    <a href="/article/3">Article Three</a>
  </div>
')
anchors <- html_elements(html, 'a')
hrefs <- html_attr(anchors, 'href')
texts <- html_text2(anchors)

# Keep only article links
article_idx <- grepl('^/article/', hrefs) & !is.na(hrefs)
article_links <- data.frame(
  text = texts[article_idx],
  href = hrefs[article_idx]
)
print(article_links)

Relative to Absolute URLs

Scraped href values are often relative paths like /page. Convert them to absolute URLs by prepending the base URL. The xml2::url_absolute() function handles this correctly.

library(rvest)
base_url <- 'https://books.toscrape.com'

html <- read_html('
  <ul>
    <li><a href="/catalogue/book1">Book One</a></li>
    <li><a href="/catalogue/book2">Book Two</a></li>
    <li><a href="https://external.com/book">External</a></li>
  </ul>
')
hrefs <- html_attr(html_elements(html, 'a'), 'href')

# xml2::url_absolute resolves relative + keeps absolute
abs_urls <- xml2::url_absolute(hrefs, base = base_url)
abs_urls
# 'https://books.toscrape.com/catalogue/book1'
# 'https://books.toscrape.com/catalogue/book2'
# 'https://external.com/book'  <- kept as-is

Targeted Link Scraping

Instead of scraping all links and filtering, use specific CSS selectors to target only the links you need. Combine element context and attribute selectors for precision.

library(rvest)
html <- read_html('
  <nav class="breadcrumb">
    <a href="/">Home</a> > <a href="/books">Books</a>
  </nav>
  <ul class="products">
    <li><a href="/books/1" class="prod-link">Clean Code</a></li>
    <li><a href="/books/2" class="prod-link">Refactoring</a></li>
  </ul>
  <footer>
    <a href="/privacy">Privacy</a>
  </footer>
')
# Target ONLY product links (not nav or footer)
prod_links <- html_elements(html, 'ul.products a.prod-link')
data.frame(
  title = html_text2(prod_links),
  url   = html_attr(prod_links, 'href')
)

Merging Table and Link Data

Often a table's cells contain links. Combine html_table() for text values with targeted html_attr() for the embedded links to build a richer data frame.

library(rvest)
html <- read_html('
  <table class="results">
    <tr><th>Book</th><th>Author</th></tr>
    <tr>
      <td><a href="/b/1">Clean Code</a></td>
      <td>Robert Martin</td>
    </tr>
    <tr>
      <td><a href="/b/2">Refactoring</a></td>
      <td>Martin Fowler</td>
    </tr>
  </table>
')
# Get text from table
df <- html_table(html_element(html, 'table'))

# Get links embedded in first column
links <- html_attr(html_elements(html, 'table td a'), 'href')
df$url <- links
print(df)

html_table() fill Parameter

HTML tables sometimes have missing cells or irregularly structured rows. The fill=TRUE parameter in html_table() fills missing values with NA instead of throwing an error.

library(rvest)
html <- read_html('
  <table>
    <tr><th>Name</th><th>Score</th><th>Grade</th></tr>
    <tr><td>Alice</td><td>95</td><td>A</td></tr>
    <tr><td>Bob</td><td>80</td></tr>
  </table>
')
# Without fill=TRUE this may error on uneven rows
df <- html_table(html_element(html, 'table'), fill = TRUE)
print(df)
# Name  Score Grade
# Alice    95     A
# Bob      80  <NA>

# fill=TRUE is safe even when rows are uniform
is.na(df[2, 'Grade'])  # TRUE

Extracting Image Sources

Images use src attributes (and sometimes data-src for lazy loading). The same html_attr() approach extracts image URLs for downloading or cataloguing.

library(rvest)
html <- read_html('
  <div class="gallery">
    <img src="/img/photo1.jpg" alt="Sunset">
    <img src="/img/photo2.jpg" alt="Mountains">
    <img data-src="/img/lazy.jpg" class="lazy" alt="Lake">
  </div>
')
# Regular images
images <- html_elements(html, 'img')
srcs  <- html_attr(images, 'src')
alts  <- html_attr(images, 'alt')

data.frame(alt = alts, src = srcs)
# For lazy-loaded images check data-src
lazy_src <- html_attr(html_element(html, '.lazy'), 'data-src')
lazy_src  # '/img/lazy.jpg'

Saving Scraped Data

After scraping tables and links into a data frame, save results with write.csv() or readr::write_csv(). Always include a timestamp or source URL for data provenance.

library(rvest)
html <- read_html('
  <table>
    <tr><th>Product</th><th>Price</th></tr>
    <tr><td>Widget</td><td>9.99</td></tr>
    <tr><td>Gadget</td><td>14.50</td></tr>
  </table>
')
df <- html_table(html_element(html, 'table'))

# Add metadata
df$scraped_at <- Sys.time()
df$source_url <- 'https://example.com/products'

# Save to CSV
# write.csv(df, 'products.csv', row.names = FALSE)
print(df)
cat('Data ready for analysis or storage')

Quick Check

Test your understanding of scraping HTML tables and links with rvest.

Recap: Tables and Links

Key takeaways: html_table() converts HTML tables to data frames automatically; use fill=TRUE for irregular tables. For links, get all anchors with html_elements('a'), text with html_text2(), and URLs with html_attr('href'). Convert relative URLs to absolute with xml2::url_absolute(). Filter unwanted links by pattern. Merge table text and embedded link URLs for richer datasets.

library(rvest)
# Table scraping pattern:
# tables <- html_table(html_elements(page, 'table'))
# df <- tables[[1]]

# Link scraping pattern:
# links <- html_elements(page, 'a[href]')
# data.frame(
#   text = html_text2(links),
#   url  = xml2::url_absolute(
#            html_attr(links, 'href'),
#            base = base_url
#          )
# )

# Combine: scrape table text AND embedded link URLs
cat('Tables + links = most of what the web offers')

Frequently asked questions

Is the “Scraping Tables and Links” lesson free?

Yes — the full text of “Scraping Tables and Links” is free to read here on the web, and the R Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the R Academy course, upgrade to CoddyKit PRO.

What will I learn in “Scraping Tables and Links”?

Parse HTML tables into data frames and collect all hyperlinks on a page. You practise R Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start R Academy?

No prior experience is required. R Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Scraping Tables and Links” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this R Academy lesson?

Yes. Every R Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to R Academy