0Pricing
Web Scraping & Bots · Aula

XPath para seleção robusta

Descubra o XPath como uma linguagem poderosa para navegar por documentos XML e HTML, permitindo recuperar dados altamente específicos.

XPath para seleção robusta é uma aula grátis de Web Scraping & Bots no CoddyKit. Esta é a aula 3 de 4. Você pode ler a aula completa abaixo gratuitamente — depois pratica ao vivo no navegador com um editor de código integrado e um tutor de IA 24/7. Faz parte do caminho de aprendizado de Web Scraping & Bots, e seu progresso é sincronizado entre a web e o app CoddyKit. O curso de Web Scraping & Bots inclui 4 aulas no total.

Partes desta aula ainda não foram traduzidas e aparecem em inglês.

What is XPath?

Welcome to XPath! It stands for XML Path Language, and it's a powerful tool for navigating and selecting nodes in XML and HTML documents.

Think of it as a specialized language for finding specific pieces of information within a web page's structure.

  • XPath is incredibly flexible for complex selections.
  • It allows navigation in any direction (up, down, sideways).
  • It's essential for robust data extraction.

XPath & the Document Tree

Before diving into syntax, let's understand how XPath "sees" a document. It views HTML/XML as a tree structure.

  • Each element, attribute, and even text is a "node" in this tree.
  • XPath expressions are like directions, guiding you from the root of the tree to the specific nodes you want to find.

This tree model allows for precise, hierarchical navigation.

Select by Tag Name

The simplest way to select elements is by their tag name. We use // to select elements anywhere in the document, or / for direct children.

For example, //h1 selects all <h1> elements.

We'll use Python's lxml library for our examples.

from lxml import html

html_doc = """
<html>
<body>
  <h1>My Product Page</h1>
  <div class="product-list">
    <h2>Product A</h2>
  </div>
</body>
</html>
"""

tree = html.fromstring(html_doc)
# Select all h1 tags
results = tree.xpath("//h1")

for element in results:
    print(element.text_content())

Targeting Elements by Attribute

Often, you need to select elements based on their attributes, like id or class. XPath uses predicates ([]) for this.

Use @attribute to refer to an attribute. For example, //div[@class='product-card'] selects all <div> elements with class="product-card".

from lxml import html

html_doc = """
<html>
<body>
  <div class="product-card" id="p1">
    <h2>Product A</h2>
  </div>
  <div class="product-card" id="p2">
    <h2>Product B</h2>
  </div>
</body>
</html>
"""

tree = html.fromstring(html_doc)
# Select div elements with class 'product-card'
results = tree.xpath("//div[@class='product-card']")

for element in results:
    print(element.get('id')) # Get the id attribute

Finding Elements by Text

XPath can also find elements based on their visible text content. This is very powerful for locating specific labels or values.

  • Use text() to refer to the direct text of an element.
  • Use contains(text(), 'part') to find elements whose text contains a specific substring.

For example, //p[text()='$19.99'] finds a paragraph with that exact price.

from lxml import html

html_doc = """
<html>
<body>
  <p class="price">$19.99</p>
  <p class="price">$29.50</p>
  <span>In Stock!</span>
</body>
</html>
"""

tree = html.fromstring(html_doc)
# Select p elements containing '$19.99'
results = tree.xpath("//p[text()='$19.99']")

for element in results:
    print(element.text_content())

Navigating Up & Down the Tree

XPath lets you move around the document tree. Use / for direct child nodes and // for any descendant.

  • div/p: Selects <p> elements that are direct children of <div>.
  • //div//p: Selects <p> elements that are descendants of any <div>, no matter how deep.
  • /parent::div: Selects the parent <div> of the current node.
from lxml import html

html_doc = """
<html>
<body>
  <div class="product-card">
    <h2>Product A</h2>
    <p class="price">$19.99</p>
  </div>
  <div class="footer">
    <p>Copyright</p>
  </div>
</body>
</html>
"""

tree = html.fromstring(html_doc)
# Find the price within a product-card
results = tree.xpath("//div[@class='product-card']/p[@class='price']")

for element in results:
    print(element.text_content())

Filtering with Positional Predicates

Predicates ([]) can also filter elements based on their position among siblings. This is useful for selecting the first, last, or specific item in a list.

  • //li[1]: Selects the first <li> element.
  • //li[last()]: Selects the last <li> element.
  • //div[position()=2]: Selects the second <div> element.
from lxml import html

html_doc = """
<html>
<body>
  <ul>
    <li>Item One</li>
    <li>Item Two</li>
    <li>Item Three</li>
  </ul>
</body>
</html>
"""

tree = html.fromstring(html_doc)
# Select the second list item
results = tree.xpath("//ul/li[2]")

for element in results:
    print(element.text_content())

Complex Selections with and/or

You can combine multiple conditions within a predicate using and or or to make your selections even more precise.

  • //a[@href and @class='button']: Selects <a> tags that have both an href attribute AND a class of 'button'.
  • //p[@class='price' or contains(text(), 'Sale')]: Selects paragraphs with class 'price' OR containing the text 'Sale'.
from lxml import html

html_doc = """
<html>
<body>
  <a href="/link1" class="button">Link 1</a>
  <a href="/link2">Link 2</a>
  <a class="button">Link 3</a>
</body>
</html>
"""

tree = html.fromstring(html_doc)
# Select 'a' tags that have both 'href' and 'class="button"'
results = tree.xpath("//a[@href and @class='button']")

for element in results:
    print(element.text_content())

Practical Extraction: Product Price

Let's put it all together. Imagine you want to extract the price of "Product B" from a list of products. We can combine attribute selection and hierarchy.

We'll first find the specific product card, then navigate to its price element.

from lxml import html

html_doc = """
<html>
<body>
  <div class="product-list">
    <div class="product-card" id="p1"><h2>Product A</h2><p class="price">$19.99</p></div>
    <div class="product-card" id="p2"><h2>Product B</h2><p class="price">$29.50</p></div>
    <div class="product-card" id="p3"><h2>Product C</h2><p class="price">$5.00</p></div>
  </div>
</body>
</html>
"""

tree = html.fromstring(html_doc)
# Find the price of Product B
xpath_exp = "//div[@id='p2']/p[@class='price']"
results = tree.xpath(xpath_exp)

for element in results:
    print(element.text_content())

XPath Challenge

Consider the following HTML snippet:

<div class="container">
  <ul id="main-menu">
    <li>Home</li>
    <li class="active">About Us</li>
    <li>Contact</li>
  </ul>
  <div class="content">
    <p>Welcome!</p>
    <a href="/learn">Learn More</a>
  </div>
</div>

Recap: XPath for Precision

Congratulations! You've learned the fundamentals of XPath, a highly versatile language for navigating and selecting elements in HTML and XML documents.

  • XPath uses a tree model to represent documents.
  • You can select elements by tag, attribute, text content, and position.
  • Predicates [] and operators like and/or allow for incredibly precise targeting.

Mastering XPath will significantly enhance your ability to extract exactly the data you need from complex web pages. Keep practicing!

Perguntas Frequentes

A aula “XPath para seleção robusta” é grátis?

Sim — o texto completo de “XPath para seleção robusta” é grátis para ler aqui na web. Para praticá-la interativamente (um editor de código integrado e um tutor de IA 24/7) e desbloquear o restante do curso de Web Scraping & Bots, atualize para CoddyKit PRO. O curso de Web Scraping & Bots inclui 4 aulas no total.

O que vou aprender em “XPath para seleção robusta”?

Descubra o XPath como uma linguagem poderosa para navegar por documentos XML e HTML, permitindo recuperar dados altamente específicos. Você pratica Web Scraping & Bots com código prático que executa diretamente no navegador, e um tutor de IA 24/7 responde suas dúvidas enquanto trabalha na aula.

Preciso ter experiência prévia para começar Web Scraping & Bots?

Nenhuma experiência prévia é necessária. Web Scraping & Bots no CoddyKit é estruturado para alunos iniciantes até avançados, então você pode começar aqui ou desde o início e aprender no seu ritmo. Esta é a aula 3 de 4.

Quanto tempo leva a aula “XPath para seleção robusta”?

A maioria das aulas CoddyKit leva cerca de 5–10 minutos. Cada uma é compacta e interativa, então você faz progresso constante e retoma exatamente de onde parou entre web e app.

Posso escrever e executar código nesta aula de Web Scraping & Bots?

Sim. Cada aula de Web Scraping & Bots inclui um editor de código integrado, então você escreve e executa código real direto no navegador e recebe feedback de IA instantaneamente — nenhuma configuração local necessária.

Todas as aulas deste curso

  1. Navegação por estruturas HTML complexas
  2. Seletores CSS para maior precisão
  3. XPath para seleção robusta
  4. Extraindo Dados de Tabelas HTML
← Voltar para Web Scraping & Bots