0Pricing
Web Scraping & Bots · Ders

Tarayıcı Etkileşimlerini Otomatikleştirme

Tıklamalar, kaydırmalar, form gönderimleri ve öğelerin yüklenmesini bekleme gibi kullanıcı eylemlerini program aracılığıyla taklit edin.

Tarayıcı Etkileşimlerini Otomatikleştirme, CoddyKit'te ücretsiz bir Web Scraping & Bots dersidir. Bu, 4 dersinin 2. dersidir. Aşağıdan dersin tamamını ücretsiz okuyabilir, sonra tarayıcıda yerleşik kod editörü ve 7/24 yapay zeka koçu ile uygulamalı olarak pratik yapabilirsin. Bu, Web Scraping & Bots öğrenme yolunun bir parçasıdır ve ilerlemeniz web ve CoddyKit uygulaması arasında senkronize olur. Web Scraping & Bots kursu toplamda 4 dersten oluşur.

Bu dersin bazı bölümleri henüz çevrilmemiş olup İngilizce olarak gösterilmektedir.

Automate Browser Actions

In the previous lesson, we introduced Selenium for handling dynamic web content. Now, let's learn how to make our bots interact with web pages!

We'll simulate real user actions like clicking buttons, typing into forms, and scrolling, making our scrapers much more powerful.

Locate Elements First

Before you can interact with an element (like a button or a text box), you need to find it on the page. Selenium provides several ways to do this.

You'll use methods like find_element(By.ID, "some_id") or find_element(By.CLASS_NAME, "some_class") to pinpoint your target.

  • By.ID: Unique ID attribute.
  • By.NAME: Name attribute.
  • By.CLASS_NAME: CSS class.
  • By.XPATH: Powerful path expressions.
  • By.CSS_SELECTOR: CSS selector syntax.

Making a Click

The most common interaction is clicking. Whether it's a button, a link, or a checkbox, the .click() method does the job.

After finding an element, just call .click() on it. This simulates a user's mouse click.

Try clicking a fictitious button:

from selenium import webdriver
from selenium.webdriver.common.by import By
import time

def main():
    driver = webdriver.Chrome() # Or Firefox, Edge, etc.
    driver.get("https://www.example.com")
    try:
        # Imagine a button with ID 'myButton'.
        # On example.com, we can click the 'More information...' link.
        more_info_link = driver.find_element(By.LINK_TEXT, "More information...")
        more_info_link.click()
        print("Clicked 'More information...' link.")
        time.sleep(2) # To see the new page
    finally:
        driver.quit()

if __name__ == "__main__":
    main()

Entering Text

To fill out forms or search bars, you use the .send_keys() method. This simulates typing text into an input field.

You can also send special keys like Keys.ENTER, Keys.TAB, etc., from selenium.webdriver.common.keys.

Let's type into a search box:

from selenium import webdriver
from selenium.webdriver.common.by import By
import time

def main():
    driver = webdriver.Chrome()
    driver.get("https://www.google.com") # Using google for a search box
    try:
        search_box = driver.find_element(By.NAME, "q")
        search_box.send_keys("CoddyKit Selenium")
        time.sleep(2) # See the typed text
    finally:
        driver.quit()

if __name__ == "__main__":
    main()

Sending Form Data

After filling out a form, you often need to submit it. You can do this in a couple of ways:

  • Click the submit button: submit_button.click()
  • Call .submit() on any input element within the form: input_field.submit()

The .submit() method is convenient as it doesn't require finding the specific submit button.

Example of submitting a form (after typing):

from selenium import webdriver
from selenium.webdriver.common.by import By
import time

def main():
    driver = webdriver.Chrome()
    driver.get("https://www.google.com")
    try:
        search_box = driver.find_element(By.NAME, "q")
        search_box.send_keys("Selenium forms")
        search_box.submit() # Submits the form containing this element
        time.sleep(3) # Observe search results
    finally:
        driver.quit()

if __name__ == "__main__":
    main()

Why We Need to Wait

Web pages often load content dynamically using JavaScript. This means elements might not be immediately available when Selenium tries to find them.

If your script tries to interact with an element that hasn't loaded yet, it will throw an error. This is where "waits" come in!

Waits tell Selenium to pause execution until a certain condition is met or a timeout occurs.

Simple Implicit Waits

An implicit wait tells the WebDriver to poll the DOM (Document Object Model) for a certain amount of time when trying to find an element.

If the element is not immediately available, the driver will wait for the specified duration before throwing a NoSuchElementException.

It's a global setting for the entire driver session:

from selenium import webdriver
import time

def main():
    driver = webdriver.Chrome()
    # Set implicit wait for 10 seconds
    driver.implicitly_wait(10)
    try:
        driver.get("https://www.example.com")
        print("Implicit wait set for 10 seconds.")
        # Any subsequent find_element calls will wait up to 10s
        # if the element isn't immediately found.
        time.sleep(2) # Just to show the wait is active
    finally:
        driver.quit()

if __name__ == "__main__":
    main()

Precise Explicit Waits

Explicit waits are more powerful and flexible. They allow you to define a specific condition to wait for, rather than a fixed time.

You use WebDriverWait in combination with expected_conditions (often imported as EC) to specify what you're waiting for. This is best for specific elements and dynamic content.

Here's how to wait for a (fictional) element with ID 'dynamicDiv' to be visible:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

def main():
    driver = webdriver.Chrome()
    driver.get("https://www.example.com")
    try:
        print("Waiting for an element to be visible...")
        # WebDriverWait waits up to 10 seconds
        # for an element with ID 'dynamicDiv' to become visible.
        # This element does not exist on example.com, so it will timeout.
        # In a real app, replace with a real ID.
        element = WebDriverWait(driver, 10).until(
            EC.visibility_of_element_located((By.ID, "dynamicDiv"))
        )
        print(f"Element found: {element.text}")
    except Exception as e:
        print(f"Element not found or timed out: {e}")
    finally:
        driver.quit()

if __name__ == "__main__":
    main()

Scrolling for More Content

Some websites load content as you scroll down (infinite scroll). To access this content, your bot needs to scroll the page.

You can use JavaScript execution to scroll to a specific position or to the bottom of the page.

To scroll to the very bottom:

from selenium import webdriver
import time

def main():
    driver = webdriver.Chrome()
    driver.get("https://www.wikipedia.org") # A page that can be scrolled
    try:
        # Scroll down to the bottom of the page
        driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
        print("Scrolled to the bottom of the page.")
        time.sleep(2) # Observe the scroll
        # You can also scroll incrementally:
        # driver.execute_script("window.scrollBy(0, 500);")
    finally:
        driver.quit()

if __name__ == "__main__":
    main()

Interacting with Select Menus

HTML <select> elements (dropdown menus) require a special approach in Selenium using the Select class.

First, find the <select> element, then create a Select object, and finally use methods like select_by_visible_text(), select_by_value(), or select_by_index().

Example:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
import time

def main():
    driver = webdriver.Chrome()
    # For this example, we'll use a test page with a dropdown.
    # In a real scenario, you'd navigate to a specific URL.
    driver.get("https://www.selenium.dev/selenium/web/formPage.html")
    try:
        # Find the select element by its ID
        select_element = driver.find_element(By.ID, "selectMenu")
        select = Select(select_element)
        
        # Select an option by its visible text
        select.select_by_visible_text("Example select text")
        print("Selected 'Example select text' by visible text.")
        time.sleep(2)
        
        # Select an option by its value attribute
        select.select_by_value("2")
        print("Selected option with value '2'.")
        time.sleep(2)

    finally:
        driver.quit()

if __name__ == "__main__":
    main()

Test Your Knowledge

You've learned about automating various browser interactions and the crucial role of waits.

Which of the following are valid ways to interact with web elements using Selenium in Python?

Recap: Automating Interactions

Well done! You now know how to make your Selenium bots perform common user actions:

  • Finding Elements: Using various By strategies.
  • Interacting: .click() for buttons/links, .send_keys() for text input, and .submit() for forms.
  • Waiting: Essential for dynamic content, using implicitly_wait() or more precise WebDriverWait with expected_conditions.
  • Advanced: Scrolling with JavaScript and handling dropdowns with the Select class.

These skills are fundamental for building robust web automation scripts!

Sıkça Sorulan Sorular

“Tarayıcı Etkileşimlerini Otomatikleştirme” dersi ücretsiz mi?

Evet — “Tarayıcı Etkileşimlerini Otomatikleştirme” dersin tüm metni burada web'de ücretsiz olarak okunabilir. Etkileşimli olarak pratik yapmak (yerleşik kod editörü ve 7/24 yapay zeka koçu) ve Web Scraping & Bots kursunun geri kalanını açmak için CoddyKit PRO'ya yükselt. Web Scraping & Bots kursu toplamda 4 dersten oluşur.

“Tarayıcı Etkileşimlerini Otomatikleştirme” dersinde ne öğreneceğim?

Tıklamalar, kaydırmalar, form gönderimleri ve öğelerin yüklenmesini bekleme gibi kullanıcı eylemlerini program aracılığıyla taklit edin. Web Scraping & Bots ile uygulamalı kodu tarayıcıda doğrudan çalıştırarak pratik yaparsın ve 7/24 yapay zeka koçu dersi çalışırken sorularını yanıtlar.

Web Scraping & Bots öğrenmeye başlamak için deneyim gerekli mi?

Önceden deneyim gerekmez. CoddyKit'te Web Scraping & Bots, başlangıçtan ileri seviyeye kadar yapılandırıldığı için buradan başlayabilir veya başından başlayıp kendi hızında ilerleme yapabilirsin. Bu, 4 dersinin 2. dersidir.

“Tarayıcı Etkileşimlerini Otomatikleştirme” dersi ne kadar sürer?

Çoğu CoddyKit dersi yaklaşık 5–10 dakika sürer. Her biri kısa ve etkileşimli olduğu için sabit ilerleme yaparsın ve web ile uygulama arasında tam olarak bıraktığın yerden devam edebilirsin.

Bu Web Scraping & Bots dersinde kod yazıp çalıştırabilir miyim?

Evet. Her Web Scraping & Bots dersi yerleşik bir kod editörü içerir, bu sayede tarayıcıda gerçek kod yazıp çalıştırabilir ve anlık yapay zeka geri bildirimi alırsın — yerel kurulum gerekli değildir.

Bu kursun tüm dersleri

  1. Selenium'a Giriş
  2. Tarayıcı Etkileşimlerini Otomatikleştirme
  3. JavaScript'ten Veri Çıkarma
  4. Dinamik Sayfalar için Bekleme Stratejileri
← Web Scraping & Bots Sayfasına Dön