Web Scraping & Bots · Lesson

Scheduling Basic Tasks

Implement simple scheduling mechanisms to run your bots at specific intervals for continuous automation.

Scheduling Basic Tasks is a free Web Scraping & Bots lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Web Scraping & Bots learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why Schedule Your Bot?

Ever wished your bot could run itself without you pressing "Go"? That's where scheduling comes in!

Scheduling allows your bot to perform tasks automatically at specific times or intervals. This is super useful for:

Monitoring: Checking a website every hour for updates.
Data Collection: Scraping news articles daily.
Notifications: Sending alerts when a price changes.

Let's learn how to make your bot work on its own schedule.

Pausing Your Bot with `time.sleep()`

The simplest way to introduce a delay in your Python bot is using the time.sleep() function. It pauses your program for a specified number of seconds.

This is great for waiting between requests to avoid overwhelming a server, or just to slow things down.

Try this simple example:

import time

def main():
    print("Starting task...")
    time.sleep(2) # Pause for 2 seconds
    print("Task finished after delay.")

if __name__ == "__main__":
    main()

Making Tasks Repeat

To make your bot do something repeatedly, you can combine time.sleep() with a while loop. This creates a basic, continuous operation.

Your bot will execute the task, pause, and then repeat indefinitely (or until you stop it).

Be careful with infinite loops in real bots – you'll need a way to stop them!

import time

def main():
    counter = 0
    while counter < 3: # Run 3 times for demo
        print(f"Bot is running task {counter + 1}...")
        time.sleep(3) # Wait 3 seconds
        counter += 1
    print("Bot finished its scheduled runs.")

if __name__ == "__main__":
    main()

When `time.sleep()` Isn't Enough

While a while loop with time.sleep() works for simple repetition, it has limitations:

Blocking: The entire program pauses. If you have other things to do, they also stop.
Fixed Intervals: It's hard to schedule tasks at specific clock times (e.g., "every day at 9 AM") rather than just "every X seconds."
Complexity: Managing multiple, different schedules becomes messy.

For more sophisticated scheduling, we need a better tool.

Meet the `schedule` Library

For more flexible and readable scheduling within Python, the schedule library is a popular choice. It lets you define tasks and their schedules in a very intuitive way.

First, you need to install it:

pip install schedule

Once installed, you can tell it exactly when to run your functions.

Scheduling a Task for the Future

The schedule library makes it easy to run a function after a certain delay. You tell it how often, and what function to call.

Let's create a simple function and schedule it to run once after a few seconds.

import schedule
import time

def my_task():
    print("This task ran as scheduled!")

def main():
    print("Scheduling task to run in 5 seconds...")
    schedule.every(5).seconds.do(my_task)

    # This loop keeps the scheduler running
    start_time = time.time()
    while True:
        schedule.run_pending()
        time.sleep(1) # Check every second
        # For this demo, let's stop after 6 seconds
        if time.time() > start_time + 6:
            break
    print("Scheduler stopped.")

if __name__ == "__main__":
    main()

Setting Up Recurring Tasks

The real power of schedule comes from defining recurring tasks. You can schedule tasks to run every second, minute, hour, day, or even specific days of the week.

Here are some examples of how you can define recurring schedules:

schedule.every(10).minutes.do(job)
schedule.every().hour.do(job)
schedule.every().day.at("10:30").do(job)
schedule.every().monday.do(job)
schedule.every().wednesday.at("13:15").do(job)

It's very flexible!

Running Your Scheduled Bot

After you've defined your scheduled tasks, you need a way for the schedule library to constantly check if any tasks are due to run. This is done with a simple loop.

The schedule.run_pending() function checks for tasks, and time.sleep() ensures your bot isn't constantly busy checking.

import schedule
import time

def scrape_news():
    print("Scraping news headlines...")
    # In a real bot, this would fetch data
    print("News scraped at", time.strftime("%H:%M:%S"))

def check_price_alert():
    print("Checking for price drops...")
    # In a real bot, this would check prices
    print("Price check at", time.strftime("%H:%M:%S"))

def main():
    print("Bot is starting. Tasks scheduled.")
    schedule.every(5).seconds.do(scrape_news)
    schedule.every(10).seconds.do(check_price_alert)

    # Main loop to run pending tasks
    start_time = time.time()
    while True:
        schedule.run_pending()
        time.sleep(1) # Check every second
        if time.time() - start_time > 25: # Run for 25 seconds for demo
            break
    print("Bot finished its scheduled operations for the demo.")

if __name__ == "__main__":
    main()

Beyond In-Bot Scheduling

While the schedule library is great for simple, in-process scheduling, large-scale or mission-critical bots often use external scheduling tools.

Cron (Linux/macOS): A powerful command-line utility for scheduling tasks.
Task Scheduler (Windows): A built-in tool for automating tasks.
Cloud Schedulers: Services like AWS EventBridge or Google Cloud Scheduler for cloud-based bots.

These tools can start your Python script itself, offering more robustness and system-level control.

Schedule It!

You've learned about different ways to schedule tasks for your bot. Now, let's test your understanding!

Recap: Scheduling Your Bots

Great job! You've mastered the basics of scheduling in Python. We covered:

Using time.sleep() for simple pauses and basic loops for repetition.
The limitations of simple loops for complex scheduling.
How to use the powerful schedule library for flexible task management.
The importance of the schedule.run_pending() loop.
A brief mention of external scheduling tools for production bots.

Now your bots can truly work for you, even when you're away!

Frequently asked questions

Is the “Scheduling Basic Tasks” lesson free?

Yes — the full text of “Scheduling Basic Tasks” is free to read here on the web, and the Web Scraping & Bots course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Web Scraping & Bots course, upgrade to CoddyKit PRO.

What will I learn in “Scheduling Basic Tasks”?

Implement simple scheduling mechanisms to run your bots at specific intervals for continuous automation. You practise Web Scraping & Bots with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Web Scraping & Bots?

No prior experience is required. Web Scraping & Bots on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Scheduling Basic Tasks” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Web Scraping & Bots lesson?

Yes. Every Web Scraping & Bots lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to Web Scraping & Bots