Go Academy · Lesson

Worker Pool Pattern

Building a fixed-size goroutine worker pool

Worker Pool Pattern is a free Go Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Go Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

What is a worker pool?

A worker pool pre-creates a fixed number of goroutines that process jobs from a shared queue. This bounds concurrency, preventing resource exhaustion from unlimited goroutine creation.

Basic implementation

Create a jobs channel and N goroutines that all read from it:

func workerPool(n int, jobs <-chan Job, results chan<- Result) {
    var wg sync.WaitGroup
    for i := 0; i < n; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for job := range jobs {
                results <- process(job)
            }
        }()
    }
    wg.Wait()
    close(results)
}

Sending jobs

Send all jobs to the channel then close it. Workers will process all jobs and exit their range loops when the channel closes.

jobs := make(chan Job, 100)
for _, j := range allJobs {
    jobs <- j
}
close(jobs) // signals workers to stop after draining

Collecting results

Read results concurrently with sending jobs. Use a goroutine for the pool so the result collection does not deadlock with job submission.

results := make(chan Result, 100)
go workerPool(5, jobs, results)
for r := range results {
    collect(r)
}

Tuning pool size

For CPU-bound work, set pool size to runtime.NumCPU(). For I/O-bound work, a higher multiplier (e.g. 10×) utilises waiting time. Profile to find the sweet spot.

n := runtime.NumCPU()
if ioHeavy { n = runtime.NumCPU() * 10 }

Adding context cancellation

Pass a context to workers so they stop early on cancellation:

for {
    select {
    case job, ok := <-jobs:
        if !ok { return }
        results <- process(ctx, job)
    case <-ctx.Done():
        return
    }
}

Semaphore alternative

Instead of a fixed goroutine pool, use a buffered channel as a semaphore to limit the number of concurrently-running goroutines while still spawning one per job.

sem := make(chan struct{}, 10)
for _, job := range jobs {
    sem <- struct{}{}
    go func(j Job) {
        defer func() { <-sem }()
        process(j)
    }(job)
}

Error handling

Return errors alongside results using a result struct. Cancel the context on the first error, or collect all errors for a final report.

type Result struct{ Val int; Err error }

Dynamic pool sizing

For adaptive pools, use golang.org/x/sync/semaphore which allows acquiring and releasing a weighted resource, supporting dynamic concurrency limits.

Worker pool vs errgroup

golang.org/x/sync/errgroup provides a simpler API for launching N goroutines and collecting the first error. Use a pool when you need a persistent set of reusable workers.

Graceful shutdown

Close the jobs channel to signal workers. After the WaitGroup is done, close results. Any pending job submissions must happen before close(jobs).

Quick Check

How does a worker pool signal workers to stop processing?

Recap: Worker Pool

Key points:

Fixed N goroutines read from one jobs channel
Close jobs channel to signal workers to finish
WaitGroup tracks completion; close results when done
CPU-bound: pool size ≈ NumCPU; I/O-bound: larger

Frequently asked questions

Is the “Worker Pool Pattern” lesson free?

Yes — the full text of “Worker Pool Pattern” is free to read here on the web, and the Go Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Go Academy course, upgrade to CoddyKit PRO.

What will I learn in “Worker Pool Pattern”?

Building a fixed-size goroutine worker pool You practise Go Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Go Academy?

No prior experience is required. Go Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Worker Pool Pattern” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Go Academy lesson?

Yes. Every Go Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to Go Academy