Worker Pool Pattern
Building a fixed-size goroutine worker pool
What is a worker pool?
A worker pool pre-creates a fixed number of goroutines that process jobs from a shared queue. This bounds concurrency, preventing resource exhaustion from unlimited goroutine creation.
Basic implementation
Create a jobs channel and N goroutines that all read from it:
func workerPool(n int, jobs <-chan Job, results chan<- Result) {
var wg sync.WaitGroup
for i := 0; i < n; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for job := range jobs {
results <- process(job)
}
}()
}
wg.Wait()
close(results)
}All lessons in this course
- The select Statement
- Timeouts and Tickers
- Fan-in and Fan-out Patterns
- Worker Pool Pattern