Concurrency & Parallelism

Ever notice how your computer can play music, download files, and browse the web all at the same time? This isn't magic; it's the power of multi-tasking!

In system design, we often need our applications to handle many operations efficiently. This is where the concepts of concurrency and parallelism become crucial.

Concurrency is about dealing with many things at once. Think of a chef juggling multiple cooking tasks in a single kitchen.

• The chef might chop vegetables, then stir a pot, then check an oven.
• They're not doing everything simultaneously, but they're making progress on several tasks by switching between them quickly.
• This gives the illusion of simultaneous execution.

In software, concurrency often means a single CPU core rapidly switches between different tasks or threads. This is called context switching.

• One task runs for a short period.
• The CPU saves its state and switches to another task.
• This happens so fast that users perceive tasks running "at the same time."

It improves responsiveness and allows a system to make progress on multiple operations.

Here's a simple Java example showing two "tasks" running concurrently. The main thread starts two new threads, and the operating system or JVM schedules them to run.

Notice how their output might interleave, showing that they are making progress without necessarily finishing one before starting the other.

Parallelism is about doing many things at once, literally simultaneously. Imagine having multiple chefs, each with their own kitchen, working on different dishes at the exact same time.

• Each chef (or CPU core) executes a task independently.
• This requires multiple processing units (like multiple cores in a CPU).
• It's about increasing throughput by truly executing multiple instructions at the same instant.

For true parallelism, your system needs multiple processing units. Modern CPUs have multiple cores, allowing multiple threads to run simultaneously.

If you run the previous Java example on a multi-core processor, the operating system might schedule Task A on one core and Task B on another, leading to actual simultaneous execution.

This is different from concurrency on a single core, which simulates simultaneous execution through rapid switching.

Let's clarify the key difference:

• Concurrency: Deals with many tasks at once, often by switching between them. (e.g., one CPU core handling multiple threads).
• Parallelism: Does many tasks at once, literally simultaneously. (e.g., multiple CPU cores each handling a thread).

A system can be concurrent without being parallel (single-core CPU). A parallel system is always concurrent (it's dealing with multiple tasks).

Both concurrency and parallelism are vital for high-performance systems:

• Improved Responsiveness: Concurrent systems can keep the user interface active while background tasks run.
• Higher Throughput: Parallel systems can process more requests or data in a given time, utilizing all available CPU power.
• Better Resource Utilization: They make efficient use of CPU cores, especially in servers handling many client connections.

While powerful, concurrency and parallelism introduce challenges:

• Race Conditions: When multiple threads access shared resources, the final outcome depends on their execution order, leading to unpredictable results.
• Deadlocks: Two or more threads get stuck waiting for each other to release resources, causing the system to halt.
• Complexity: Designing and debugging concurrent/parallel systems is harder due to non-deterministic behavior.

Careful synchronization and design patterns are needed to mitigate these issues.

Consider a web server running on a single-core CPU that handles multiple client requests by rapidly switching between them. Which of the following best describes this scenario?

We've explored concurrency, which is about managing multiple tasks by switching between them, and parallelism, which is about executing multiple tasks truly simultaneously using multiple processing units.

Both are fundamental for designing high-performance, responsive, and scalable backend systems, though they introduce complexities like race conditions and deadlocks that require careful handling.