Designing for Crash-First
Embrace the 'crash-first' principle to design self-healing systems, where failures are expected and handled by supervisors.
Designing for Crash-First is a free Erlang OTP: Distributed & Fault-Tolerant Systems Programming lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Erlang OTP: Distributed & Fault-Tolerant Systems Programming learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Embrace the Crash-First Philosophy
In Erlang, we don't just handle errors; we embrace them! This is the "crash-first" principle.
Instead of trying to prevent every possible error with complex checks, Erlang systems are designed to let processes crash when something unexpected happens.
The system then relies on another component, the supervisor, to detect the crash and restart the failed process, ensuring continuous operation.
This approach leads to more robust, self-healing applications.
Defensive vs. Crash-First
Many programming paradigms emphasize "defensive programming":
- Extensive input validation.
- Complex error codes and handling logic.
- Trying to recover *within* the failing function.
Crash-first flips this: If a process encounters an unrecoverable error, it should just crash. Let a higher-level entity (the supervisor) deal with the recovery.
Supervisors Make it Work
The 'crash-first' strategy wouldn't work without supervisors. Supervisors are special Erlang processes designed to monitor other processes (their children).
When a child process crashes, the supervisor detects its termination and acts according to a predefined restart strategy.
This allows the faulty process to be restarted in a known, clean state, without affecting the rest of the system.
Isolation is Key
Erlang's lightweight processes are isolated from each other. This isolation is crucial for crash-first.
If one process crashes, it doesn't directly bring down other processes. Each process has its own memory and execution context.
This means a supervisor can restart a faulty process without fear of corrupting the state of its siblings or the entire application.
A Process That Crashes
Let's see a simple Erlang process that will intentionally crash. We'll use division by zero, a common way to trigger an error.
Notice how start/0 spawns a new process that tries to perform the faulty operation.
When you run this, you'll see an error message, but the Erlang VM itself won't crash.
-module(crashy_process_example).
-export([start/0, crash_me/0]).
crash_me() ->
io:format("Crashy process ~p starting...~n", [self()]),
timer:sleep(500), % Give it a moment
Result = 10 / 0, % This will cause a badarith error
io:format("This line will not be reached: ~p~n", [Result]).
start() ->
spawn(fun crash_me/0).
% To run:
% 1. Compile: c(crashy_process_example).
% 2. Start: crashy_process_example:start().Supervisor Restarts a Crash
Now, let's wrap our crashing logic with a simple supervisor. The supervisor will detect its child's crash and restart it.
We define a child spec for our crashing function (now defined directly within the supervisor module) and use the one_for_one strategy.
Run the code. Observe how the process crashes and is restarted automatically by the supervisor! You'll see "Crashy process X starting..." multiple times.
-module(my_supervisor_example).
-behaviour(supervisor).
-export([start_link/0, init/1, crash_me/0]). % Export crash_me for child_spec
crash_me() ->
io:format("Crashy process ~p starting...~n", [self()]),
timer:sleep(1000), % Give it a second
Result = 10 / 0, % This will cause a badarith error
io:format("This line will not be reached: ~p~n", [Result]).
start_link() ->
supervisor:start_link({local, ?MODULE}, ?MODULE, []).
init([]) ->
% Child spec for our crashing function
CrashyChild = {crashy_child_id, % Unique ID for the child
{?MODULE, crash_me, []}, % {Module, Function, Args}
permanent, % Restart if it terminates
5000, % Max restart intensity
worker, % Type of process
[?MODULE]}, % List of modules it depends on (used for code loading)
Children = [CrashyChild],
% Restart strategy: one_for_one means only the crashing child restarts
Strategy = {one_for_one, 1, 5}, % Max 1 restart in 5 seconds
{ok, {Strategy, Children}}.
% To run:
% 1. Compile: c(my_supervisor_example).
% 2. Start: my_supervisor_example:start_link().Design with Failure in Mind
Embracing crash-first means a shift in how you design your applications.
- Identify Failure Domains: Group related processes under supervisors. If one fails, only that group is affected.
- Idempotent Operations: Design processes so that restarting them or re-executing an operation doesn't cause negative side effects.
- External State: Minimize mutable state held within a process that would be lost on a crash. Use persistent storage (like ETS or Mnesia) for critical data.
Managing State in Crash-First
When a process crashes and restarts, its internal state is lost. This is by design, providing a clean slate.
For processes that manage important state, you need a strategy:
- Initialize from Source: On restart, fetch the necessary state from a reliable source (database, configuration file, another persistent process).
- Externalize State: Store critical, shared state in ETS tables, Mnesia, or a database, rather than solely within a process's heap.
This ensures that even after a crash, the process can resume its duties with correct information.
Why Crash-First is Powerful
Adopting the crash-first principle offers significant advantages:
- Increased Reliability: Systems automatically recover from transient errors.
- Simpler Code: Less need for complex, defensive error-handling logic within each function.
- Fault Tolerance: The application continues to operate even if parts fail.
- Easier Debugging: Crashes clearly indicate unexpected states, rather than masking issues with complex recovery attempts.
Quick Check: Crash-First
Which statements accurately describe the "crash-first" principle in Erlang and its benefits?
Recap: Designing for Crash-First
We've explored the powerful "crash-first" philosophy in Erlang:
- It's about letting processes crash on unrecoverable errors.
- Supervisors are key, monitoring and restarting crashed processes.
- Erlang's process isolation ensures local crashes don't bring down the whole system.
- Designing for crash-first involves thinking about failure domains, idempotent operations, and externalizing critical state.
This approach simplifies code, increases reliability, and builds truly fault-tolerant systems.
Frequently asked questions
Is the “Designing for Crash-First” lesson free?
Yes — the full text of “Designing for Crash-First” is free to read here on the web, and the Erlang OTP: Distributed & Fault-Tolerant Systems Programming course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Erlang OTP: Distributed & Fault-Tolerant Systems Programming course, upgrade to CoddyKit PRO.
What will I learn in “Designing for Crash-First”?
Embrace the 'crash-first' principle to design self-healing systems, where failures are expected and handled by supervisors. You practise Erlang OTP: Distributed & Fault-Tolerant Systems Programming with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Erlang OTP: Distributed & Fault-Tolerant Systems Programming?
No prior experience is required. Erlang OTP: Distributed & Fault-Tolerant Systems Programming on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Designing for Crash-First” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Erlang OTP: Distributed & Fault-Tolerant Systems Programming lesson?
Yes. Every Erlang OTP: Distributed & Fault-Tolerant Systems Programming lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Links and Monitors Explained
- Robust Error Handling
- Designing for Crash-First
- The Let-It-Crash Philosophy