awk Functions, printf Formatting, and Report Generation

As your awk programs grow from one-liners to multi-step pipelines, two features become essential: user-defined functions and printf formatting.

Functions let you encapsulate reusable logic — a percentage calculator, a string trimmer, a unit converter — so you write it once and call it anywhere in the same program. printf lets you control exactly how output looks: column widths, decimal places, padding, and alignment.

Together they transform raw log lines into polished, readable reports that engineers and managers can act on.

• Functions reduce duplication and make programs testable in isolation.
• printf aligns data into fixed-width columns so reports stay legible even with varying input lengths.
• Both features work in POSIX awk, gawk, and mawk — no extensions needed.

A user function is declared with the function keyword, before or after any pattern-action rules. The syntax is:

function name(param1, param2,    local1, local2) {
    # body
    return value
}

A key awk idiom: local variables are simply extra parameters with extra whitespace before them. There is no local keyword — the convention of extra spaces signals that those parameters are locals, not arguments the caller passes.

• All variables not declared as parameters are global by default.
• Functions can call themselves recursively.
• return is optional; without it the function returns an empty string.

Functions are especially useful for repeated string operations. Consider trimming leading and trailing whitespace — something raw log data often needs before comparison or reporting.

The example below defines trim(s) using gsub and uses it in every record, keeping the main rule clean and readable.

• gsub(regex, replacement, target) modifies target in place and returns the count of substitutions.
• Putting the regex logic inside a function means the rule block stays focused on business logic.
• You can unit-test by passing hand-crafted strings directly in a BEGIN block.

printf in awk follows the same conventions as C and Bash's built-in printf. The key difference from print is that printf never adds a newline automatically — you must add \n yourself.

Common format specifiers:

• %s — string
• %d — integer
• %f — floating-point
• %e — scientific notation
• %g — shorter of %f/%e

Width and precision are controlled by placing numbers between % and the specifier: %-20s left-aligns in a 20-character field; %8.2f gives an 8-wide float with 2 decimal places.

A polished report needs a header row and a separator line. The BEGIN block is the natural place to emit these — it runs once before any input is processed.

The trick is to match the widths in the header exactly to the widths used in the data rows. Defining width constants in BEGIN (or in a function) keeps everything in sync when you later adjust column widths.

• Use printf with a separator string of dashes to draw the divider line.
• Always end header lines with \n.
• Combine a BEGIN header, per-record rows, and an END footer for a complete report structure.

Once you are building multi-column reports, it is common to need the same formatting call in several places — for example, rendering a percentage bar or converting bytes to a human-readable unit. Putting this inside a function avoids duplication and makes the format easy to change globally.

The function below converts bytes into KB, MB, or GB and returns a formatted string. The calling rule simply passes $NF (the last field) and prints whatever comes back.

Real reports often need totals, averages, or rankings — values you cannot compute until you have seen all the input. The pattern is:

• Accumulate data into associative arrays during the per-record phase.
• Compute derived values (averages, percentages) in END.
• Emit formatted output only from END, so widths can adapt to the full dataset.

This deferred-output pattern is one of the most powerful idioms in awk report generation. The example below tallies request counts and response bytes per HTTP status code from an access log.

awk supports recursion. A classic use-case is computing factorials or generating repetitive strings (like a line of dashes sized to the terminal width). Recursion is less common in awk than in general-purpose languages, but it works correctly and is available in all major implementations.

The example below defines a recursive repeat(s, n) function and uses it to draw separator lines that automatically match the width of the widest data row — no hardcoded dashes needed.

• Avoid deep recursion on large inputs — awk has no tail-call optimisation.
• For simple repetition, sprintf("%*s", n, "") with gsub(/ /, "-") is faster but less illustrative.

awk's printf (and print) can redirect output to files or pipe into shell commands — directly from within the awk program, without wrapping the whole thing in a subshell.

• printf "..." > "file.txt" — writes to a file (truncates once, then appends within the same run).
• printf "..." >> "file.txt" — appends to a file.
• printf "..." | "sort -rn" — pipes into a shell command; awk keeps the pipe open across records and closes it at program end.

A common pattern is to split a single pass over a log into multiple per-status or per-host output files, avoiding repeated scans of a large file.

Not all reports are for humans. Sometimes the output must be machine-readable CSV that feeds a spreadsheet or a downstream pipeline. awk's printf handles this cleanly: set OFS to a comma, or control every delimiter explicitly.

Two important rules for robust CSV from awk:

• Wrap fields that might contain commas or newlines in double quotes.
• Escape any literal double quotes inside a field by doubling them: He said ""hello"".

The function csv_field(s) below encapsulates this quoting logic so it can be applied consistently to every string field.

This final example ties everything together: user functions, printf formatting, array accumulation, and a structured header/footer. The input is a simplified web access log; the output is a human-readable summary table with per-method totals and a grand-total line.

Key techniques used:

• count[method]++ and bytes[method] += size accumulate per-method stats in a single pass.
• human_bytes() (from an earlier scene) converts byte totals.
• printf with matched widths in header, data, and footer ensures the table stays aligned regardless of input size.
• The END block iterates the array with for (k in arr) and emits sorted output via a pipe to sort.

Which of the following correctly declares result as a local variable inside an awk function, following the POSIX convention?

In this lesson you learned how to write production-quality awk programs that generate formatted reports from raw data streams. Here is what to carry forward:

• User functions are declared with function name(params, locals). Extra whitespace before the local-variable parameters is the POSIX convention — there is no local keyword.
• printf gives precise control over output: width (%10s), alignment (%-10s), and precision (%8.2f). Remember to add \n explicitly.
• Use BEGIN to emit headers and separators, per-record rules to accumulate data, and END to compute totals and print the finished report.
• Redirect printf output to files (> file) or pipelines (| "sort") from within awk to split reports or post-process output.
• Wrap string fields in a csv_field() helper when emitting CSV to handle embedded commas and quotes safely.
• Functions that convert units (human_bytes), repeat characters (repeat), or sanitise strings (trim, csv_field) are worth keeping in a personal awk library — they compose cleanly across projects.