0Pricing
R Academy · Lesson

Parsing TSV and Fixed-Width Files

Use read_tsv() and read_fwf() for tab-separated and fixed-width data.

Parsing TSV and Fixed-Width Files is a free R Academy lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the R Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Beyond CSV: Other File Formats

Not all tabular data comes as CSV. Tab-separated files (TSV), pipe-delimited files, and fixed-width format (FWF) files are common in government data, legacy systems, and scientific databases. The readr package handles all of these with consistent syntax.

library(readr)

# readr's flat file readers:
# read_csv()    - comma-separated
# read_csv2()   - semicolon-separated (European)
# read_tsv()    - tab-separated
# read_delim()  - any delimiter
# read_fwf()    - fixed-width format
# read_lines()  - raw text, one line per element

cat('All readr functions return tibbles with the same interface!')

read_tsv() — Tab-Separated Values

read_tsv() reads tab-delimited files. It has the same arguments as read_csv() — col_types, na, skip, etc. TSV is preferred over CSV when data contains commas (addresses, descriptions).

library(readr)

# Simulated TSV using \t as delimiter
tsv_data <- 'name\tscore\tcity
Alice Smith\t85\tNew York, NY
Bob Jones\t92\tLos Angeles, CA
Carol White\t78\tChicago, IL'

# Note: cities contain commas — TSV handles this cleanly
df <- read_tsv(tsv_data, show_col_types=FALSE)
print(df)

read_delim() — Any Delimiter

read_delim(file, delim='|') handles any single-character delimiter. Pipe-delimited files (|) are common in data warehouses and government exports because pipes rarely appear in the data itself.

library(readr)

# Pipe-delimited data
pipe_data <- 'ID|Name|Department|Salary
1|Alice Smith|Engineering|95000
2|Bob Jones|Marketing|72000
3|Carol White|HR|65000'

df <- read_delim(pipe_data, delim='|', show_col_types=FALSE)
print(df)

read_delim() for Semi-Colon Files

European CSV files often use semicolons as delimiters and commas as decimal marks. read_csv2() is a shortcut for read_delim(delim=';', locale=locale(decimal_mark=',')), or you can configure read_delim() directly.

library(readr)

# European-style: semicolon delimiter, comma decimal
european_data <- 'name;price;quantity
Widget;9,99;100
Gadget;24,50;50
Sprocket;4,75;200'

# Method 1: read_csv2() shortcut
df1 <- read_csv2(european_data, show_col_types=FALSE)
print(df1)

# Method 2: explicit read_delim
# read_delim(data, delim=';', locale=locale(decimal_mark=','))

read_fwf() — Fixed-Width Format

Fixed-width files have no delimiters — columns are identified by their character positions. Common in legacy mainframe exports and government data. Use fwf_widths() to specify column widths, or fwf_cols() for start/end positions.

library(readr)

# Fixed-width data (positions matter!)
fwf_data <- 'Alice    085NYC      
Bob      092LA       
Carol    078Chicago  '

# Column widths: name=9, score=3, city=10
df <- read_fwf(
  fwf_data,
  col_positions = fwf_widths(
    widths = c(9, 3, 10),
    col_names = c('name','score','city')
  ),
  show_col_types = FALSE
)

print(df)

fwf_positions() for Start/End Columns

fwf_positions(start, end, col_names) specifies exact character start and end positions (1-indexed). This is precise when column widths vary or when you need to skip some columns by not specifying them.

library(readr)

# Another fixed-width example using start/end positions
fwf_data <- '001ALICE   02024-01-15
002BOB     01   2024-02-20
003CAROL   03   2024-03-10'

df <- read_fwf(
  fwf_data,
  col_positions = fwf_positions(
    start = c(1, 4, 11, 14),
    end   = c(3, 10, 12, 23),
    col_names = c('id','name','score','date')
  ),
  show_col_types = FALSE
)

print(df)

fwf_empty() — Detect Width Automatically

fwf_empty(file, col_names) attempts to detect column boundaries automatically based on whitespace gaps. Provide the column names separately. Works best for cleanly-formatted FWF files with consistent spacing.

library(readr)

# Well-spaced fixed-width data
fwf_data <- 'name     score  grade
Alice    85     B
Bob      92     A
Carol    78     C
Dave     88     B'

# Auto-detect column positions from whitespace
df <- read_fwf(
  fwf_data,
  col_positions = fwf_empty(fwf_data, col_names=c('name','score','grade')),
  show_col_types = FALSE
)

print(df)

read_lines() — Raw Text Parsing

read_lines(file) reads a text file one line at a time, returning a character vector. This is the lowest-level read function — use it when you need to pre-process lines before parsing (e.g., filtering comments, handling inconsistent formats).

library(readr)

# Read lines and filter manually
raw_text <- '# Comment line
# Another comment
name,score
Alice,85
Bob,92'

lines <- read_lines(raw_text)
print(lines)

# Filter out comment lines
clean_lines <- lines[!startsWith(lines, '#')]
df <- read_csv(paste(clean_lines, collapse='\n'), show_col_types=FALSE)
print(df)

read_lines() for Incremental Inspection

read_lines(file, n_max=10) previews just the first few lines of any text file before committing to a full read. This is useful for diagnosing encoding issues, finding the actual header row, or detecting the delimiter type.

library(readr)

# Preview first few lines before reading
sample_file <- 'METADATA: version 1.2
DATE: 2024-01-15
---
id,name,value
1,Alice,100
2,Bob,200'

# Inspect to find where the data starts
preview <- read_lines(sample_file, n_max=4)
print(preview)
# From this we know: skip=3, then read as CSV

Handling Encoding Issues

Non-ASCII characters (accents, non-Latin scripts) require specifying the correct encoding. Use locale(encoding='latin1') or locale(encoding='UTF-8') inside read_csv() or read_delim(). Default is UTF-8.

library(readr)

# Detect encoding of a file
# readr::guess_encoding('file.csv')  # Returns likely encodings

# Specify encoding explicitly
# df <- read_csv('data_latin1.csv',
#   locale = locale(encoding = 'latin1'))

# For UTF-8 with BOM (common in Windows exports):
# df <- read_csv('data_utf8bom.csv',
#   locale = locale(encoding = 'UTF-8-BOM'))

cat('Encoding matters for accented chars: \xc3\xa9 (UTF-8 for e-acute)\n')
print(guess_encoding(chartr('\xe9','e','caf\xe9')))

Choosing the Right Reader

Quick guide for choosing the correct readr function:

  • Commas → read_csv()
  • Semicolons + European decimals → read_csv2()
  • Tabs → read_tsv()
  • Other delimiters → read_delim(delim='|')
  • Fixed positions → read_fwf()
  • Raw inspection → read_lines()
library(readr)

# Quick comparison of delimiters
csv_data  <- 'a,b,c\n1,2,3'
tsv_data  <- 'a\tb\tc\n1\t2\t3'
pipe_data <- 'a|b|c\n1|2|3'

read_csv(csv_data, show_col_types=FALSE)
read_tsv(tsv_data, show_col_types=FALSE)
read_delim(pipe_data, delim='|', show_col_types=FALSE)

Quick Check

Which readr function is the best choice for reading a file where columns are defined by fixed character positions (no delimiters)?

Recap: TSV and Fixed-Width Files

Key takeaways for non-CSV flat file formats:

  • read_tsv() — tab-separated; best when data contains commas
  • read_delim(delim='|') — any single-character delimiter
  • read_csv2() — semicolon-separated with European decimal marks
  • read_fwf(fwf_widths(c(10,5,8))) — fixed-width by column widths
  • read_fwf(fwf_positions(start, end)) — fixed-width by exact positions
  • read_lines(file, n_max=10) — inspect raw lines before parsing
  • All functions share the same col_types, na, skip, locale() arguments
library(readr)

# Pipe-delimited with custom NA values
df <- read_delim(
  'ID|Name|Score|City
  1|Alice|85|NYC
  2|Bob|N/A|N/A
  3|Carol|78|Chicago',
  delim = '|',
  na = c('', 'NA', 'N/A'),
  show_col_types = FALSE
)

print(df)
print(colSums(is.na(df)))

Frequently asked questions

Is the “Parsing TSV and Fixed-Width Files” lesson free?

Yes — the full text of “Parsing TSV and Fixed-Width Files” is free to read here on the web, and the R Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the R Academy course, upgrade to CoddyKit PRO.

What will I learn in “Parsing TSV and Fixed-Width Files”?

Use read_tsv() and read_fwf() for tab-separated and fixed-width data. You practise R Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start R Academy?

No prior experience is required. R Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Parsing TSV and Fixed-Width Files” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this R Academy lesson?

Yes. Every R Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Reading CSV Files with read_csv()
  2. Parsing TSV and Fixed-Width Files
  3. Importing Excel Files with readxl
  4. Writing Data to Multiple Formats
← Back to R Academy