7  Data import: reading CSV and flat files

Objectives

  • Read your own data into R. Up to now we have worked with datasets that come bundled with packages. In practice you will often need to import data from your own files. This session introduces the readr functions for reading plain text rectangular data (CSV, TSV and other delimited formats). You will learn how to specify file paths and handle column names, types and missing values.
  • Control column names, missing values and types. When you run read_csv() it prints the number of rows and columns read and a summary of the column specification. You will learn how to rename variables, skip header lines, define which strings should be treated as missing, and override the type guessing heuristic that readr uses.
  • Read multiple files and combine them. Real projects often involve multiple data files (for example, one file per month). You can pass a vector of file paths to read_csv() and use the id argument to record the source of each observation. We will practice finding files with list.files(), reading them into a single tibble and combining them using bind_rows().
  • Choose the right format for saving intermediate results. Compare write_csv(), write_rds(), and Parquet files, and know which one actually preserves your column types.
  • Recognize a handful of import traps. A file can read without any error at all and still hand you the wrong values, because readr made a completely reasonable guess that happened not to match your data.

Notes

library(tidyverse)

Why import data?

Working with data included in R packages is convenient when you are learning, but eventually you need to apply the tools to your own data. R for Data Science focuses this chapter on reading plain text rectangular files and gives practical advice for handling column names, types, and missing data. The goal is always the same: get your data into a tidy tibble so you can immediately start transforming and visualizing it.

Reading CSV files with readr

The readr package is part of the tidyverse and provides fast functions for reading delimited files. The most common function is read_csv(), which expects a path to a comma separated file. When you read a file, readr prints a message showing the number of rows and columns, the delimiter used, and the column specification, including the guessed type for each column. That message can be silenced with show_col_types = FALSE.

students <- read_csv("https://pos.it/r4ds-students-csv")
students
# A tibble: 6 × 5
  `Student ID` `Full Name`      favourite.food     mealPlan            AGE  
         <dbl> <chr>            <chr>              <chr>               <chr>
1            1 Sunil Huffmann   Strawberry yoghurt Lunch only          4    
2            2 Barclay Lynn     French fries       Lunch only          5    
3            3 Jayendra Lyne    N/A                Breakfast and lunch 7    
4            4 Leon Rossini     Anchovies          Lunch only          <NA> 
5            5 Chidiegwu Dunkel Pizza              Breakfast and lunch five 
6            6 Güvenç Attila    Ice cream          Lunch only          6

Use a local file path (as in data/students.csv) to read files stored in your own project rather than a URL. It is good practice to keep data in a dedicated data/ folder and refer to it with a relative path, which is exactly why every example elsewhere in this book reads from "data/...". If your file has a header row containing the column names, as most CSVs do, readr uses it automatically. If not, set col_names = FALSE, or supply your own names with col_names = c("...", ...).

Extra reading options: skipping lines and other delimiters

Real files are not always clean CSVs with a header on line one. skip = n drops the first n lines before reading (handy when a file starts with a title or metadata row), and comment = "#" drops any line starting with that character.

messy_csv <- "This file was exported on 2026-01-01\n# do not edit below this line\nx,y\n1,2\n3,4"
read_csv(messy_csv, skip = 1, comment = "#", show_col_types = FALSE)
# A tibble: 2 × 2
      x     y
  <dbl> <dbl>
1     1     2
2     3     4

readr also provides close relatives of read_csv() for other plain text formats: read_csv2() for semicolon delimited files (common in regions where a comma is the decimal separator; see Example 6.2), read_tsv() for tab delimited files, read_delim() when you need to specify an arbitrary delimiter yourself, and read_fwf() for fixed width files where columns are defined by character position rather than a delimiter at all.

Handling missing values and non-syntactic names

A CSV file does not encode missing values explicitly, so readr treats empty fields as NA. In practice, missing values are often recorded with sentinel strings such as "N/A" or "." instead. You can tell read_csv() which strings should be considered missing with the na argument. In the students data above, the string "N/A" marks a missing food preference:

students2 <- read_csv(
  "https://pos.it/r4ds-students-csv",
  na = c("N/A", ""),
  show_col_types = FALSE
)
students2
# A tibble: 6 × 5
  `Student ID` `Full Name`      favourite.food     mealPlan            AGE  
         <dbl> <chr>            <chr>              <chr>               <chr>
1            1 Sunil Huffmann   Strawberry yoghurt Lunch only          4    
2            2 Barclay Lynn     French fries       Lunch only          5    
3            3 Jayendra Lyne    <NA>               Breakfast and lunch 7    
4            4 Leon Rossini     Anchovies          Lunch only          <NA> 
5            5 Chidiegwu Dunkel Pizza              Breakfast and lunch five 
6            6 Güvenç Attila    Ice cream          Lunch only          6

Sometimes column names contain spaces or other characters that make them non-syntactic in R, like Full Name above. In that case they are surrounded by backticks in the tibble, and you must use backticks (`Full Name`) to refer to them in code. A simple fix is to rename columns after import with rename(), or with the janitor::clean_names() helper, which converts every name to snake_case in one call.

How readr guesses column types

Because a CSV file carries no type information, readr guesses the type of each column by sampling up to 1,000 values spread across the file and checking, in order: does the column contain only logical values? Only numbers? Does it match the ISO 8601 date or date-time format? If none of those succeed, readr falls back to treating the column as a plain character string. This heuristic works well for clean data but can fail when a column contains an unexpected value, such as a period used to mean “missing.”

simple_csv <- "x\n10\n.\n20\n30"

# the "." makes readr's default guess treat the whole column as a string
df_default <- read_csv(simple_csv, show_col_types = FALSE)
df_default
# A tibble: 4 × 1
  x    
  <chr>
1 10   
2 .    
3 20   
4 30   
# force the column to be numeric, and see exactly where that assumption breaks
df_num <- read_csv(simple_csv, col_types = list(x = col_double()), show_col_types = FALSE)
problems(df_num)
# A tibble: 1 × 5
    row   col expected actual file                                              
  <int> <int> <chr>    <chr>  <chr>                                             
1     3     1 a double .      C:/Users/Joshua_Patrick/AppData/Local/Temp/RtmpeG…
# or tell readr up front that "." means NA, and the default guess succeeds
df_fixed <- read_csv(simple_csv, na = ".", show_col_types = FALSE)
df_fixed
# A tibble: 4 × 1
      x
  <dbl>
1    10
2    NA
3    20
4    30

When type guessing fails, you provide your own column specification through col_types, either a named list or a cols() call, pairing each column name with a type function: col_logical(), col_double(), col_integer(), col_character(), col_factor(), col_date(), col_datetime(), col_number() (a permissive numeric parser that strips things like currency symbols and commas), or col_skip() to drop a column entirely. cols(.default = col_character()) sets a fallback type for every column you don’t name explicitly, and cols_only() reads only the columns you list.

parse_number() is the standalone version of the logic behind col_number(): hand it a messy character vector and it extracts just the numeric part.

parse_number(c("$1,234.56", "45%", "12 dollars"))
[1] 1234.56   45.00   12.00

Reading multiple files and combining them

In many projects you receive data split across multiple files, perhaps one file per month or per site. Instead of reading each file separately and binding the results yourself, you can pass a vector of file paths straight to read_csv(). It reads all of them and stacks the rows together, and the optional id argument adds a column recording which file each row came from.

sales_files <- c("data/01-sales.csv", "data/02-sales.csv", "data/03-sales.csv")
sales <- read_csv(sales_files, id = "file")

You often don’t know every file name ahead of time. list.files() finds files matching a pattern (for example, "sales\\.csv$"), and full.names = TRUE returns full paths ready to hand straight to read_csv().

sales_files <- list.files("data", pattern = "sales\\.csv$", full.names = TRUE)
sales <- read_csv(sales_files, id = "file")

If you ever need to combine tibbles you already read in separately, dplyr::bind_rows() stacks them by matching column name (the columns don’t even need to be in the same order), and bind_cols() glues data frames side by side by position.

Entering data by hand

Not every small dataset needs its own file. tibble() builds a data frame column by column, while tribble() (a “transposed tibble”) lets you lay data out row by row, which is often easier to read and check for a small, hand-typed table. A ~ marks each column name.

tribble(
  ~grade, ~gpa_cutoff,
  "A",    3.7,
  "B",    2.7,
  "C",    1.7
)
# A tibble: 3 × 2
  grade gpa_cutoff
  <chr>      <dbl>
1 A            3.7
2 B            2.7
3 C            1.7

Writing data back to disk

readr also provides write_csv() and write_tsv() for saving a tibble as plain text. Remember that a CSV file has no way to record column types, so anyone (including you, later) who reads it back has to let readr re-guess every type from scratch (see Example 6.4). For intermediate results you plan to read back into R, write_rds() and read_rds() store the object in a binary format that preserves types exactly.

A third option, Parquet, is a binary format built for exactly this kind of interim storage, and it works across R, Python, and other tools, not just R. This project already ships with starwars.parquet at the top level; read it with arrow::read_parquet():

starwars_data <- arrow::read_parquet("starwars.parquet")
dim(starwars_data)
[1] 87 14

Fringe cases and common pitfalls

ExampleExample 6.1

readr protects leading zeros by default, but forcing a numeric type throws that protection away.

A ZIP code like 02138 has a meaningful leading zero. Read plainly, readr is actually careful about this:

zip_csv <- "zip\n02138\n02139\n94103"
read_csv(zip_csv, show_col_types = FALSE)
# A tibble: 3 × 1
  zip  
  <chr>
1 02138
2 02139
3 94103

Notice zip was guessed as a character column, leading zeros intact, precisely because a genuine number can never start with a zero. The trouble starts if you “fix” what looks like an annoying character column by forcing it to be numeric yourself:

read_csv(zip_csv, col_types = cols(zip = col_double()), show_col_types = FALSE)
# A tibble: 3 × 1
    zip
  <dbl>
1  2138
2  2139
3 94103

02138 silently becomes 2138, an entirely different (and very much real) ZIP code, with no warning at all, because 2138 is a perfectly valid double. Before overriding a guessed type, ask whether the column is really a number you would ever do arithmetic on, or an identifier that just happens to look numeric (ZIP codes, phone numbers, student IDs); identifiers almost always belong in col_character().

ExampleExample 6.2

A semicolon delimited file, read with the wrong function, produces one column of garbage instead of an error.

Some regions use a comma as the decimal separator, which means a comma can no longer double as the column delimiter in a CSV, so spreadsheet software in those regions exports semicolon delimited files instead, often still with a .csv extension.

euro_csv <- "price;qty\n1234,56;2\n2000,00;5"

# read_csv() assumes a comma delimiter, so it finds only one column per line
read_csv(euro_csv, show_col_types = FALSE)
# A tibble: 2 × 1
  `price;qty`
  <chr>      
1 1234,56;2  
2 2000,00;5

Nothing errors. You simply get a single garbled column named `price;qty` containing whole unparsed lines like "1234,56;2". The fix is read_csv2(), readr’s variant built for exactly this convention: semicolon as the delimiter, comma as the decimal mark.

read_csv2(euro_csv, show_col_types = FALSE)
# A tibble: 2 × 2
  price   qty
  <dbl> <dbl>
1 1235.     2
2 2000      5

If a .csv file loads as a single suspicious looking column, check for semicolons before assuming the file itself is broken.

ExampleExample 6.3

parse_number() can rescue a column that looks unusably messy.

Suppose a spreadsheet export hands you prices as formatted text rather than plain numbers:

raw_prices <- c("$1,234.56", "$45.00", "$1,200")
parse_number(raw_prices)
[1] 1234.56   45.00 1200.00

parse_number() strips the currency symbol, the thousands separator, and any other non-numeric text around the number, keeping just the numeric value as a double. It is far less error prone than trying to write your own gsub() calls to strip out $ and , by hand, and it is worth reaching for any time a supposedly numeric column was guessed as character because of formatting like this.

ExampleExample 6.4

Writing to CSV and reading it back can silently drop your factor levels.

original <- tibble(
  id = 1:3,
  grade = factor(c("A", "B", "A"), levels = c("A", "B", "C")),
  test_date = as.Date(c("2026-01-15", "2026-01-16", "2026-01-17"))
)
sapply(original, class)
       id     grade test_date 
"integer"  "factor"    "Date" 
csv_path <- tempfile(fileext = ".csv")
write_csv(original, csv_path)
back_from_csv <- read_csv(csv_path, show_col_types = FALSE)
sapply(back_from_csv, class)
         id       grade   test_date 
  "numeric" "character"      "Date"

The date column happens to survive, because it was written in a format readr’s guesser recognizes as ISO 8601, but grade comes back as a plain character column: the fact that it was a factor, and the specific order "A", "B", "C" (including the unused level "C"), is gone for good, because a CSV file has no way to record that information in the first place. Compare that to a round trip through RDS:

rds_path <- tempfile(fileext = ".rds")
write_rds(original, rds_path)
back_from_rds <- read_rds(rds_path)
identical(original, back_from_rds)
[1] TRUE

If you need the exact same R object back, including factor levels, dates, and everything else, save it with write_rds() rather than write_csv(). Save to CSV only when the destination is meant to be read by something other than R, or when you specifically want a plain text, human-readable file.

Recap

Term Definition
read_csv() Reads a comma delimited plain text file into a tibble, guessing each column’s type.
show_col_types = FALSE Silences the column specification message read_csv() prints by default.
na argument Tells read_csv() which strings (besides an empty field) should become NA.
col_types / cols() Overrides readr’s guessed type for one or more columns.
col_character() for identifiers The safe type for numeric-looking IDs (ZIP codes, phone numbers) that should never be treated as numbers.
read_csv2() Reads a semicolon delimited file where a comma is used as the decimal mark.
parse_number() Extracts the numeric value out of a messy formatted string, discarding currency symbols and separators.
list.files() + id Finds files matching a pattern and, when passed to read_csv() with id, records which file each row came from.
tribble() Builds a small tibble by typing it out row by row instead of column by column.
write_rds() / read_rds() Save and reload an R object in a binary format that preserves types exactly, unlike CSV.
Parquet A binary, cross-language file format for storing tabular data with types intact; read with arrow::read_parquet().

Check your understanding

NoteProblems
  1. Why does read_csv() print a message about column types by default, and how do you turn it off?
  2. A column of product codes like 00452 gets read in as a character column by default. A classmate “fixes” this by adding col_types = cols(product_code = col_double()). What happens to the data, and why is it a bad idea here?
  3. A .csv file downloaded from a European website loads into a single messy column instead of several clean ones. What is the most likely cause, and which function should you try instead of read_csv()?
  4. Explain what parse_number() does with the input "€2.500,00" versus what read_csv() alone would guess for a whole column of values formatted that way. (You do not need to know the exact output; explain the general behavior.)
  5. Your analysis produces a tibble with a factor column and a Date column. You need to hand the exact same R object to a groupmate to continue the analysis tomorrow. Should you save it with write_csv() or write_rds()? Justify your answer.

  1. readr cannot know a column’s type just from a CSV file, since plain text carries no type information, so it prints its best guess for each column so you can catch a wrong guess early rather than discovering it much later in an analysis. Add show_col_types = FALSE to suppress the message once you trust the guesses.

  2. Forcing product_code to col_double() converts "00452" to the number 452, silently dropping the leading zeros, because 452 is a perfectly valid double and readr has no way to know the leading zeros were meaningful. This is a bad idea whenever the “number” is really an identifier (product codes, ZIP codes, phone numbers) that will never be used in arithmetic; col_character() is the safer choice.

  3. The file most likely uses a semicolon as its delimiter (common where a comma is the decimal separator instead), which read_csv() does not expect, since it always assumes a comma delimiter. read_csv2() is built for exactly this format: semicolon delimited, comma as the decimal mark.

  4. parse_number() strips out the currency symbol and separators and returns just the numeric value as a double, handling formatting like this directly. Left to guess on its own, read_csv() would very likely treat a whole column formatted like "€2.500,00" as a character column, since values like that do not match its numeric-guessing heuristic at all.

  5. write_rds(), because it preserves the tibble exactly as an R object, including the factor’s levels and order and the Date column’s type. write_csv() would flatten the factor down to plain character text and lose the level information entirely, and would require your groupmate to correctly re-specify every column’s type by hand to get back to where you started.