10  Variable types II: strings and regular expressions

Objectives

  • Create and manipulate strings. Recall that strings are sequences of characters defined by quotes. In R you can use single ' or double " quotes to create strings; the R4DS text recommends double quotes by default, switching to single quotes only when the string itself contains double quotes, and raw strings when a string is otherwise full of backslashes. You’ll learn to escape special characters, measure length with str_length(), and extract or replace substrings with str_sub().
  • Combine and format strings. Concatenate strings with str_c() (stringr’s equivalent to paste0(), but stricter about missing values), and build templated strings with str_glue(). Learn to collapse a vector of strings into one with str_flatten().
  • Understand regular expressions. A regular expression (regex) is a concise language for describing patterns within strings. Learn the difference between literal characters and metacharacters, how quantifiers (?, *, +, {n,m}) control repetition, how character classes ([...]) and their shortcuts (\d, \s, \w) work, and how alternation (|) and anchors (^, $, \b) narrow a match.
  • Match, extract, and replace patterns. Use str_detect(), str_subset(), and str_which() to test or filter strings, str_count() to count matches, str_extract()/str_extract_all()/str_match() to pull out matches (including captured groups), and str_replace()/str_remove() (and their _all() variants) to edit strings in place.
  • Use capturing groups and literal matching. Recognize how parentheses create capturing groups you can reuse in a replacement, and know when to reach for fixed() instead of treating a search string as a regex at all.
  • Tidy multiple variables encoded in one string. Recognize when a single string stores several pieces of information and use stringr plus tidyr’s separate_wider_delim() to split it into proper columns.

Notes

library(tidyverse)

Creating strings: quotes, escapes, and raw strings

Strings can be created with single or double quotes. Double quotes are preferred by convention; use single quotes only when the string itself contains double quotes, to avoid escaping them. Special characters (backslash, newline, tab) require escaping with \\, \n, or \t.

quote_example <- "He said, 'R is great!'"
quote_example
[1] "He said, 'R is great!'"
path <- "C:\\Users\\Data\\mydata\n"
path
[1] "C:\\Users\\Data\\mydata\n"

When a string needs many literal backslashes, such as a Windows path or a regex pattern you don’t want to double-escape, a raw string written as r"(...)" treats everything inside as literal text, with no escaping at all:

raw_path <- r"(C:\Users\Data\mydata)"
raw_path
[1] "C:\\Users\\Data\\mydata"

The length of a string (its number of characters) is computed with str_length(), which counts human-visible characters rather than bytes, so it handles multi-byte characters like emoji sensibly:

str_length(c("", "abc", "😊"))
[1] 0 3 1

str_sub() extracts or replaces substrings by position. Positive indices count from the start and negative indices count from the end, and the function is vectorized:

fruit <- c("banana", "apple", "pear")
str_sub(fruit, 1, 3)      # first three letters
[1] "ban" "app" "pea"
str_sub(fruit, -3, -1)    # last three letters
[1] "ana" "ple" "ear"
str_sub(fruit, 1, 1) <- str_to_upper(str_sub(fruit, 1, 1))  # capitalize first letter
fruit
[1] "Banana" "Apple"  "Pear"

Combining and formatting strings

str_c() concatenates strings and returns a character vector; it behaves like paste0() but integrates more predictably with dplyr and, unlike paste0(), does not quietly turn a missing value into the text "NA" (see Example 9.2). A sep argument inserts a separator between pieces, and collapse further collapses the whole result into a single string.

species <- c("Adelie", "Chinstrap", "Gentoo")
n <- c(3, 2, 1)
str_c(n, species, sep = " ", collapse = "; ")
[1] "3 Adelie; 2 Chinstrap; 1 Gentoo"

Inside a mutate(), if any input to str_c() is NA, the whole result is NA; wrap a column in coalesce(column, "") first if you would rather treat a missing value as an empty string.

df <- tibble(first = c("John", NA, "Jenny"), last = c("Smith", "Nguyen", NA))
df |> mutate(full = str_c(coalesce(first, ""), coalesce(last, ""), sep = " "))
# A tibble: 3 × 3
  first last   full        
  <chr> <chr>  <chr>       
1 John  Smith  "John Smith"
2 <NA>  Nguyen " Nguyen"   
3 Jenny <NA>   "Jenny "

str_glue() builds templated strings, inserting the value of any R expression written inside {}:

df |> mutate(message = str_glue("Hello {first} {last}!"))
# A tibble: 3 × 3
  first last   message          
  <chr> <chr>  <glue>           
1 John  Smith  Hello John Smith!
2 <NA>  Nguyen Hello NA Nguyen! 
3 Jenny <NA>   Hello Jenny NA!

To combine many strings into one, str_flatten() collapses a character vector with a given delimiter, which reads more clearly than building the same string up manually:

str_flatten(c("a", "b", "c"), collapse = ", ")
[1] "a, b, c"

Regular expressions: pattern basics

A regular expression describes a set of strings rather than one specific string. Most letters and digits match themselves literally; a handful of characters are metacharacters with special meaning:

  • . matches any single character except a newline.
  • Quantifiers control repetition: ? means zero or one, * means zero or more, + means one or more, and {n,m} means between n and m repetitions (either bound can be omitted).
  • Character classes like [aeiou] match any one of the enclosed characters, and [^0-9] (a caret at the start) matches any character not in the class. \d, \s, and \w are shortcuts for a digit, whitespace, and a “word” character (letter, digit, or underscore), respectively.
  • Alternation | matches one of several alternatives, so apple|pear matches either word.
  • Anchors pin a match to a position rather than to any characters: ^ matches the start of a string, $ matches the end, and \b matches a word boundary, the (zero-width) transition between a word character and a non-word character (see Example 9.4).

Detecting, counting, and extracting patterns

str_detect() tests whether each string matches a pattern and returns a logical vector, which pairs naturally with filter(). str_subset() returns the matching strings themselves rather than a logical vector, and str_which() returns their positions. str_count() counts how many times a pattern matches within each string. str_extract() returns the first match (str_extract_all() returns every match, as a list), and str_replace()/str_remove() replace or delete the first match (str_replace_all()/str_remove_all() handle every match).

library(nycflights13)

set.seed(42)
flights_small <- flights |>
  select(carrier, flight) |>
  slice_sample(n = 5) |>
  mutate(flight_id = str_c(carrier, flight, sep = "-"))

flights_small |> mutate(is_three_digit = str_detect(flight_id, "-\\d{3}$"))
# A tibble: 5 × 4
  carrier flight flight_id is_three_digit
  <chr>    <int> <chr>     <lgl>         
1 WN        1716 WN-1716   FALSE         
2 AA         178 AA-178    TRUE          
3 DL        1585 DL-1585   FALSE         
4 WN        3494 WN-3494   FALSE         
5 DL        2231 DL-2231   FALSE         
flights_small |> mutate(number_only = str_extract(flight_id, "\\d+"))
# A tibble: 5 × 4
  carrier flight flight_id number_only
  <chr>    <int> <chr>     <chr>      
1 WN        1716 WN-1716   1716       
2 AA         178 AA-178    178        
3 DL        1585 DL-1585   1585       
4 WN        3494 WN-3494   3494       
5 DL        2231 DL-2231   2231

\d matches any digit, so \d+ extracts one or more consecutive digits; the backslash itself has to be doubled (\\d) so that R’s own string-escaping rules pass a single backslash through to the regex engine.

messy <- c("(202) 555-0198", "+1-303-555-1212")
str_replace_all(messy, "[^0-9]", "")
[1] "2025550198"  "13035551212"

Capturing groups and backreferences

Wrapping part of a pattern in parentheses creates a capturing group, which does two things at once: it groups that piece of the pattern together (useful with quantifiers and alternation), and it remembers exactly what matched so you can reuse it. str_match() returns the whole match together with the text captured by each group, as a matrix:

dates <- c("2026-01-15", "2026-02-20")
str_match(dates, "(\\d{4})-(\\d{2})-(\\d{2})")
     [,1]         [,2]   [,3] [,4]
[1,] "2026-01-15" "2026" "01" "15"
[2,] "2026-02-20" "2026" "02" "20"

Inside a replacement string, \1, \2, and so on refer back to whatever each group captured, which makes it possible to rearrange a match rather than just deleting or replacing it outright:

str_replace(dates, "(\\d{4})-(\\d{2})-(\\d{2})", "\\2/\\3/\\1")
[1] "01/15/2026" "02/20/2026"

Literal matching with fixed()

Sometimes the string you are searching for is not a pattern at all, just literal text that happens to contain regex metacharacters, such as a period, parenthesis, or dollar sign. Wrapping the search string in fixed() tells stringr to match it character for character, with no regex interpretation (see Example 9.3 for what happens if you forget to).

str_detect(c("cat.txt", "cat"), fixed("."))
[1]  TRUE FALSE

Splitting a column into several

When you know the delimiter, tidyr’s separate_wider_delim() splits one column into several. Suppose a column stores names as "LAST, First Middle":

people <- c("SMITH, John A.", "O'NEILL, Anne", "Lee, Chen")

tibble(full = people) |>
  separate_wider_delim(full, delim = ",", names = c("last", "rest")) |>
  mutate(rest = str_trim(rest))
# A tibble: 3 × 2
  last    rest   
  <chr>   <chr>  
1 SMITH   John A.
2 O'NEILL Anne   
3 Lee     Chen

separate_wider_delim() only consumes the literal delimiter itself, so the piece after a ", " delimiter keeps its leading space; str_trim() cleans up any leading or trailing whitespace left behind by a split like this.

Working with messy strings in tibbles

Character columns often encode multiple variables in one field. The built-in who2 dataset has column names like sp_m_014, packing together a diagnosis method, gender, and age group; combining pivot_longer() with a separator unpacks them into proper variables in one step, exactly as in Session 8.

who2_long <- who2 |>
  pivot_longer(
    cols = !(country:year),
    names_to = c("diagnosis", "gender", "age"),
    names_sep = "_",
    values_to = "count"
  )

who2_long |> head()
# A tibble: 6 × 6
  country      year diagnosis gender age   count
  <chr>       <dbl> <chr>     <chr>  <chr> <dbl>
1 Afghanistan  1980 sp        m      014      NA
2 Afghanistan  1980 sp        m      1524     NA
3 Afghanistan  1980 sp        m      2534     NA
4 Afghanistan  1980 sp        m      3544     NA
5 Afghanistan  1980 sp        m      4554     NA
6 Afghanistan  1980 sp        m      5564     NA

Fringe cases and common pitfalls

ExampleExample 9.1

Quantifiers are greedy by default, and it shows.

html <- "<a><b>"
str_extract(html, "<.+>")     # greedy: matches as much as possible
[1] "<a><b>"
str_extract(html, "<.+?>")    # lazy: matches as little as possible
[1] "<a>"

+ (like * and {n,m}) grabs as much text as it possibly can while still letting the overall pattern match, so <.+> starts at the first < and only stops at the last > in the string, swallowing <a><b> whole instead of stopping at the first >. Adding ? right after a quantifier (+?, *?, ??) makes it lazy instead, matching as little as possible, which is almost always what you actually want when extracting something bounded by delimiters that might repeat.

ExampleExample 9.2

str_c() and paste0() treat a missing value very differently.

x <- c("a", NA, "c")
str_c("val: ", x)     # NA stays a genuine NA
[1] "val: a" NA       "val: c"
paste0("val: ", x)    # NA becomes the literal text "val: NA"
[1] "val: a"  "val: NA" "val: c"

str_c() propagates NA the way arithmetic does: if any input piece is missing, the whole result for that element is missing too, which is almost always the behavior you want, since it keeps is.na() meaningful on the result. paste0() instead coerces NA to the three-character string "NA" and glues it in like any other text, which can silently turn a genuinely missing value into a string that looks like real data (and will not be caught by a later is.na() check). If you’re translating old base R code that leans on paste()/paste0() into the tidyverse, this is one of the easiest behavior changes to miss.

ExampleExample 9.3

Forgetting that a search string is a regex changes what “matches” means.

words <- c("cat.txt", "cat", "hat")
str_detect(words, ".")              # "." means "any character," not a literal period
[1] TRUE TRUE TRUE
str_detect(words, fixed("."))       # fixed() treats "." as a literal period
[1]  TRUE FALSE FALSE

Every one of these strings contains at least one character, so the unquoted . pattern matches all three, which is almost certainly not what someone searching for filenames with a literal dot in them intended. The fix is either fixed("."), which turns off regex interpretation entirely, or escaping the metacharacter directly in the pattern ("\\."). This mistake is easy to make with any of the regex metacharacters (., (, ), $, ^, +, *, [, ]), not just the period.

ExampleExample 9.4

Without a word boundary, a pattern matches inside unrelated words too.

text <- c("cat", "category", "concatenate", "the cat sat")
str_detect(text, "cat")            # matches inside "category" and "concatenate" too
[1] TRUE TRUE TRUE TRUE
str_detect(text, "\\bcat\\b")      # only matches "cat" as a whole word
[1]  TRUE FALSE FALSE  TRUE

The plain pattern "cat" matches any occurrence of those three letters in sequence, including the middle of “concatenate,” which has nothing to do with cats. \b matches the zero-width boundary between a word character and a non-word character (or the start/end of the string), so \bcat\b only matches “cat” when it is not glued to other letters on either side. Whenever you want to match a whole word rather than a substring, word boundaries are the fix.

Recap

Term Definition
Raw string (r"(...)") A string literal where nothing is escaped, useful for paths and regexes full of backslashes.
str_length() / str_sub() Measure a string’s character length, or extract/replace a substring by position.
str_c() vs. paste0() Both concatenate strings, but str_c() propagates NA while paste0() converts it to the text "NA".
Character class shortcuts \d (digit), \s (whitespace), \w (word character: letter, digit, or underscore).
Quantifier ?, *, +, or {n,m}, controlling how many times the preceding piece may repeat; greedy by default, lazy with a trailing ?.
Word boundary (\b) A zero-width anchor matching the edge of a word, preventing a pattern from matching inside a longer word.
Capturing group ((...)) Groups part of a pattern and remembers what it matched, for reuse via \1, \2, and so on, or extraction with str_match().
fixed() Tells a stringr function to treat its pattern as literal text rather than a regex.
separate_wider_delim() Splits one column into several at a literal delimiter.

Check your understanding

NoteProblems
  1. What is the difference between str_extract(x, "<.+>") and str_extract(x, "<.+?>") on the string "<a><b>"? Which one would you want if you were trying to extract just the first tag?
  2. A dataset has a column where missing values were introduced with str_c(). Explain why is.na() still works correctly on that column, and why the same code using paste0() instead might not have.
  3. You want to find every row where a notes column contains a literal question mark. str_detect(notes, "?") returns strange results (or an error). What went wrong, and what are two ways to fix it?
  4. Explain what \bcat\b matches that plain cat does not, and give an example string where the two patterns would disagree.
  5. In str_replace(dates, "(\\d{4})-(\\d{2})-(\\d{2})", "\\2/\\3/\\1"), what do \\1, \\2, and \\3 refer to?

  1. "<.+>" is greedy and matches the entire string "<a><b>", since + grabs as much as possible while still allowing the pattern to match, stopping only at the last >. "<.+?>" is lazy and matches just "<a>", stopping at the first > it can. To extract just the first tag, use the lazy version.

  2. str_c() propagates missingness: if any piece being concatenated is NA, the whole result is NA, so is.na() still correctly identifies those rows afterward. paste0() instead converts a missing value into the literal three-character string "NA", which is not a missing value at all as far as R is concerned, so is.na() would return FALSE for those rows even though the data started out missing.

  3. ? is a regex metacharacter (a quantifier meaning “zero or one of the preceding thing”), not a literal question mark, so a bare "?" pattern is not valid on its own without something for it to quantify (or is silently interpreted as an empty, always-matching pattern, depending on context). Fix it with str_detect(notes, fixed("?")), which turns off regex interpretation entirely, or by escaping it directly: str_detect(notes, "\\?").

  4. \b anchors to a word boundary, so \bcat\b only matches “cat” as a complete, standalone word, not as part of a longer word. In "concatenate", plain cat matches the “cat” hidden inside “concatenate,” while \bcat\b does not match anywhere in that string at all, since “cat” there is not bounded by non-word characters on both sides.

  5. \1 refers to whatever the first parenthesized group captured (the four-digit year), \2 refers to the second group (the two-digit month), and \3 refers to the third group (the two-digit day). The replacement string rearranges them into month/day/year order using those backreferences instead of the original matched text.