flowchart TD
A[dplyr Verbs] --> B[select<br/>Pick columns]
A --> C[filter<br/>Pick rows]
A --> D[mutate<br/>Create/modify columns]
A --> E[arrange<br/>Sort rows]
A --> F[summarise<br/>Calculate summaries]
B --> B1["select(animal_id, weight)"]
C --> C1["filter(weight > 450)"]
D --> D1["mutate(bmi = weight/height)"]
E --> E1["arrange(weight)"]
F --> F1["summarise(mean_weight = mean(weight))"]
4 Data Manipulation with dplyr
4.1 Learning Objectives
By the end of this chapter, you will be able to:
- Create and modify variables using
dplyr::mutate() - Sort data using
dplyr::arrange()in ascending and descending order - Calculate summary statistics using
dplyr::summarise() - Perform grouped operations with
group_by()andsummarise() - Use helper functions:
count(),rename(), andrelocate() - Apply conditional logic with
if_else()andcase_when() - Work across multiple columns efficiently using
across() - Handle missing data with
is.na(),drop_na(),replace_na(), andcoalesce() - Use window functions:
lag(),lead(),cumsum(), androw_number() - Chain multiple dplyr operations into complex data transformation pipelines
4.2 Introduction to dplyr Verbs
In the previous chapter, you learned select() and filter() for choosing columns and rows. This chapter covers the remaining core dplyr verbs that complete your data manipulation toolkit.
4.2.1 The Five Main dplyr Verbs
This chapter focuses on: mutate(), arrange(), summarise(), and group_by()
NoteReview: Data Used in This Chapter
We’ll use animal science datasets throughout. Make sure you have these packages loaded:
library(tidyverse) # Includes dplyr, ggplot2, tidyr, etc.
library(lubridate) # For dates4.3 Creating and Modifying Variables with mutate()
mutate() creates new columns or modifies existing columns in your data frame.
4.3.1 Basic Usage
# Create example cattle data
cattle <- tibble(
animal_id = c("H001", "H002", "H003", "J001", "J002"),
breed = c("Holstein", "Holstein", "Holstein", "Jersey", "Jersey"),
weight_kg = c(600, 625, 580, 450, 470),
height_cm = c(145, 148, 142, 130, 133),
age_months = c(30, 36, 28, 30, 32)
)
cattle# A tibble: 5 × 5
animal_id breed weight_kg height_cm age_months
<chr> <chr> <dbl> <dbl> <dbl>
1 H001 Holstein 600 145 30
2 H002 Holstein 625 148 36
3 H003 Holstein 580 142 28
4 J001 Jersey 450 130 30
5 J002 Jersey 470 133 32# Create new column: weight in pounds
cattle %>%
mutate(weight_lb = weight_kg * 2.20462)# A tibble: 5 × 6
animal_id breed weight_kg height_cm age_months weight_lb
<chr> <chr> <dbl> <dbl> <dbl> <dbl>
1 H001 Holstein 600 145 30 1323.
2 H002 Holstein 625 148 36 1378.
3 H003 Holstein 580 142 28 1279.
4 J001 Jersey 450 130 30 992.
5 J002 Jersey 470 133 32 1036.# Create multiple new columns
cattle %>%
mutate(
weight_lb = weight_kg * 2.20462,
height_m = height_cm / 100,
bmi = weight_kg / (height_m^2)
)# A tibble: 5 × 8
animal_id breed weight_kg height_cm age_months weight_lb height_m bmi
<chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
1 H001 Holstein 600 145 30 1323. 1.45 285.
2 H002 Holstein 625 148 36 1378. 1.48 285.
3 H003 Holstein 580 142 28 1279. 1.42 288.
4 J001 Jersey 450 130 30 992. 1.3 266.
5 J002 Jersey 470 133 32 1036. 1.33 266.
Important
mutate() Adds Columns, Doesn’t Replace Data
mutate() returns a new data frame. To keep changes, assign the result:
# ❌ Changes are lost
cattle %>% mutate(weight_lb = weight_kg * 2.20462)
# ✅ Save the result
cattle <- cattle %>% mutate(weight_lb = weight_kg * 2.20462)
# OR
cattle_with_lb <- cattle %>% mutate(weight_lb = weight_kg * 2.20462)4.3.2 Overwriting Existing Columns
# Modify existing column: convert breed to uppercase
cattle %>%
mutate(breed = str_to_upper(breed))# A tibble: 5 × 5
animal_id breed weight_kg height_cm age_months
<chr> <chr> <dbl> <dbl> <dbl>
1 H001 HOLSTEIN 600 145 30
2 H002 HOLSTEIN 625 148 36
3 H003 HOLSTEIN 580 142 28
4 J001 JERSEY 450 130 30
5 J002 JERSEY 470 133 32# Round weight to nearest 10 kg
cattle %>%
mutate(weight_kg = round(weight_kg / 10) * 10)# A tibble: 5 × 5
animal_id breed weight_kg height_cm age_months
<chr> <chr> <dbl> <dbl> <dbl>
1 H001 Holstein 600 145 30
2 H002 Holstein 620 148 36
3 H003 Holstein 580 142 28
4 J001 Jersey 450 130 30
5 J002 Jersey 470 133 324.3.3 Using New Columns Immediately
You can reference newly created columns within the same mutate():
cattle %>%
mutate(
height_m = height_cm / 100, # Create height_m
bmi = weight_kg / (height_m^2) # Use height_m immediately!
)# A tibble: 5 × 7
animal_id breed weight_kg height_cm age_months height_m bmi
<chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 H001 Holstein 600 145 30 1.45 285.
2 H002 Holstein 625 148 36 1.48 285.
3 H003 Holstein 580 142 28 1.42 288.
4 J001 Jersey 450 130 30 1.3 266.
5 J002 Jersey 470 133 32 1.33 266.4.3.4 Useful Functions Inside mutate()
Common operations you’ll use with mutate():
| Operation | Function | Example |
|---|---|---|
| Math | +, -, *, /, ^ |
weight_kg * 2.20462 |
| Rounding | round(), floor(), ceiling() |
round(weight, 1) |
| Logarithms | log(), log10(), exp() |
log(concentration) |
| Ranking | min_rank(), dense_rank() |
min_rank(desc(weight)) |
| String functions | str_*() |
str_to_upper(breed) |
| Date functions | year(), month(), day() |
year(birth_date) |
| Conditional | if_else(), case_when() |
if_else(weight > 500, "Heavy", "Light") |
4.4 Conditional Operations
4.4.1 Simple Conditions with if_else()
if_else() creates values based on a TRUE/FALSE condition:
Syntax: if_else(condition, value_if_true, value_if_false)
# Classify animals as heavy or light
cattle %>%
mutate(
weight_class = if_else(weight_kg > 500, "Heavy", "Light")
)# A tibble: 5 × 6
animal_id breed weight_kg height_cm age_months weight_class
<chr> <chr> <dbl> <dbl> <dbl> <chr>
1 H001 Holstein 600 145 30 Heavy
2 H002 Holstein 625 148 36 Heavy
3 H003 Holstein 580 142 28 Heavy
4 J001 Jersey 450 130 30 Light
5 J002 Jersey 470 133 32 Light # Create logical column
cattle %>%
mutate(
is_mature = if_else(age_months >= 30, TRUE, FALSE)
)# A tibble: 5 × 6
animal_id breed weight_kg height_cm age_months is_mature
<chr> <chr> <dbl> <dbl> <dbl> <lgl>
1 H001 Holstein 600 145 30 TRUE
2 H002 Holstein 625 148 36 TRUE
3 H003 Holstein 580 142 28 FALSE
4 J001 Jersey 450 130 30 TRUE
5 J002 Jersey 470 133 32 TRUE # Can use existing values
cattle %>%
mutate(
adjusted_weight = if_else(breed == "Jersey",
weight_kg * 1.1, # Boost Jersey weights by 10%
weight_kg) # Keep others same
)# A tibble: 5 × 6
animal_id breed weight_kg height_cm age_months adjusted_weight
<chr> <chr> <dbl> <dbl> <dbl> <dbl>
1 H001 Holstein 600 145 30 600
2 H002 Holstein 625 148 36 625
3 H003 Holstein 580 142 28 580
4 J001 Jersey 450 130 30 495
5 J002 Jersey 470 133 32 517
Tip
if_else() vs base R ifelse()
dplyr’s if_else() is stricter and safer than base R’s ifelse():
- Type safety: Both true and false values must be the same type
- NA handling: Explicit
missingargument for NA values - Speed: Faster for large datasets
# ✅ Good (both character)
if_else(condition, "yes", "no")
# ❌ Error (different types)
if_else(condition, "yes", 0)
# ✅ With NAs
if_else(condition, "yes", "no", missing = "unknown")4.4.2 Multiple Conditions with case_when()
For multiple conditions, use case_when():
# Classify into three weight categories
cattle %>%
mutate(
weight_category = case_when(
weight_kg < 500 ~ "Light",
weight_kg < 600 ~ "Medium",
weight_kg >= 600 ~ "Heavy"
)
)# A tibble: 5 × 6
animal_id breed weight_kg height_cm age_months weight_category
<chr> <chr> <dbl> <dbl> <dbl> <chr>
1 H001 Holstein 600 145 30 Heavy
2 H002 Holstein 625 148 36 Heavy
3 H003 Holstein 580 142 28 Medium
4 J001 Jersey 450 130 30 Light
5 J002 Jersey 470 133 32 Light # Multiple factors
cattle %>%
mutate(
category = case_when(
breed == "Jersey" & weight_kg > 450 ~ "Large Jersey",
breed == "Jersey" & weight_kg <= 450 ~ "Small Jersey",
breed == "Holstein" & weight_kg > 600 ~ "Large Holstein",
breed == "Holstein" & weight_kg <= 600 ~ "Small Holstein",
TRUE ~ "Other" # Catch-all (like "else")
)
)# A tibble: 5 × 6
animal_id breed weight_kg height_cm age_months category
<chr> <chr> <dbl> <dbl> <dbl> <chr>
1 H001 Holstein 600 145 30 Small Holstein
2 H002 Holstein 625 148 36 Large Holstein
3 H003 Holstein 580 142 28 Small Holstein
4 J001 Jersey 450 130 30 Small Jersey
5 J002 Jersey 470 133 32 Large Jersey How case_when() works: 1. Evaluates conditions in order from top to bottom 2. Returns the value (~ right side) for the first TRUE condition 3. Stops checking once a match is found 4. Use TRUE ~ value as a catch-all for everything else
WarningOrder Matters in
case_when()!
# ❌ WRONG: Everything becomes "Heavy"
cattle %>%
mutate(
wrong = case_when(
weight_kg > 0 ~ "Heavy", # This matches EVERYTHING!
weight_kg > 500 ~ "Medium", # Never reached
TRUE ~ "Light" # Never reached
)
) %>%
select(animal_id, weight_kg, wrong)# A tibble: 5 × 3
animal_id weight_kg wrong
<chr> <dbl> <chr>
1 H001 600 Heavy
2 H002 625 Heavy
3 H003 580 Heavy
4 J001 450 Heavy
5 J002 470 Heavy# ✅ CORRECT: Specific conditions first
cattle %>%
mutate(
correct = case_when(
weight_kg > 600 ~ "Heavy",
weight_kg > 500 ~ "Medium",
TRUE ~ "Light"
)
) %>%
select(animal_id, weight_kg, correct)# A tibble: 5 × 3
animal_id weight_kg correct
<chr> <dbl> <chr>
1 H001 600 Medium
2 H002 625 Heavy
3 H003 580 Medium
4 J001 450 Light
5 J002 470 Light Rule: Put specific conditions before general ones!
4.4.3 Real-World Example: Creating Treatment Groups
# Create feed trial data
feed_trial <- tibble(
animal_id = sprintf("A%03d", 1:10),
baseline_weight = c(450, 480, 445, 490, 455, 470, 460, 475, 465, 485),
age_months = c(24, 30, 22, 32, 26, 28, 24, 30, 26, 31),
sex = c("F", "F", "M", "F", "M", "F", "F", "M", "F", "M")
)
feed_trial# A tibble: 10 × 4
animal_id baseline_weight age_months sex
<chr> <dbl> <dbl> <chr>
1 A001 450 24 F
2 A002 480 30 F
3 A003 445 22 M
4 A004 490 32 F
5 A005 455 26 M
6 A006 470 28 F
7 A007 460 24 F
8 A008 475 30 M
9 A009 465 26 F
10 A010 485 31 M # Assign animals to treatment groups based on multiple factors
feed_trial_assigned <- feed_trial %>%
mutate(
# Age category
age_group = case_when(
age_months < 26 ~ "Young",
age_months <= 30 ~ "Adult",
TRUE ~ "Mature"
),
# Weight category
weight_group = if_else(baseline_weight > 470, "Heavy", "Light"),
# Treatment assignment (balanced by sex and weight)
treatment = case_when(
sex == "F" & baseline_weight > 470 ~ "A",
sex == "F" & baseline_weight <= 470 ~ "B",
sex == "M" & baseline_weight > 470 ~ "B",
sex == "M" & baseline_weight <= 470 ~ "A"
)
)
feed_trial_assigned# A tibble: 10 × 7
animal_id baseline_weight age_months sex age_group weight_group treatment
<chr> <dbl> <dbl> <chr> <chr> <chr> <chr>
1 A001 450 24 F Young Light B
2 A002 480 30 F Adult Heavy A
3 A003 445 22 M Young Light A
4 A004 490 32 F Mature Heavy A
5 A005 455 26 M Adult Light A
6 A006 470 28 F Adult Light B
7 A007 460 24 F Young Light B
8 A008 475 30 M Adult Heavy B
9 A009 465 26 F Adult Light B
10 A010 485 31 M Mature Heavy B 4.5 Sorting Data with arrange()
arrange() sorts rows by one or more columns.
4.5.1 Basic Sorting
# Sort by weight (ascending, lightest first)
cattle %>%
arrange(weight_kg)# A tibble: 5 × 5
animal_id breed weight_kg height_cm age_months
<chr> <chr> <dbl> <dbl> <dbl>
1 J001 Jersey 450 130 30
2 J002 Jersey 470 133 32
3 H003 Holstein 580 142 28
4 H001 Holstein 600 145 30
5 H002 Holstein 625 148 36# Sort by weight (descending, heaviest first)
cattle %>%
arrange(desc(weight_kg))# A tibble: 5 × 5
animal_id breed weight_kg height_cm age_months
<chr> <chr> <dbl> <dbl> <dbl>
1 H002 Holstein 625 148 36
2 H001 Holstein 600 145 30
3 H003 Holstein 580 142 28
4 J002 Jersey 470 133 32
5 J001 Jersey 450 130 30# Sort by breed, then by weight within breed
cattle %>%
arrange(breed, weight_kg)# A tibble: 5 × 5
animal_id breed weight_kg height_cm age_months
<chr> <chr> <dbl> <dbl> <dbl>
1 H003 Holstein 580 142 28
2 H001 Holstein 600 145 30
3 H002 Holstein 625 148 36
4 J001 Jersey 450 130 30
5 J002 Jersey 470 133 32# Sort by breed (ascending), weight (descending)
cattle %>%
arrange(breed, desc(weight_kg))# A tibble: 5 × 5
animal_id breed weight_kg height_cm age_months
<chr> <chr> <dbl> <dbl> <dbl>
1 H002 Holstein 625 148 36
2 H001 Holstein 600 145 30
3 H003 Holstein 580 142 28
4 J002 Jersey 470 133 32
5 J001 Jersey 450 130 304.5.2 Sorting with Missing Values
# Data with missing weights
cattle_na <- tibble(
animal_id = c("H001", "H002", "H003", "H004", "H005"),
weight_kg = c(600, NA, 580, 625, NA)
)
cattle_na# A tibble: 5 × 2
animal_id weight_kg
<chr> <dbl>
1 H001 600
2 H002 NA
3 H003 580
4 H004 625
5 H005 NA# By default, NAs go to the end
cattle_na %>%
arrange(weight_kg)# A tibble: 5 × 2
animal_id weight_kg
<chr> <dbl>
1 H003 580
2 H001 600
3 H004 625
4 H002 NA
5 H005 NA# Descending: NAs still at the end
cattle_na %>%
arrange(desc(weight_kg))# A tibble: 5 × 2
animal_id weight_kg
<chr> <dbl>
1 H004 625
2 H001 600
3 H003 580
4 H002 NA
5 H005 NA
TipUse
arrange() to Check Your Work
When creating or modifying columns, sort to verify results:
# Did the weight classification work correctly?
cattle %>%
mutate(weight_class = if_else(weight_kg > 500, "Heavy", "Light")) %>%
arrange(weight_kg) %>%
select(animal_id, weight_kg, weight_class)# A tibble: 5 × 3
animal_id weight_kg weight_class
<chr> <dbl> <chr>
1 J001 450 Light
2 J002 470 Light
3 H003 580 Heavy
4 H001 600 Heavy
5 H002 625 Heavy Sorting makes it easy to spot errors!
4.6 Grouped Operations with group_by() and summarise()
The real power of dplyr comes from grouped operations: calculating summaries for each group separately.
4.6.1 Understanding group_by()
group_by() doesn’t change your data—it adds invisible grouping information:
# Create farm data with multiple breeds and farms
farm_data <- tibble(
animal_id = sprintf("A%03d", 1:12),
breed = rep(c("Holstein", "Jersey", "Angus"), each = 4),
farm = rep(c("North", "South"), 6),
weight_kg = c(600, 620, 590, 610, 450, 470, 455, 465,
550, 570, 540, 560)
)
farm_data# A tibble: 12 × 4
animal_id breed farm weight_kg
<chr> <chr> <chr> <dbl>
1 A001 Holstein North 600
2 A002 Holstein South 620
3 A003 Holstein North 590
4 A004 Holstein South 610
5 A005 Jersey North 450
6 A006 Jersey South 470
7 A007 Jersey North 455
8 A008 Jersey South 465
9 A009 Angus North 550
10 A010 Angus South 570
11 A011 Angus North 540
12 A012 Angus South 560# Group by breed
farm_data %>%
group_by(breed)# A tibble: 12 × 4
# Groups: breed [3]
animal_id breed farm weight_kg
<chr> <chr> <chr> <dbl>
1 A001 Holstein North 600
2 A002 Holstein South 620
3 A003 Holstein North 590
4 A004 Holstein South 610
5 A005 Jersey North 450
6 A006 Jersey South 470
7 A007 Jersey North 455
8 A008 Jersey South 465
9 A009 Angus North 550
10 A010 Angus South 570
11 A011 Angus North 540
12 A012 Angus South 560Notice: “Groups: breed [3]” in the output. The data looks the same, but it’s now grouped!
4.6.2 Summarizing Data with summarise()
summarise() (or summarize()) calculates summary statistics:
# Overall mean weight (no grouping)
farm_data %>%
summarise(
mean_weight = mean(weight_kg),
sd_weight = sd(weight_kg),
n = n() # n() counts rows
)# A tibble: 1 × 3
mean_weight sd_weight n
<dbl> <dbl> <int>
1 540 63.7 12# Mean weight BY BREED
farm_data %>%
group_by(breed) %>%
summarise(
mean_weight = mean(weight_kg),
sd_weight = sd(weight_kg),
n = n()
)# A tibble: 3 × 4
breed mean_weight sd_weight n
<chr> <dbl> <dbl> <int>
1 Angus 555 12.9 4
2 Holstein 605 12.9 4
3 Jersey 460 9.13 4
ImportantThe
group_by() + summarise() Pattern
This is one of the most powerful patterns in data analysis:
data %>%
group_by(category_column) %>%
summarise(
summary_name = summary_function(numeric_column)
)Read as: “For each category, calculate the summary”
4.6.3 Common Summary Functions
| Function | What it does | Example |
|---|---|---|
mean() |
Average | mean(weight) |
median() |
Median (50th percentile) | median(weight) |
sd() |
Standard deviation | sd(weight) |
var() |
Variance | var(weight) |
min() |
Minimum value | min(weight) |
max() |
Maximum value | max(weight) |
sum() |
Sum of all values | sum(milk_yield) |
n() |
Count of rows | n() |
n_distinct() |
Count unique values | n_distinct(animal_id) |
first() |
First value | first(weight) |
last() |
Last value | last(weight) |
4.6.4 Grouping by Multiple Variables
# Mean weight by breed AND farm
farm_data %>%
group_by(breed, farm) %>%
summarise(
mean_weight = mean(weight_kg),
n = n(),
.groups = "drop" # Remove grouping after summarise
)# A tibble: 6 × 4
breed farm mean_weight n
<chr> <chr> <dbl> <int>
1 Angus North 545 2
2 Angus South 565 2
3 Holstein North 595 2
4 Holstein South 615 2
5 Jersey North 452. 2
6 Jersey South 468. 2
NoteThe
.groups Argument
After summarise(), data remains grouped by all but the last grouping variable. This can cause unexpected behavior!
# Recommended: explicitly drop groups
data %>%
group_by(var1, var2) %>%
summarise(mean_x = mean(x), .groups = "drop")Options: - .groups = "drop": Remove all grouping (recommended) - .groups = "keep": Keep all grouping - .groups = "drop_last": Drop last grouping variable (default)
4.6.5 Multiple Summaries at Once
# Comprehensive summary by breed
farm_data %>%
group_by(breed) %>%
summarise(
n = n(),
mean_weight = mean(weight_kg),
sd_weight = sd(weight_kg),
min_weight = min(weight_kg),
max_weight = max(weight_kg),
median_weight = median(weight_kg),
.groups = "drop"
)# A tibble: 3 × 7
breed n mean_weight sd_weight min_weight max_weight median_weight
<chr> <int> <dbl> <dbl> <dbl> <dbl> <dbl>
1 Angus 4 555 12.9 540 570 555
2 Holstein 4 605 12.9 590 620 605
3 Jersey 4 460 9.13 450 470 4604.6.6 Filtering on Grouped Data
filter() works with groups too:
# Keep only animals heavier than the breed average
farm_data %>%
group_by(breed) %>%
filter(weight_kg > mean(weight_kg)) %>%
ungroup() # Remove grouping when done# A tibble: 6 × 4
animal_id breed farm weight_kg
<chr> <chr> <chr> <dbl>
1 A002 Holstein South 620
2 A004 Holstein South 610
3 A006 Jersey South 470
4 A008 Jersey South 465
5 A010 Angus South 570
6 A012 Angus South 5604.6.7 Mutating on Grouped Data
mutate() calculates within groups:
# Calculate deviation from breed mean
farm_data %>%
group_by(breed) %>%
mutate(
breed_mean = mean(weight_kg),
deviation = weight_kg - breed_mean
) %>%
ungroup() %>%
arrange(breed, animal_id)# A tibble: 12 × 6
animal_id breed farm weight_kg breed_mean deviation
<chr> <chr> <chr> <dbl> <dbl> <dbl>
1 A009 Angus North 550 555 -5
2 A010 Angus South 570 555 15
3 A011 Angus North 540 555 -15
4 A012 Angus South 560 555 5
5 A001 Holstein North 600 605 -5
6 A002 Holstein South 620 605 15
7 A003 Holstein North 590 605 -15
8 A004 Holstein South 610 605 5
9 A005 Jersey North 450 460 -10
10 A006 Jersey South 470 460 10
11 A007 Jersey North 455 460 -5
12 A008 Jersey South 465 460 54.7 Helper Functions
4.7.1 Counting with count()
count() is a shortcut for group_by() + summarise() + n():
# How many animals per breed?
# Long way
farm_data %>%
group_by(breed) %>%
summarise(n = n(), .groups = "drop")# A tibble: 3 × 2
breed n
<chr> <int>
1 Angus 4
2 Holstein 4
3 Jersey 4# Short way with count()
farm_data %>%
count(breed)# A tibble: 3 × 2
breed n
<chr> <int>
1 Angus 4
2 Holstein 4
3 Jersey 4# Count by multiple variables
farm_data %>%
count(breed, farm)# A tibble: 6 × 3
breed farm n
<chr> <chr> <int>
1 Angus North 2
2 Angus South 2
3 Holstein North 2
4 Holstein South 2
5 Jersey North 2
6 Jersey South 2# Sort by count (most common first)
farm_data %>%
count(breed, sort = TRUE)# A tibble: 3 × 2
breed n
<chr> <int>
1 Angus 4
2 Holstein 4
3 Jersey 4
Tip
count() Pro Tips
# Rename the count column
farm_data %>%
count(breed, name = "number_of_animals")# A tibble: 3 × 2
breed number_of_animals
<chr> <int>
1 Angus 4
2 Holstein 4
3 Jersey 4# Add a total row (using add_tally)
farm_data %>%
count(breed) %>%
mutate(proportion = n / sum(n))# A tibble: 3 × 3
breed n proportion
<chr> <int> <dbl>
1 Angus 4 0.333
2 Holstein 4 0.333
3 Jersey 4 0.3334.7.2 Renaming Columns with rename()
rename() changes column names:
Syntax: rename(new_name = old_name)
# Rename columns
cattle %>%
rename(
id = animal_id,
weight = weight_kg,
height = height_cm
)# A tibble: 5 × 5
id breed weight height age_months
<chr> <chr> <dbl> <dbl> <dbl>
1 H001 Holstein 600 145 30
2 H002 Holstein 625 148 36
3 H003 Holstein 580 142 28
4 J001 Jersey 450 130 30
5 J002 Jersey 470 133 32# Rename with a function
cattle %>%
rename_with(str_to_upper) # All columns to uppercase# A tibble: 5 × 5
ANIMAL_ID BREED WEIGHT_KG HEIGHT_CM AGE_MONTHS
<chr> <chr> <dbl> <dbl> <dbl>
1 H001 Holstein 600 145 30
2 H002 Holstein 625 148 36
3 H003 Holstein 580 142 28
4 J001 Jersey 450 130 30
5 J002 Jersey 470 133 32# Rename specific columns with a function
cattle %>%
rename_with(str_to_upper, starts_with("weight"))# A tibble: 5 × 5
animal_id breed WEIGHT_KG height_cm age_months
<chr> <chr> <dbl> <dbl> <dbl>
1 H001 Holstein 600 145 30
2 H002 Holstein 625 148 36
3 H003 Holstein 580 142 28
4 J001 Jersey 450 130 30
5 J002 Jersey 470 133 324.7.3 Reordering Columns with relocate()
relocate() moves columns to different positions:
# Move breed to the front
cattle %>%
relocate(breed)# A tibble: 5 × 5
breed animal_id weight_kg height_cm age_months
<chr> <chr> <dbl> <dbl> <dbl>
1 Holstein H001 600 145 30
2 Holstein H002 625 148 36
3 Holstein H003 580 142 28
4 Jersey J001 450 130 30
5 Jersey J002 470 133 32# Move weight_kg to the end
cattle %>%
relocate(weight_kg, .after = last_col())# A tibble: 5 × 5
animal_id breed height_cm age_months weight_kg
<chr> <chr> <dbl> <dbl> <dbl>
1 H001 Holstein 145 30 600
2 H002 Holstein 148 36 625
3 H003 Holstein 142 28 580
4 J001 Jersey 130 30 450
5 J002 Jersey 133 32 470# Move height_cm before breed
cattle %>%
relocate(height_cm, .before = breed)# A tibble: 5 × 5
animal_id height_cm breed weight_kg age_months
<chr> <dbl> <chr> <dbl> <dbl>
1 H001 145 Holstein 600 30
2 H002 148 Holstein 625 36
3 H003 142 Holstein 580 28
4 J001 130 Jersey 450 30
5 J002 133 Jersey 470 32# Move all numeric columns to the front
cattle %>%
relocate(where(is.numeric))# A tibble: 5 × 5
weight_kg height_cm age_months animal_id breed
<dbl> <dbl> <dbl> <chr> <chr>
1 600 145 30 H001 Holstein
2 625 148 36 H002 Holstein
3 580 142 28 H003 Holstein
4 450 130 30 J001 Jersey
5 470 133 32 J002 Jersey 4.8 Working Across Multiple Columns with across()
across() applies the same operation to multiple columns at once.
4.8.1 Basic Usage
# Create test data
test_data <- tibble(
animal_id = c("A001", "A002", "A003"),
weight_kg = c(600.123, 450.456, 550.789),
height_cm = c(145.678, 130.234, 140.567),
age_months = c(30.5, 28.3, 32.1)
)
test_data# A tibble: 3 × 4
animal_id weight_kg height_cm age_months
<chr> <dbl> <dbl> <dbl>
1 A001 600. 146. 30.5
2 A002 450. 130. 28.3
3 A003 551. 141. 32.1# Round all numeric columns to 1 decimal place
test_data %>%
mutate(across(where(is.numeric), round, 1))# A tibble: 3 × 4
animal_id weight_kg height_cm age_months
<chr> <dbl> <dbl> <dbl>
1 A001 600. 146. 30.5
2 A002 450. 130. 28.3
3 A003 551. 141. 32.1# Convert all character columns to uppercase
farm_data %>%
mutate(across(where(is.character), str_to_upper)) %>%
head(3)# A tibble: 3 × 4
animal_id breed farm weight_kg
<chr> <chr> <chr> <dbl>
1 A001 HOLSTEIN NORTH 600
2 A002 HOLSTEIN SOUTH 620
3 A003 HOLSTEIN NORTH 5904.8.2 Selecting Columns for across()
# Apply to specific columns
test_data %>%
mutate(across(c(weight_kg, height_cm), round, 1))# A tibble: 3 × 4
animal_id weight_kg height_cm age_months
<chr> <dbl> <dbl> <dbl>
1 A001 600. 146. 30.5
2 A002 450. 130. 28.3
3 A003 551. 141. 32.1# Apply to columns matching a pattern
test_data %>%
mutate(across(ends_with("_kg"), ~ . * 2.20462)) # Convert kg to lb# A tibble: 3 × 4
animal_id weight_kg height_cm age_months
<chr> <dbl> <dbl> <dbl>
1 A001 1323. 146. 30.5
2 A002 993. 130. 28.3
3 A003 1214. 141. 32.1# Apply to all except some columns
test_data %>%
mutate(across(-animal_id, round, 0))# A tibble: 3 × 4
animal_id weight_kg height_cm age_months
<chr> <dbl> <dbl> <dbl>
1 A001 600 146 30
2 A002 450 130 28
3 A003 551 141 32
NoteAnonymous Functions with
across()
The ~ creates an anonymous function, and . represents each column:
# These are equivalent:
across(cols, ~ round(., 1))
across(cols, round, 1)
across(cols, function(x) round(x, 1))
# Use ~ when you need more complex operations:
across(cols, ~ . * 2 + 1)
across(cols, ~ if_else(. > 100, ., . * 2))4.8.3 Multiple Summaries with across()
# Calculate multiple summaries for multiple columns
farm_data %>%
group_by(breed) %>%
summarise(
across(
weight_kg,
list(
mean = mean,
sd = sd,
min = min,
max = max
)
),
n = n(),
.groups = "drop"
)# A tibble: 3 × 6
breed weight_kg_mean weight_kg_sd weight_kg_min weight_kg_max n
<chr> <dbl> <dbl> <dbl> <dbl> <int>
1 Angus 555 12.9 540 570 4
2 Holstein 605 12.9 590 620 4
3 Jersey 460 9.13 450 470 4# Cleaner column names
farm_data %>%
group_by(breed) %>%
summarise(
across(
weight_kg,
list(
mean = mean,
sd = sd
),
.names = "{.col}_{.fn}" # Creates: weight_kg_mean, weight_kg_sd
),
.groups = "drop"
)# A tibble: 3 × 3
breed weight_kg_mean weight_kg_sd
<chr> <dbl> <dbl>
1 Angus 555 12.9
2 Holstein 605 12.9
3 Jersey 460 9.134.9 Handling Missing Data
Missing data (NA) is common in real datasets. dplyr provides several tools to handle it.
4.9.1 Detecting Missing Values
# Create data with missing values
cattle_missing <- tibble(
animal_id = c("H001", "H002", "H003", "H004", "H005"),
breed = c("Holstein", "Jersey", NA, "Angus", "Holstein"),
weight_kg = c(600, NA, 580, 625, NA),
age_months = c(30, 28, NA, 32, 26)
)
cattle_missing# A tibble: 5 × 4
animal_id breed weight_kg age_months
<chr> <chr> <dbl> <dbl>
1 H001 Holstein 600 30
2 H002 Jersey NA 28
3 H003 <NA> 580 NA
4 H004 Angus 625 32
5 H005 Holstein NA 26# Check for missing values
cattle_missing %>%
mutate(
breed_is_missing = is.na(breed),
weight_is_missing = is.na(weight_kg)
)# A tibble: 5 × 6
animal_id breed weight_kg age_months breed_is_missing weight_is_missing
<chr> <chr> <dbl> <dbl> <lgl> <lgl>
1 H001 Holstein 600 30 FALSE FALSE
2 H002 Jersey NA 28 FALSE TRUE
3 H003 <NA> 580 NA TRUE FALSE
4 H004 Angus 625 32 FALSE FALSE
5 H005 Holstein NA 26 FALSE TRUE # Count missing values per column
cattle_missing %>%
summarise(
across(everything(), ~ sum(is.na(.)))
)# A tibble: 1 × 4
animal_id breed weight_kg age_months
<int> <int> <int> <int>
1 0 1 2 14.9.2 Removing Missing Values with drop_na()
# Remove rows with ANY missing values
cattle_missing %>%
drop_na()# A tibble: 2 × 4
animal_id breed weight_kg age_months
<chr> <chr> <dbl> <dbl>
1 H001 Holstein 600 30
2 H004 Angus 625 32# Remove rows with missing values in specific columns
cattle_missing %>%
drop_na(weight_kg)# A tibble: 3 × 4
animal_id breed weight_kg age_months
<chr> <chr> <dbl> <dbl>
1 H001 Holstein 600 30
2 H003 <NA> 580 NA
3 H004 Angus 625 32# Remove rows missing BOTH weight and age
cattle_missing %>%
drop_na(weight_kg, age_months)# A tibble: 2 × 4
animal_id breed weight_kg age_months
<chr> <chr> <dbl> <dbl>
1 H001 Holstein 600 30
2 H004 Angus 625 32
WarningBe Careful When Dropping NAs!
Removing missing data can: - Reduce sample size significantly - Introduce bias if missingness isn’t random - Lose valuable information in other columns
Better approach: Understand WHY data is missing, then decide how to handle it.
4.9.3 Replacing Missing Values with replace_na()
# Replace NAs with specific values
cattle_missing %>%
mutate(
breed = replace_na(breed, "Unknown"),
weight_kg = replace_na(weight_kg, 0)
)# A tibble: 5 × 4
animal_id breed weight_kg age_months
<chr> <chr> <dbl> <dbl>
1 H001 Holstein 600 30
2 H002 Jersey 0 28
3 H003 Unknown 580 NA
4 H004 Angus 625 32
5 H005 Holstein 0 26# Replace NAs in multiple columns
cattle_missing %>%
mutate(
across(
where(is.numeric),
~ replace_na(., mean(., na.rm = TRUE)) # Replace with column mean
)
)# A tibble: 5 × 4
animal_id breed weight_kg age_months
<chr> <chr> <dbl> <dbl>
1 H001 Holstein 600 30
2 H002 Jersey 602. 28
3 H003 <NA> 580 29
4 H004 Angus 625 32
5 H005 Holstein 602. 264.9.4 Filling Missing Values with coalesce()
coalesce() returns the first non-missing value:
# Multiple weight measurements, use first available
weights <- tibble(
animal_id = c("A001", "A002", "A003", "A004"),
weight_measurement1 = c(600, NA, 580, NA),
weight_measurement2 = c(605, 450, NA, 625),
weight_measurement3 = c(NA, 455, 585, 630)
)
weights# A tibble: 4 × 4
animal_id weight_measurement1 weight_measurement2 weight_measurement3
<chr> <dbl> <dbl> <dbl>
1 A001 600 605 NA
2 A002 NA 450 455
3 A003 580 NA 585
4 A004 NA 625 630# Use first non-missing weight
weights %>%
mutate(
weight_final = coalesce(weight_measurement1,
weight_measurement2,
weight_measurement3)
)# A tibble: 4 × 5
animal_id weight_measurement1 weight_measurement2 weight_measurement3
<chr> <dbl> <dbl> <dbl>
1 A001 600 605 NA
2 A002 NA 450 455
3 A003 580 NA 585
4 A004 NA 625 630
# ℹ 1 more variable: weight_final <dbl>4.9.5 Handling NAs in Calculations
Most R functions return NA if any input is NA:
# Mean returns NA if any value is NA
mean(c(1, 2, NA, 4))[1] NA# Use na.rm = TRUE to remove NAs before calculating
mean(c(1, 2, NA, 4), na.rm = TRUE)[1] 2.333333# In a summarise
cattle_missing %>%
summarise(
mean_weight_with_na = mean(weight_kg), # Returns NA
mean_weight_removed = mean(weight_kg, na.rm = TRUE) # Calculates mean
)# A tibble: 1 × 2
mean_weight_with_na mean_weight_removed
<dbl> <dbl>
1 NA 602.4.10 Window Functions
Window functions operate on groups of rows and return a value for each row (unlike summarise() which returns one value per group).
4.10.1 Ranking Functions
# Create competition data
competition <- tibble(
animal_id = sprintf("A%03d", 1:8),
breed = rep(c("Holstein", "Jersey"), each = 4),
score = c(92, 88, 88, 85, 78, 76, 76, 74)
)
competition# A tibble: 8 × 3
animal_id breed score
<chr> <chr> <dbl>
1 A001 Holstein 92
2 A002 Holstein 88
3 A003 Holstein 88
4 A004 Holstein 85
5 A005 Jersey 78
6 A006 Jersey 76
7 A007 Jersey 76
8 A008 Jersey 74# Rank animals by score
competition %>%
mutate(
rank = min_rank(desc(score)), # Rank (ties get same rank, gaps after)
dense = dense_rank(desc(score)), # Dense rank (ties, no gaps)
row_num = row_number(desc(score)), # Row number (no ties, arbitrary order)
percentile = percent_rank(desc(score)) # Percentile (0 to 1)
)# A tibble: 8 × 7
animal_id breed score rank dense row_num percentile
<chr> <chr> <dbl> <int> <int> <int> <dbl>
1 A001 Holstein 92 1 1 1 0
2 A002 Holstein 88 2 2 2 0.143
3 A003 Holstein 88 2 2 3 0.143
4 A004 Holstein 85 4 3 4 0.429
5 A005 Jersey 78 5 4 5 0.571
6 A006 Jersey 76 6 5 6 0.714
7 A007 Jersey 76 6 5 7 0.714
8 A008 Jersey 74 8 6 8 1 # Rank within breed
competition %>%
group_by(breed) %>%
mutate(
breed_rank = min_rank(desc(score))
) %>%
ungroup() %>%
arrange(breed, breed_rank)# A tibble: 8 × 4
animal_id breed score breed_rank
<chr> <chr> <dbl> <int>
1 A001 Holstein 92 1
2 A002 Holstein 88 2
3 A003 Holstein 88 2
4 A004 Holstein 85 4
5 A005 Jersey 78 1
6 A006 Jersey 76 2
7 A007 Jersey 76 2
8 A008 Jersey 74 44.10.2 Offset Functions: lag() and lead()
lag() and lead() access previous or next values:
# Weight measurements over time
growth <- tibble(
animal_id = rep("A001", 5),
week = 1:5,
weight_kg = c(450, 465, 478, 492, 505)
)
growth# A tibble: 5 × 3
animal_id week weight_kg
<chr> <int> <dbl>
1 A001 1 450
2 A001 2 465
3 A001 3 478
4 A001 4 492
5 A001 5 505# Calculate weight change from previous week
growth %>%
mutate(
previous_weight = lag(weight_kg), # Previous row
weight_gain = weight_kg - lag(weight_kg), # Change from previous
next_weight = lead(weight_kg) # Next row
)# A tibble: 5 × 6
animal_id week weight_kg previous_weight weight_gain next_weight
<chr> <int> <dbl> <dbl> <dbl> <dbl>
1 A001 1 450 NA NA 465
2 A001 2 465 450 15 478
3 A001 3 478 465 13 492
4 A001 4 492 478 14 505
5 A001 5 505 492 13 NA# Lag by multiple rows
growth %>%
mutate(
two_weeks_ago = lag(weight_kg, n = 2)
)# A tibble: 5 × 4
animal_id week weight_kg two_weeks_ago
<chr> <int> <dbl> <dbl>
1 A001 1 450 NA
2 A001 2 465 NA
3 A001 3 478 450
4 A001 4 492 465
5 A001 5 505 4784.10.3 Cumulative Functions
# Running totals
milk_production <- tibble(
day = 1:7,
milk_liters = c(25, 28, 26, 29, 27, 30, 28)
)
milk_production# A tibble: 7 × 2
day milk_liters
<int> <dbl>
1 1 25
2 2 28
3 3 26
4 4 29
5 5 27
6 6 30
7 7 28# Cumulative statistics
milk_production %>%
mutate(
cumulative_milk = cumsum(milk_liters), # Running total
running_mean = cummean(milk_liters), # Running mean
running_min = cummin(milk_liters), # Running minimum
running_max = cummax(milk_liters) # Running maximum
)# A tibble: 7 × 6
day milk_liters cumulative_milk running_mean running_min running_max
<int> <dbl> <dbl> <dbl> <dbl> <dbl>
1 1 25 25 25 25 25
2 2 28 53 26.5 25 28
3 3 26 79 26.3 25 28
4 4 29 108 27 25 29
5 5 27 135 27 25 29
6 6 30 165 27.5 25 30
7 7 28 193 27.6 25 304.10.4 Real-World Example: Growth Rates
# Multiple animals, multiple measurements
growth_data <- tibble(
animal_id = rep(c("A001", "A002", "A003"), each = 4),
week = rep(c(0, 4, 8, 12), 3),
weight_kg = c(
# A001
450, 485, 518, 548,
# A002
445, 475, 502, 530,
# A003
455, 492, 525, 556
)
)
growth_data# A tibble: 12 × 3
animal_id week weight_kg
<chr> <dbl> <dbl>
1 A001 0 450
2 A001 4 485
3 A001 8 518
4 A001 12 548
5 A002 0 445
6 A002 4 475
7 A002 8 502
8 A002 12 530
9 A003 0 455
10 A003 4 492
11 A003 8 525
12 A003 12 556# Calculate growth metrics for each animal
growth_summary <- growth_data %>%
group_by(animal_id) %>%
mutate(
previous_weight = lag(weight_kg),
weight_gain = weight_kg - lag(weight_kg),
weeks_elapsed = week - lag(week),
daily_gain = weight_gain / (weeks_elapsed * 7),
total_gain = weight_kg - first(weight_kg),
cumulative_gain = cumsum(replace_na(weight_gain, 0))
) %>%
ungroup()
growth_summary# A tibble: 12 × 9
animal_id week weight_kg previous_weight weight_gain weeks_elapsed
<chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 A001 0 450 NA NA NA
2 A001 4 485 450 35 4
3 A001 8 518 485 33 4
4 A001 12 548 518 30 4
5 A002 0 445 NA NA NA
6 A002 4 475 445 30 4
7 A002 8 502 475 27 4
8 A002 12 530 502 28 4
9 A003 0 455 NA NA NA
10 A003 4 492 455 37 4
11 A003 8 525 492 33 4
12 A003 12 556 525 31 4
# ℹ 3 more variables: daily_gain <dbl>, total_gain <dbl>, cumulative_gain <dbl>4.11 Putting It All Together: Complex Pipelines
Real data analysis often requires chaining many operations. Here’s a comprehensive example:
# Create realistic farm data
set.seed(123)
farm_complete <- tibble(
animal_id = sprintf("F%04d", 1:100),
farm = sample(c("North", "South", "East", "West"), 100, replace = TRUE),
breed = sample(c("Holstein", "Jersey", "Angus", "Hereford"), 100,
replace = TRUE, prob = c(0.4, 0.3, 0.2, 0.1)),
sex = sample(c("M", "F"), 100, replace = TRUE),
birth_date = as.Date("2022-01-01") + sample(0:365, 100, replace = TRUE),
weight_kg = rnorm(100, mean = 500, sd = 80),
health_status = sample(c("Good", "Fair", "Poor", NA), 100,
replace = TRUE, prob = c(0.7, 0.2, 0.05, 0.05))
)
# Preview
head(farm_complete, 3)# A tibble: 3 × 7
animal_id farm breed sex birth_date weight_kg health_status
<chr> <chr> <chr> <chr> <date> <dbl> <chr>
1 F0001 East Jersey F 2022-08-06 397. Fair
2 F0002 East Holstein F 2022-04-16 454. Good
3 F0003 East Jersey F 2022-07-05 549. Good # Complex analysis pipeline
analysis_result <- farm_complete %>%
# Data cleaning
drop_na(health_status) %>% # Remove missing health status
filter(health_status != "Poor") %>% # Exclude poor health
mutate(
# Calculate age
age_days = as.numeric(Sys.Date() - birth_date),
age_months = age_days / 30.44,
# Standardize breed names
breed = str_to_title(breed),
# Weight categories
weight_class = case_when(
weight_kg < 450 ~ "Light",
weight_kg < 550 ~ "Medium",
TRUE ~ "Heavy"
),
# Round weight
weight_kg = round(weight_kg, 1)
) %>%
# Filter to mature animals only
filter(age_months >= 10) %>%
# Group analysis
group_by(farm, breed) %>%
summarise(
n = n(),
mean_weight = round(mean(weight_kg), 1),
sd_weight = round(sd(weight_kg), 1),
min_weight = min(weight_kg),
max_weight = max(weight_kg),
prop_heavy = mean(weight_class == "Heavy"),
.groups = "drop"
) %>%
# Filter to groups with at least 3 animals
filter(n >= 3) %>%
# Sort by mean weight
arrange(desc(mean_weight)) %>%
# Add ranking
mutate(rank = row_number())
analysis_result# A tibble: 13 × 9
farm breed n mean_weight sd_weight min_weight max_weight prop_heavy
<chr> <chr> <int> <dbl> <dbl> <dbl> <dbl> <dbl>
1 North Hereford 4 549. 47.1 485. 599. 0.75
2 North Jersey 6 538. 64.3 438. 628. 0.5
3 South Holstein 8 538. 89.1 443. 663 0.375
4 North Holstein 13 532. 72.7 409 683. 0.385
5 East Angus 6 529. 70.5 440. 604 0.5
6 East Jersey 10 500 99.9 339. 649. 0.3
7 West Holstein 6 492. 105. 367. 652. 0.333
8 South Jersey 12 482. 71.9 362. 660. 0.0833
9 West Jersey 5 481. 60.4 395. 559. 0.2
10 West Hereford 3 463. 50 419. 517 0
11 South Angus 3 461. 17.1 441. 471. 0
12 West Angus 3 457. 59.1 392. 507. 0
13 East Holstein 9 452. 77.7 360. 577 0.111
# ℹ 1 more variable: rank <int>
TipComplex Pipeline Best Practices
- Comment your steps: Explain what each section does
- One operation per line: Easier to read and debug
- Use intermediate results: Break very long pipes into steps
- Check intermediate output: Run the pipe up to a certain point to verify
- Consistent indentation: 2 spaces after pipe
# Good structure
result <- data %>%
# Step 1: Clean data
filter(!is.na(important_var)) %>%
mutate(clean_var = str_trim(var)) %>%
# Step 2: Calculate new variables
mutate(
var1 = calculation1,
var2 = calculation2
) %>%
# Step 3: Summarize by group
group_by(category) %>%
summarise(
mean_value = mean(value),
.groups = "drop"
)4.12 Summary
This chapter covered powerful dplyr verbs for data manipulation:
mutate()creates and modifies columns, works with functions and calculationsif_else()applies simple conditional logic (if/then/else)case_when()handles multiple conditions elegantly (replaces nested if_else)arrange()sorts data by one or more columns in ascending or descending ordersummarise()calculates summary statistics (mean, sd, min, max, count, etc.)group_by()+summarise()performs grouped operations (summaries by category)count()quickly counts observations per grouprename()changes column names;relocate()reorders columnsacross()applies functions to multiple columns efficiently- Missing data can be detected (
is.na()), removed (drop_na()), or replaced (replace_na(),coalesce()) - Window functions (
lag(),lead(),cumsum(), ranking) operate within groups - Complex pipelines chain many operations together for complete data transformations
Key Principle: Start simple, build complexity gradually, and always check your work!
Next chapter: Data visualization with ggplot2!
4.13 Homework Assignment
4.13.1 Assignment: Data Transformation and Analysis
Due: Before Week 5
4.13.1.1 Part 1: Creating Variables (25 points)
You will receive a dataset called pig_growth.csv with the following columns: - pig_id: Pig identifier - birth_date: Date of birth (YYYY-MM-DD) - breed: Pig breed - sex: M or F - weight_day0: Birth weight (kg) - weight_day28: Weight at 28 days (kg) - weight_day56: Weight at 56 days (kg) - feed_type: Type of feed given
Tasks:
- Read the data and examine its structure
- Create new variables using
mutate():age_days: Calculate age in days from birth_date to todaygain_0_28: Weight gain from day 0 to day 28gain_28_56: Weight gain from day 28 to day 56adg_0_28: Average daily gain for first period (gain / 28)adg_28_56: Average daily gain for second period (gain / 28)
- Create categorical variables:
birth_weight_class: “Low” (<1.2 kg), “Normal” (1.2-1.6 kg), “High” (>1.6 kg)sex_label: Convert “M” to “Male”, “F” to “Female”
- Round all weight and gain variables to 2 decimal places
4.13.1.2 Part 2: Conditional Logic (25 points)
Using your data with new variables:
- Create a performance category using
case_when():- “Excellent”: adg_28_56 > 0.50 kg/day AND weight_day56 > 18 kg
- “Good”: adg_28_56 > 0.45 kg/day
- “Fair”: adg_28_56 > 0.40 kg/day
- “Poor”: everything else
- Create a treatment recommendation using conditional logic:
- If performance is “Poor” AND sex is “M”: “Supplement + Monitor”
- If performance is “Poor” AND sex is “F”: “Supplement”
- If performance is “Fair”: “Monitor”
- Otherwise: “Continue”
- Identify concerning cases:
- Create a logical variable
needs_attentionthat is TRUE if:- Weight gain in second period is LESS than first period, OR
- Current weight is below 12 kg
- Create a logical variable
4.13.1.3 Part 3: Grouped Summaries (30 points)
Calculate comprehensive summary statistics:
- Overall summaries (no grouping):
- Number of pigs
- Mean birth weight, day 28 weight, day 56 weight
- Mean ADG for both periods
- Number and proportion needing attention
- Summaries by breed:
- Count of pigs per breed
- Mean ADG (both periods) per breed
- SD of ADG per breed
- Min and max day 56 weight per breed
- Summaries by feed type AND sex:
- Count per combination
- Mean day 56 weight
- Proportion in each performance category
- Summaries by performance category:
- Count per category
- Mean ADG in period 2
- What proportion of each category is male vs female?
Hint: Use group_by() + summarise() for each question. You may need across() for multiple columns.
4.13.1.4 Part 4: Complex Pipeline (20 points)
Create ONE pipeline that:
- Starts with the raw data
- Removes any pigs with missing weight measurements
- Calculates all new variables from Part 1
- Creates categories from Part 2
- Filters to pigs with birth_weight_class == “Normal”
- Filters to feed_type “A” or “B” only
- Groups by feed_type and sex
- Calculates mean day 56 weight and ADG for period 2
- Arranges by mean day 56 weight (descending)
- Adds a rank column
Show: - The complete pipeline (with comments explaining each step) - The final output - How many rows in the final result?
4.13.2 Recommended YAML
---
title: "Week 4 Homework: Data Manipulation with dplyr"
author: "Your Name"
date: today
format:
html:
toc: true
toc-depth: 3
code-fold: false
theme: cosmo
embed-resources: true
execute:
warning: false
message: false
---4.13.3 Grading Rubric
- Part 1: Creating Variables (25%):
- All variables created correctly (15%)
- Proper use of
mutate()(5%) - Rounding applied correctly (5%)
- Part 2: Conditional Logic (25%):
- Performance category correct (10%)
- Treatment recommendation correct (8%)
- Needs attention logic correct (7%)
- Part 3: Grouped Summaries (30%):
- Overall summaries (8%)
- Breed summaries (8%)
- Feed/sex summaries (7%)
- Performance category summaries (7%)
- Part 4: Complex Pipeline (20%):
- Pipeline executes correctly (12%)
- All steps included (5%)
- Clear comments and documentation (3%)
4.13.4 Bonus (10 points)
- Window functions:
- For each pig, calculate the difference between their day 56 weight and the breed average day 56 weight
- Rank pigs within their breed by day 56 weight
- Advanced grouping:
- Identify the top 3 breeds by mean ADG in period 2
- For just those breeds, create a detailed summary with all statistics
4.14 Additional Resources
4.14.1 Required Reading
- R for Data Science (2e) - Chapters 3-5: Data transformation
- dplyr documentation
- dplyr vignettes - Introduction to dplyr
4.14.2 Optional Reading
- Wickham, H., et al. (2019). “Welcome to the Tidyverse.” Journal of Open Source Software, 4(43), 1686. Link
- Programming with dplyr - Advanced techniques
- Row-wise operations - Working row-by-row
4.14.3 Videos
- “Data Manipulation in R” by StatQuest with Josh Starmer
- “dplyr Tutorial” by RStudio / Posit
- “Tidy Data and tidyr” by RStudio / Posit
- “grouped operations in dplyr” by Data Science Dojo
4.14.4 Cheat Sheets
4.14.5 Interactive Learning
4.14.6 Useful Websites
- Tidyverse - Main tidyverse homepage
- Stack Overflow: dplyr - Q&A community
- RStudio Community - Friendly help forum
- dplyr GitHub - Source code and issues
Next Chapter: Introduction to Data Visualization with ggplot2