Why is R important for data analysis and machine learning?
KnownRevise
R is a language and environment built for statistics and data analysis, and it matters in data science because it offers an unmatched range of statistical methods (from linear and mixed models to survival analysis and Bayesian modelling), first-class data visualization with ggplot2, a fluent data-manipulation toolkit in the tidyverse, a unified machine learning interface in tidymodels and caret, and reproducible reporting with R Markdown and Quarto. Over 20,000 packages on CRAN, plus Bioconductor for genomics, mean that new statistical methods often appear in R first, which is why it is widely used in academia, biostatistics, pharma, finance and analytics teams.
library(dplyr)
library(ggplot2)
# Summarise and model a built-in data set
mtcars |>
group_by(cyl) |>
summarise(mean_mpg = mean(mpg), n = n())
# cyl 4: mean_mpg 26.7 (n 11); cyl 6: 19.7 (n 7); cyl 8: 15.1 (n 14)
model <- lm(mpg ~ wt + hp, data = mtcars)
summary(model) # coefficients, standard errors, p-values, R-squared
ggplot(mtcars, aes(wt, mpg, colour = factor(cyl))) +
geom_point() +
geom_smooth(method = "lm", se = TRUE) +
labs(x = "Weight (1000 lbs)", y = "Miles per gallon", colour = "Cylinders")
R vs Python, Honestly
Aspect
R
Python
Classical statistics
Excellent, very broad
Good (statsmodels, SciPy)
Visualization
ggplot2 is a standard
Matplotlib, seaborn, plotly
Deep learning
Possible (torch for R, keras3) but smaller
Dominant (PyTorch, TensorFlow, JAX)
Production / engineering
Plumber APIs, Shiny; smaller ecosystem
Larger web and MLOps ecosystem
Community
Statisticians, researchers
Broad software and ML community
Common Pitfalls
Growing vectors in loops instead of vectorized operations.
Mixing up the native pipe |> (R 4.1+) and magrittr’s %>% in older code.
Ignoring factor levels when modelling categorical data.
Interview Tips
Lead with statistics, visualization and reporting as R’s strengths.
Acknowledge Python’s lead in deep learning and production, and mention that teams often use both (reticulate).
2
How do you install and load packages in R?
KnownRevise
In R you install a package once from CRAN with install.packages("name"), which downloads it into your library, and load it in every session where you use it with library(name), which attaches its functions to the search path. Packages from other sources use helpers such as BiocManager::install() for Bioconductor and remotes::install_github() or pak::pak() for GitHub, and project-specific package versions are best managed with renv for reproducibility.
Read full answerHide full answer
Applies toR 4.4+
Key Functions
Task
Code
Install from CRAN
install.packages("dplyr")
Install several
install.packages(c("dplyr", "ggplot2"))
Load (attach)
library(dplyr)
Check if installed and load
requireNamespace("dplyr", quietly = TRUE)
Use one function without attaching
dplyr::filter(df, x > 1)
Update packages
update.packages(ask = FALSE)
Remove
remove.packages("dplyr")
List installed
installed.packages()[, "Package"]
Package version
packageVersion("dplyr")
Bioconductor
BiocManager::install("DESeq2")
GitHub
remotes::install_github("user/repo") or pak::pak("user/repo")
Code Example
# Install only the packages that are missing
needed <- c("dplyr", "ggplot2", "readr")
missing <- needed[!vapply(needed, requireNamespace, logical(1), quietly = TRUE)]
if (length(missing) > 0) install.packages(missing)
library(dplyr) # attach for this session
library(ggplot2)
packageVersion("dplyr") # e.g. '1.1.4'
# Call a function without attaching the package (avoids name clashes)
readr::read_csv(readr::readr_example("mtcars.csv"), show_col_types = FALSE)
# Reproducible project library
# renv::init() # create a project-specific library and lockfile
# renv::snapshot() # record exact package versions in renv.lock
# renv::restore() # reinstall those versions on another machine
library() vs require()
library() stops with an error if the package is missing, which is what scripts should do.
require() returns FALSE with a warning; use requireNamespace() to test availability without attaching.
Name Conflicts
Loading dplyr masks stats::filter() and stats::lag(); use pkg::fun() to be explicit, or the conflicted package to force a choice.
Common Pitfalls
Putting install.packages() at the top of a shared script, which reinstalls every run.
Forgetting to load the package in a new session (“could not find function”).
Mixing package versions across team members; use renv.
Interview Tips
Explain install once vs load each session.
Mention renv for reproducibility and :: for namespacing.
3
What are the different data types in R?
KnownRevise
R has six basic (atomic) data types: logical (TRUE/FALSE), integer (5L), double (numeric, 3.14), character ("text"), complex (2+3i) and raw (bytes), and it organises values into data structures: atomic vectors (one type), lists (any types), matrices and arrays (vectors with dimensions), factors (categorical data with levels), and data frames or tibbles (lists of equal-length columns). There are no scalars in R: a single value is a vector of length 1, and missing values are represented by NA, with typed versions for each type.
Read full answerHide full answer
Applies toR 4.4+
Atomic Types
Type
Example
typeof()
Note
Logical
TRUE, FALSE, NA
"logical"
Use TRUE, not T
Integer
5L, 1:3
"integer"
The L suffix makes an integer
Double
3.14, 5
"double"
Default for numbers; class() is "numeric"
Character
"hello"
"character"
Strings
Complex
2+3i
"complex"
Rarely used
Raw
as.raw(255)
"raw"
Bytes
Data Structures
Structure
Dimensions
Types
Example
Vector
1
One
c(1, 2, 3)
List
1
Any
list(a = 1, b = "x")
Matrix
2
One
matrix(1:6, nrow = 2)
Array
n
One
array(1:24, dim = c(2, 3, 4))
Factor
1
Integer codes + levels
factor(c("low", "high"))
Data frame / tibble
2
One per column
data.frame(x = 1:2, y = c("a", "b"))
Code Example
x <- c(1.5, 2, 3)
typeof(x) # "double"
class(x) # "numeric"
typeof(1:3) # "integer"
typeof("a") # "character"
typeof(TRUE) # "logical"
# Vectors hold one type; mixing coerces (logical < integer < double < character)
c(1, "a", TRUE) # "1" "a" "TRUE"
size <- factor(c("small", "large", "small"), levels = c("small", "large"))
levels(size) # "small" "large"
as.integer(size) # 1 2 1
df <- data.frame(id = 1:3, name = c("A", "B", "C"))
str(df) # 'data.frame': 3 obs. of 2 variables: id int, name chr
l <- list(n = 1, s = "text", v = 1:3)
str(l)
NA_integer_; NA_real_; NA_character_ # typed missing values
is.na(c(1, NA, 3)) # FALSE TRUE FALSE
Special Values
NA: missing; NULL: absence of an object; NaN: not a number (0/0); Inf/-Inf: infinity (1/0).
Common Pitfalls
Expecting 5 to be an integer; it is a double unless written 5L.
Factors converted with as.numeric() give level codes, not the original numbers; use as.numeric(as.character(f)).
Using T and F, which can be reassigned.
Interview Tips
Distinguish types (typeof) from structures and classes (class).
Mention coercion order and the factor-to-number pitfall.
4
How do you convert data types in R?
KnownRevise
R converts data types with the as.*() family, such as as.numeric(), as.integer(), as.character(), as.logical(), as.factor() and as.Date(), and checks types with the matching is.*() functions; conversions that fail produce NA with a warning instead of an error. R also coerces implicitly when you combine types in a vector, following the order logical → integer → double → character, and data frame columns can be converted together with dplyr’s mutate(across(...)).
Read full answerHide full answer
Applies toR 4.4+, dplyr 1.1+
Common Conversions
From → To
Function
Example
Character → number
as.numeric()
as.numeric("3.5") is 3.5
Number → integer
as.integer()
as.integer(3.9) is 3 (truncates)
Anything → text
as.character()
as.character(42) is “42”
Text/number → logical
as.logical()
as.logical("TRUE"), as.logical(0) is FALSE
Vector → factor
factor() / as.factor()
factor(x, levels = c("lo", "hi"))
Factor → number
as.numeric(as.character(f))
Avoids getting level codes
Text → date
as.Date()
as.Date("2026-11-11")
Text → date-time
as.POSIXct()
as.POSIXct("2026-11-11 10:30", tz = "UTC")
Matrix ↔ data frame
as.data.frame(), as.matrix()
List → vector
unlist()
Code Example
library(dplyr)
as.numeric(c("1", "2.5", "abc")) # 1.0 2.5 NA, with a warning "NAs introduced by coercion"
as.integer(c(3.9, -3.9)) # 3 -3 (truncates toward zero)
as.logical(c("yes", "TRUE", "F")) # NA TRUE FALSE
f <- factor(c("10", "20", "10"))
as.numeric(f) # 1 2 1 (level codes, usually not wanted)
as.numeric(as.character(f)) # 10 20 10
as.Date("11/11/2026", format = "%m/%d/%Y") # "2026-11-11"
df <- data.frame(id = c("1", "2", "3"), score = c("90", "85", "x"), grade = c("A", "B", "A"))
df <- df |>
mutate(across(c(id, score), as.numeric), # several columns at once ("x" becomes NA)
grade = factor(grade))
str(df)
# Checking types
is.numeric(df$score) # TRUE
is.factor(df$grade) # TRUE
Reading Data with the Right Types
readr::read_csv(file, col_types = cols(id = col_integer(), date = col_date())) sets types on import.
readr::parse_number("$1,234") extracts numbers from formatted text (1234).
Common Pitfalls
Converting factors directly to numbers.
Silent NAs from failed conversions; check sum(is.na(x)) after converting.
Dates in non-ISO formats without a format argument.
Interview Tips
Mention the coercion hierarchy and the factor pitfall.
Show mutate(across()) for converting many columns.
5
What is the difference between a list and a data frame in R?
KnownRevise
A list is R’s most flexible container: an ordered collection whose elements can be of any type and any length, including other lists, while a data frame is a special list in which every element is a column vector of the same length, giving a rectangular table where each column has one type and each row is an observation. Lists suit nested or irregular data (model outputs, JSON, configuration); data frames and tibbles suit tabular data for analysis, modelling and plotting.
Read full answerHide full answer
Applies toR 4.4+, tibble 3.2+
Comparison
Aspect
List
Data frame
Structure
Any elements, any lengths
Columns of equal length
Element types
Mixed, nested
One type per column (list-columns possible)
Dimensions
1 (length)
2 (rows × columns)
Indexing
l[["a"]], l$a, l[1]
df[rows, cols], df$col, df[["col"]]
Typical use
Model results, JSON, mixed objects
Tabular data, modelling, ggplot2
Created with
list()
data.frame(), tibble()
class()
"list"
"data.frame" (tibbles add "tbl_df", "tbl")
Code Example
# A list: mixed types and lengths
person <- list(name = "Ana", age = 31, skills = c("R", "SQL"), address = list(city = "Lisbon"))
person$skills # "R" "SQL"
person[["address"]]$city # "Lisbon"
person["age"] # a list of length 1 containing age
length(person) # 4
# A data frame: equal-length columns
df <- data.frame(name = c("Ana", "Ben"), age = c(31, 28), stringsAsFactors = FALSE)
dim(df) # 2 2
df[df$age > 30, ] # rows where age > 30
is.list(df) # TRUE: a data frame is a list of columns
# This fails: columns must have the same length
# data.frame(a = 1:3, b = 1:2) # error: arguments imply differing number of rows
# Tibble with a list-column (nested data in a table)
library(tibble)
tb <- tibble(id = 1:2, scores = list(c(90, 85), c(70)))
tb$scores[[1]] # 90 85
Indexing Difference
l[1] returns a list containing the first element; l[[1]] returns the element itself.
df["age"] returns a one-column data frame; df[["age"]] and df$age return the vector.
When to Use Which
Data frame / tibble: observations × variables, analysis with dplyr and ggplot2.
List: results of functions (lm() returns a list-based object), heterogeneous collections, iterating with lapply/purrr::map.
Common Pitfalls
Confusing [ and [[ on lists.
Expecting a data frame to hold columns of different lengths.
df[, 1] drops to a vector for data frames but not for tibbles.
Interview Tips
State that a data frame is a list of equal-length vectors.
Explain [ vs [[ clearly.
6
How do you handle missing values in R?
KnownRevise
In R, missing values are NA (with typed variants), detected with is.na(), counted with sum(is.na(x)) or colSums(is.na(df)), skipped in calculations with na.rm = TRUE, removed with na.omit(), complete.cases() or tidyr::drop_na(), and filled with tidyr::replace_na(), dplyr::coalesce(), grouped means or medians, or model-based imputation packages such as mice. The right method depends on why the data is missing (completely at random, at random, or not at random), and imputation for models should be learned on training data only, for example inside a tidymodels recipe.
Read full answerHide full answer
Applies toR 4.4+, dplyr 1.1+, tidyr 1.3+, recipes 1.1+
library(dplyr)
library(tidyr)
colSums(is.na(airquality)) # Ozone 37, Solar.R 7 missing; other columns 0
mean(airquality$Ozone) # NA
mean(airquality$Ozone, na.rm = TRUE) # about 42.1
nrow(drop_na(airquality)) # 111 complete rows out of 153
imputed <- airquality |>
group_by(Month) |>
mutate(Ozone = if_else(is.na(Ozone), median(Ozone, na.rm = TRUE), as.numeric(Ozone)),
Solar.R_missing = is.na(Solar.R)) |> # keep an indicator before filling
ungroup() |>
mutate(Solar.R = replace_na(Solar.R, median(Solar.R, na.rm = TRUE)))
anyNA(imputed$Ozone) # FALSE
# For modelling: impute inside a recipe so it is learned on training data only
library(recipes)
rec <- recipe(Ozone ~ ., data = airquality) |>
step_impute_median(all_numeric_predictors())
Choosing a Method
Few missing, random: drop rows or simple imputation.
Many missing or informative missingness: indicators plus imputation, or models that handle NA.
Time series: carry forward or interpolate (zoo::na.approx).
Inference: multiple imputation (mice) to keep uncertainty honest.
Common Pitfalls
Comparing with x == NA (always NA); use is.na(x).
Imputing before splitting into train and test.
Forgetting that lm() silently drops rows with NA (na.action = na.omit).
Interview Tips
Mention na.rm, drop_na, replace_na and mice.
Discuss missingness mechanisms (MCAR, MAR, MNAR).
7
What is the apply family of functions in R?
KnownRevise
The apply family is a set of base R functions that apply a function over the elements, rows, columns or groups of a data structure without writing an explicit loop: apply() works on matrix rows or columns, lapply() on each list element and returns a list, sapply() simplifies that result to a vector or matrix, vapply() does the same with a declared output type, mapply() iterates over several inputs in parallel, and tapply() applies a function to groups. They make code shorter and clearer; the tidyverse equivalent is purrr’s map() family.
vapply() in functions and packages (safe output type).
sapply() interactively.
apply() for matrices, not data frames (it converts them to a matrix, coercing types).
Vectorized functions (rowSums, colMeans) are faster than apply() for simple sums and means.
For grouped data frames, dplyr::group_by() + summarise() is usually clearer than tapply().
Common Pitfalls
sapply() returning a list unexpectedly when results have different lengths.
Using apply() on a data frame with character columns, which turns everything into text.
Assuming apply functions are always faster than loops; they are mainly clearer.
Interview Tips
Explain the output type of each function.
Mention vapply() for safety and purrr’s map_*() as the modern alternative.
8
How does variable scope work in R?
KnownRevise
R uses lexical (static) scoping: when a function looks up a variable, it searches first in its own local environment, then in the environment where the function was defined (its enclosing environment), and so on up to the global environment, attached packages and base R. Assignments with <- inside a function create local variables that disappear when the function returns, the super-assignment operator <<- modifies a variable in an enclosing environment, and functions that capture their defining environment are closures, which makes function factories possible.
Read full answerHide full answer
Applies toR 4.4+
Environments
Environment
Contents
Local (execution)
Arguments and variables created in the current function call
Enclosing
Environment where the function was defined
Global (globalenv())
Objects created at the console or top of a script
Package / search path
Attached packages, then base
Empty
End of the chain
Rules
Name lookup: local → enclosing → … → global → search path.
Assignment: <- always creates or modifies in the current environment.
<<-: searches enclosing environments for the name and modifies it there (or creates it in global if not found).
Fresh start: each call gets a new local environment, so variables don’t persist between calls (unless stored in an enclosing environment).
Lazy evaluation: arguments are evaluated only when used.
Code Example
x <- 10 # global
f <- function() {
x <- 20 # local, shadows the global x
g <- function() x + 1 # g looks up x in f's environment
g()
}
f() # 21
x # 10: global unchanged
h <- function() x * 2 # no local x: uses the global
h() # 20
# Closures: a function factory
make_counter <- function() {
count <- 0
function() {
count <<- count + 1 # modifies count in the enclosing environment
count
}
}
counter <- make_counter()
counter() # 1
counter() # 2
# Inspecting environments
environment(counter) # the factory's environment
exists("count") # FALSE in the global environment
Dynamic Lookup at Call Time
Lookup happens when the function runs, not when it is defined, so a global variable changed later affects the result. Pass values as arguments to avoid this.
Common Pitfalls
Functions that depend on global variables silently; pass everything as arguments.
Overusing <<-, which makes code hard to follow; prefer returning values.
Expecting a variable created inside a function to exist afterwards.
Interview Tips
Name it “lexical scoping” and walk through the lookup order.
Show a closure with <<- as the classic example.
9
How do you read and write data in R?
KnownRevise
R reads and writes data with base functions such as read.csv()/write.csv() and readRDS()/saveRDS(), and with faster, more consistent packages: readr (read_csv(), write_csv()) for text files, readxl and writexl for Excel, data.table’s fread()/fwrite() for very large CSVs, arrow for Parquet and Feather, haven for SPSS, Stata and SAS files, jsonlite for JSON, and DBI with a driver for databases. Use .rds to save a single R object exactly as it is, and Parquet to share large tables with other tools.
library(readr)
path <- tempfile(fileext = ".csv")
write_csv(mtcars, path) # write a CSV
cars <- read_csv(path, show_col_types = FALSE) # read it back as a tibble
dim(cars) # 32 11
# Specify column types explicitly for reliable imports
spec <- cols(mpg = col_double(), cyl = col_integer(), .default = col_double())
cars <- read_csv(path, col_types = spec)
# Exact R object round trip (keeps factors, attributes)
rds <- tempfile(fileext = ".rds")
saveRDS(iris, rds)
identical(readRDS(rds), iris) # TRUE
# Parquet for large data and other tools
# arrow::write_parquet(cars, "cars.parquet"); arrow::read_parquet("cars.parquet")
# Database with DBI (SQLite in memory)
library(DBI)
con <- dbConnect(RSQLite::SQLite(), ":memory:")
dbWriteTable(con, "mtcars", mtcars)
dbGetQuery(con, "SELECT cyl, AVG(mpg) AS mpg FROM mtcars GROUP BY cyl")
dbDisconnect(con)
Choosing
Small to medium CSVs: readr (clear types, progress, tibbles).
Very large CSVs: data.table::fread() or arrow::open_dataset().
Exchanging with Python or Spark: Parquet.
Saving R work: .rds.
Common Pitfalls
write.csv() adds row names by default; use row.names = FALSE or readr.
Locale issues with decimal commas; use read_csv2() or locale(decimal_mark = ",").
Relying on guessed column types for IDs with leading zeros; read them as character.
Interview Tips
Mention readr, readxl, arrow and DBI as the modern toolkit.
Explain why .rds and Parquet are better than CSV for some uses.
10
What are the key differences between R and Python for machine learning?
KnownRevise
R and Python can both do machine learning well, but they lean in different directions: R excels at statistical modelling, inference, visualization and reproducible reporting, with tidymodels and caret as unified ML frameworks, while Python dominates deep learning, large-scale and production machine learning, with scikit-learn, PyTorch, TensorFlow and a broad engineering ecosystem. Many teams use R for analysis and experimentation and Python for deep learning and deployment, and the two interoperate through reticulate, Arrow and shared file formats.
Read full answerHide full answer
Applies toR 4.4+ (tidymodels 1.2), Python 3.12 (scikit-learn 1.5+)
Python: deep learning, NLP and computer vision, integrating ML into software and production systems.
Both: reticulate calls Python from R; Arrow and Parquet share data.
Common Pitfalls
Claiming one language is universally better.
Ignoring the team’s skills and the existing production stack.
Interview Tips
Give an honest, balanced comparison and pick based on the task.
Mention interoperability (reticulate, Arrow).
11
How do you select a subset of a data frame in R?
KnownRevise
You subset a data frame in R either with base indexing, df[rows, columns] using row numbers, logical conditions or column names, df$col and df[["col"]] for single columns, and the subset() function, or with dplyr verbs: filter() for rows, select() for columns, slice() for row positions and distinct() for unique rows. Base indexing needs no packages, while dplyr reads like a sentence, uses column names directly and chains with the pipe.
library(dplyr)
# Base R: logical row index, named columns
mtcars[mtcars$mpg > 30, c("mpg", "hp")]
# 4 cars: Fiat 128 (32.4), Honda Civic (30.4), Toyota Corolla (33.9), Lotus Europa (30.4)
# Watch out for NA in conditions: which() drops them
airquality[which(airquality$Ozone > 100), c("Ozone", "Month")]
# dplyr: the same idea, chained
mtcars |>
filter(cyl == 4, hp > 90) |>
select(mpg, hp, wt) |>
arrange(desc(mpg))
# Select helpers
iris |> select(starts_with("Petal"), Species) |> head(3)
# Top 3 by a column
mtcars |> slice_max(mpg, n = 3)
Comparison
Aspect
Base R
dplyr
Dependencies
None
dplyr
NA in conditions
Returns NA rows unless which()
filter() drops NA rows
Row names
Kept
Dropped for tibbles (use tibble::rownames_to_column())
Readability
Compact, more symbols
Verb-based, chainable
Common Pitfalls
Forgetting the comma: df[df$x > 1] selects columns, not rows.
df[, "col"] returns a vector for data frames (use drop = FALSE to keep a data frame).
subset() inside functions can behave unexpectedly because of non-standard evaluation.
Interview Tips
Show both base and dplyr, and mention the NA behaviour difference.
Use which() or filter() for conditions on columns with missing values.
12
How is the dplyr package used for data manipulation in R?
KnownRevise
dplyr is the tidyverse package for data manipulation, built around a small set of verbs that each do one thing: filter() keeps rows, select() picks columns, mutate() adds or changes columns, arrange() sorts, summarise() reduces groups to summaries, and group_by() makes the other verbs work per group, with joins (left_join() and others) to combine tables. The verbs take a data frame first and return a data frame, so they chain with the pipe (|> or %>%) into readable pipelines, and the same code can run against databases through dbplyr.
Readable pipelines that mirror the analysis steps.
Consistent: first argument is data, columns referred to by name.
Fast (C++ backend), and works on databases (dbplyr), data.table (dtplyr) and Arrow/DuckDB.
Common Pitfalls
Forgetting to ungroup() (or .groups = "drop"), so later operations stay grouped.
dplyr::filter masks stats::filter; use dplyr::filter() if both are needed.
Using $ inside verbs (filter(df$x > 1)); refer to columns directly.
Interview Tips
Name the core verbs and group_by() + summarise() pattern.
Mention across() and case_when() for modern dplyr.
13
How do you reshape data with the tidyr package?
KnownRevise
tidyr reshapes data between wide and long formats with pivot_longer(), which gathers several columns into name-value pairs (wide → long), and pivot_wider(), which spreads name-value pairs into columns (long → wide); it also splits and combines columns with separate_wider_delim() and unite(), and handles nested data with nest(), unnest() and missing combinations with complete(). The goal is “tidy data”: each variable a column, each observation a row and each value a cell, which is the format dplyr, ggplot2 and modelling functions expect.
pivot_wider() with duplicate keys creates list-columns; aggregate first or use values_fn = sum.
Using the old gather()/spread(), which are superseded.
Forgetting names_sep/names_pattern when column names hold several variables.
Interview Tips
Define tidy data and when you’d pivot each way.
Mention values_fn for duplicate keys.
14
What does the aggregate() function do in R?
KnownRevise
aggregate() is a base R function that splits data into groups, applies a summary function such as mean, sum or length to each group, and returns the results as a data frame; it is most often used with a formula, aggregate(y ~ group, data = df, FUN = mean), which reads “summarise y by group”. It handles several grouping variables and several value columns at once, and is the base R counterpart of dplyr’s group_by() + summarise().
Read full answerHide full answer
Applies toR 4.4+
Syntax
Form
Example
Formula
aggregate(mpg ~ cyl, data = mtcars, FUN = mean)
Several groups
aggregate(mpg ~ cyl + am, data = mtcars, FUN = mean)
Several values
aggregate(cbind(mpg, hp) ~ cyl, data = mtcars, FUN = mean)
All other columns
aggregate(. ~ Species, data = iris, FUN = mean)
Data frame + by list
aggregate(mtcars$mpg, by = list(cyl = mtcars$cyl), FUN = mean)
aggregate(mpg ~ cyl, data = mtcars, FUN = mean)
# cyl mpg
# 1 4 26.66364
# 2 6 19.74286
# 3 8 15.10000
aggregate(cbind(mpg, hp) ~ cyl + am, data = mtcars, FUN = mean) # 6 groups
aggregate(. ~ Species, data = iris, FUN = mean) # every numeric column by species
# Several statistics: FUN returns a vector, giving a matrix column
res <- aggregate(mpg ~ cyl, data = mtcars, FUN = function(x) c(mean = mean(x), n = length(x)))
res$mpg # matrix with mean and n
# The formula method drops rows with NA by default (na.action = na.omit)
aggregate(Ozone ~ Month, data = airquality, FUN = mean)
# dplyr equivalent
library(dplyr)
mtcars |> group_by(cyl) |> summarise(mpg = mean(mpg))
aggregate() vs Alternatives
Tool
Notes
aggregate()
Base R, formula interface, data frame output
tapply()
One value column, returns an array
dplyr::summarise()
Readable, several named summaries, works on databases
data.tabledt[, .(m = mean(x)), by = g]
Fastest on large data
Common Pitfalls
Functions returning several values produce a matrix column, which surprises later code; use dplyr for multiple summaries.
The formula method silently drops rows with any NA in used columns.
Very slow on large data compared with data.table or dplyr.
Interview Tips
Show the formula form and cbind() for several columns.
Mention the NA behaviour and the dplyr equivalent.
15
How do you merge data frames in R?
KnownRevise
Data frames are merged in R with base merge(x, y, by = "key"), where all.x = TRUE, all.y = TRUE or all = TRUE turn the default inner join into a left, right or full outer join, or with dplyr’s join functions, inner_join(), left_join(), right_join(), full_join(), plus the filtering joins semi_join() and anti_join(). Rows are matched on one or more key columns; stacking data frames with the same columns is done with rbind() or dplyr::bind_rows(), and side by side with cbind() or bind_cols().
Read full answerHide full answer
Applies toR 4.4+, dplyr 1.1+
Join Types
Join
Keeps
dplyr
Base merge()
Inner
Matching rows in both
inner_join(x, y, by = "id")
merge(x, y, by = "id")
Left
All of x
left_join()
all.x = TRUE
Right
All of y
right_join()
all.y = TRUE
Full
All of both
full_join()
all = TRUE
Semi
x rows with a match (x columns only)
semi_join()
—
Anti
x rows without a match
anti_join()
—
Code Example
library(dplyr)
customers <- tibble(id = c(1, 2, 3), name = c("Ana", "Ben", "Cai"))
orders <- tibble(order_id = 101:104, customer_id = c(1, 1, 3, 4), amount = c(20, 35, 15, 50))
inner_join(customers, orders, by = join_by(id == customer_id)) # 3 rows (Ana x2, Cai)
left_join(customers, orders, by = join_by(id == customer_id)) # 4 rows: Ben with NA amount
full_join(customers, orders, by = join_by(id == customer_id)) # 5 rows: includes customer 4
anti_join(customers, orders, by = join_by(id == customer_id)) # Ben: no orders
semi_join(customers, orders, by = join_by(id == customer_id)) # Ana, Cai
# Base R equivalent of the left join
merge(customers, orders, by.x = "id", by.y = "customer_id", all.x = TRUE)
# Detect unexpected duplicate keys (one-to-one expected)
left_join(customers, orders, by = join_by(id == customer_id), relationship = "one-to-many")
# Stacking rows and columns
bind_rows(customers, tibble(id = 4, name = "Dee"))
Merge Checks
Count rows before and after; joins on duplicated keys multiply rows.
Use relationship = (dplyr 1.1+) to declare and verify the expected cardinality.
Use anti_join() to find keys that don’t match (typos, different types).
Make key types match (character vs numeric IDs).
Common Pitfalls
Many-to-many joins that silently explode the row count (dplyr warns).
Joining on columns with the same name that aren’t keys, creating .x/.y suffixes.
Factor vs character keys with different levels.
Interview Tips
Draw the join types and mention semi/anti joins.
Stress checking row counts and key uniqueness.
16
How do you handle categorical variables in R models?
KnownRevise
In R, categorical variables are stored as factors, and modelling functions such as lm() and glm() automatically turn a factor into dummy (indicator) variables using treatment contrasts, with the first level as the reference; you control this by setting the reference with relevel() or forcats::fct_relevel(), ordering levels, grouping rare levels with fct_lump_n(), and choosing encodings such as one-hot dummies, contrasts, or target and embedding encodings for high-cardinality variables. In tidymodels, recipes steps like step_dummy(), step_other() and step_novel() make these encodings reproducible on new data.
Read full answerHide full answer
Applies toR 4.4+, forcats 1.0+, recipes 1.1+
Strategies
Strategy
Tool
When
Factor with treatment contrasts
factor(), used by lm()/glm()
Default for statistical models
Set the reference level
relevel(f, ref = "b"), fct_relevel()
Interpretable comparisons
One-hot / dummy variables
model.matrix(), step_dummy(one_hot = TRUE)
Models needing numeric input (glmnet, xgboost)
Ordered factors
factor(x, ordered = TRUE)
Ordinal data (low < medium < high)
Group rare levels
fct_lump_n(), step_other()
Many levels with few rows
Unseen levels at prediction
step_novel(), step_unknown()
Production data
Target / effect encoding
embed::step_lencode_glm()
High cardinality
Native handling
ranger, rpart, lightgbm handle factors
Tree models
Code Example
library(forcats)
df <- transform(mtcars, cyl = factor(cyl), am = factor(am, labels = c("auto", "manual")))
fit <- lm(mpg ~ cyl + am + wt, data = df)
coef(fit) # cyl6 and cyl8 are compared with the reference level cyl4
df$cyl <- relevel(df$cyl, ref = "8")
coef(lm(mpg ~ cyl + am + wt, data = df)) # now compared with 8 cylinders
head(model.matrix(~ cyl + am, data = df)) # the dummy columns R creates
# Group rare levels
fct_lump_n(factor(c("a", "a", "a", "b", "b", "c", "d")), n = 2) # c and d become "Other"
# tidymodels recipe for a model needing numeric input
library(recipes)
rec <- recipe(Species ~ ., data = iris) |>
step_novel(all_nominal_predictors()) |>
step_other(all_nominal_predictors(), threshold = 0.05) |>
step_dummy(all_nominal_predictors(), one_hot = FALSE)
Interpreting Dummy Coefficients
Each coefficient is the difference from the reference level, holding other variables constant.
Test the overall factor effect with anova() or car::Anova(), not single coefficients.
Common Pitfalls
Numeric codes (1, 2, 3) treated as continuous; convert to factor.
Full one-hot encoding plus an intercept in linear models (perfect collinearity); R’s default drops one level.
New levels at prediction time causing errors; handle them in the recipe.
Interview Tips
Explain treatment contrasts and reference levels.
Mention recipes for consistent preprocessing.
17
How do you scale or normalize data in R?
KnownRevise
In R you standardize numeric data (mean 0, standard deviation 1) with the base scale() function, normalize to the 0-1 range with min-max scaling (x − min) / (max − min), and apply either consistently in modelling with tidymodels recipes (step_normalize(), step_range()) or caret’s preProcess(), which learn the parameters from the training data and reuse them on new data. Scaling matters for distance- and gradient-based methods (k-NN, k-means, SVM, PCA, regularized regression, neural networks) but not for tree-based models.
Read full answerHide full answer
Applies toR 4.4+, recipes 1.1+, caret 6.0-94
Methods
Method
Formula
R
Standardization (z-score)
(x − mean) / sd
scale(x), step_normalize()
Min-max
(x − min) / (max − min)
step_range(min = 0, max = 1)
Robust
(x − median) / IQR
custom mutate()
Log
log(x) or log1p(x)
step_log()
Yeo-Johnson / Box-Cox
Power transforms
step_YeoJohnson(), step_BoxCox()
Code Example
library(dplyr)
z <- scale(mtcars$mpg) # matrix with attributes "scaled:center" and "scaled:scale"
round(c(mean(z), sd(z)), 3) # 0 1
attr(z, "scaled:center") # 20.09062 (the mean of mpg)
min_max <- function(x) (x - min(x, na.rm = TRUE)) / (max(x, na.rm = TRUE) - min(x, na.rm = TRUE))
range(min_max(mtcars$hp)) # 0 1
mtcars_std <- mtcars |> mutate(across(c(mpg, hp, wt), ~ as.numeric(scale(.x))))
# Learn scaling on training data, apply to test data
library(recipes)
set.seed(42)
idx <- sample(nrow(iris), 120)
train <- iris[idx, ]
test <- iris[-idx, ]
rec <- recipe(Species ~ ., data = train) |>
step_normalize(all_numeric_predictors()) |>
prep(training = train) # means and sds from training data only
train_std <- bake(rec, new_data = NULL)
test_std <- bake(rec, new_data = test) # same parameters applied to test data
round(colMeans(test_std[, 1:4]), 2) # near 0 but not exactly
When to Scale
Model
Scaling needed?
k-NN, k-means, SVM, PCA
Yes
Ridge, lasso, elastic net
Yes
Neural networks
Yes
Linear regression (unregularized)
No for predictions; helps interpretation
Decision trees, random forests, gradient boosting
No
Common Pitfalls
Scaling the full dataset before splitting (leakage).
scale() returns a matrix; wrap with as.numeric() inside mutate().
Min-max scaling is sensitive to outliers.
Interview Tips
Explain fit-on-train, apply-to-test with prep()/bake().
Say which models need scaling and which don’t.
18
Why should you split data into training and test sets in R?
KnownRevise
Splitting data into a training set and a test set is essential because a model must be judged on data it has never seen: performance on the training data is optimistic, since flexible models can memorize noise (overfitting), while performance on a held-out test set estimates how the model will do on new data. In R this is done with rsample::initial_split() (often stratified), caret::createDataPartition() or base sample(), with all preprocessing learned on the training set only and the test set used once at the end; cross-validation on the training set is used for tuning.
Read full answerHide full answer
Applies toR 4.4+, rsample 1.2+, caret 6.0-94
Why It Matters
Honest evaluation: training error underestimates real error.
Detect overfitting: a big gap between training and test performance.
Model selection: compare models fairly on data none of them saw.
Avoid leakage: preprocessing (scaling, imputation, feature selection) must be learned on training data only.
Typical Splits
Scheme
Use
70-80% train / 20-30% test
Default for medium data
Train / validation / test
Tuning on validation, final check on test
k-fold CV on training + test
Best for small and medium data
Stratified split
Classification, especially imbalanced
Time-based split
Time series: train on the past
Group split
Multiple rows per patient or customer
Code Example
library(rsample)
library(dplyr)
set.seed(42)
split <- initial_split(iris, prop = 0.8, strata = Species)
train <- training(split)
test <- testing(split)
nrow(train); nrow(test) # 120 and 30
count(train, Species) # about 40 of each species (stratified)
# Training vs test error shows overfitting
fit <- lm(mpg ~ poly(hp, 6), data = mtcars[1:24, ])
rmse <- function(obs, pred) sqrt(mean((obs - pred)^2))
rmse(mtcars$mpg[1:24], predict(fit)) # small
rmse(mtcars$mpg[25:32], predict(fit, mtcars[25:32, ])) # usually much larger
# Time-based split
ts_split <- initial_time_split(airquality, prop = 0.8) # first 80% of rows to train
# Cross-validation folds on the training data for tuning
folds <- vfold_cv(train, v = 5, strata = Species)
Common Pitfalls
Using the test set repeatedly during tuning, which makes it a validation set.
Random splits for time series or grouped data.
Forgetting set.seed(), so splits can’t be reproduced.
Preprocessing before the split.
Interview Tips
Explain overfitting and data leakage in one sentence each.
Mention stratification and time-based splits as context-dependent choices.
19
What is the purpose of the caret package in R?
KnownRevise
caret (Classification And REgression Training) gives R a single, consistent interface to hundreds of machine learning models: one train() function handles resampling (cross-validation, bootstrap), hyperparameter tuning over a grid, preprocessing (centering, scaling, imputation, PCA), and performance evaluation, with helpers for data splitting (createDataPartition()), feature selection and confusionMatrix(). It removes the need to learn each package’s own syntax; it is still widely used, though its author now develops tidymodels as the successor, so new projects often use tidymodels.
Read full answerHide full answer
Applies toR 4.4+, caret 6.0-94
Main Functions
Function
Purpose
createDataPartition()
Stratified train/test split
trainControl()
Resampling method, folds, repeats, summary function
train()
Fit and tune any supported model (method = "rf", "glmnet", "svmRadial", "xgbTree", …)
Forgetting set.seed() before train(), so folds differ between models.
Preprocessing outside train(), which leaks across folds.
Needing the underlying model package installed (e.g. randomForest for method = "rf").
Interview Tips
Describe trainControl() + train() as the core pattern.
Mention tidymodels as the modern successor.
20
How do you perform linear regression analysis in R?
KnownRevise
Linear regression in R is done with lm(y ~ x1 + x2, data = df), which fits the model by ordinary least squares; summary() then reports the coefficients with standard errors, t-values and p-values, the residual standard error, R² and adjusted R², and the overall F-test, while confint() gives confidence intervals, predict() makes predictions, and plot(model) shows diagnostic plots to check the assumptions. The formula syntax supports factors, interactions (x1 * x2), polynomial terms (poly(x, 2) or I(x^2)) and transformations (log(y)).
Read full answerHide full answer
Applies toR 4.4+ (stats), broom 1.0+
Workflow
Explore relationships with plots and correlations.
model <- lm(mpg ~ wt + hp, data = mtcars)
summary(model)
# Coefficients (rounded): intercept 37.23, wt -3.88, hp -0.032; both predictors significant
# Multiple R-squared about 0.83
confint(model) # 95% intervals for each coefficient
new_cars <- data.frame(wt = c(2.5, 3.5), hp = c(100, 150))
predict(model, new_cars, interval = "confidence") # mean response with interval
predict(model, new_cars, interval = "prediction") # individual car, wider interval
par(mfrow = c(2, 2))
plot(model) # residuals vs fitted, Q-Q, scale-location, leverage
par(mfrow = c(1, 1))
# Factor and interaction
model2 <- lm(mpg ~ wt * factor(am) + hp, data = mtcars)
anova(model, model2) # does the extra term improve the fit?
# Tidy output
library(broom)
tidy(model, conf.int = TRUE)
glance(model) # R-squared, AIC, BIC
Reading summary()
Output
Meaning
Estimate
Change in y for one unit of x, others held constant
Std. Error, t value, Pr(>|t|)
Uncertainty and significance of each coefficient
Residual standard error
Typical prediction error in y units
R² / adjusted R²
Share of variance explained (adjusted penalizes extra terms)
F-statistic
Whether the model beats an intercept-only model
Common Pitfalls
Interpreting coefficients causally from observational data.
Ignoring assumption checks and influential points.
Extrapolating beyond the range of the data.
Comparing models with different responses (y vs log(y)) by R².
Interview Tips
Walk through the summary() output confidently.
Distinguish confidence and prediction intervals.
21
How does hypothesis testing work in R?
KnownRevise
Hypothesis testing in R follows the usual steps: state a null hypothesis (no effect or difference) and an alternative, choose a test that fits the data, run it with a built-in function such as t.test(), wilcox.test(), chisq.test(), prop.test(), cor.test() or aov(), and compare the reported p-value with a significance level (often 0.05) while also reading the effect size and confidence interval. A small p-value means the data would be unlikely if the null hypothesis were true; it doesn’t measure the size or importance of the effect, and it doesn’t prove the alternative.
Read full answerHide full answer
Applies toR 4.4+ (stats)
Common Tests
Question
Test
R function
One mean vs a value
One-sample t-test
t.test(x, mu = 20)
Two group means
Welch t-test (default)
t.test(y ~ group, data = df)
Paired measurements
Paired t-test
t.test(after, before, paired = TRUE)
Non-normal two groups
Wilcoxon rank-sum
wilcox.test(y ~ group, data = df)
Several group means
One-way ANOVA
aov(y ~ group, data = df), then TukeyHSD()
Two categorical variables
Chi-squared test
chisq.test(table(a, b))
Small counts
Fisher’s exact test
fisher.test(table(a, b))
Proportions
Proportion test
prop.test(x = c(45, 30), n = c(100, 100))
Correlation
Pearson / Spearman
cor.test(x, y, method = "spearman")
Normality
Shapiro-Wilk
shapiro.test(x)
Code Example
# Do manual and automatic cars differ in fuel economy?
t.test(mpg ~ am, data = mtcars)
# Welch t-test: t about -3.77, p about 0.0014; mean mpg 17.1 (automatic) vs 24.4 (manual)
# Is transmission associated with the number of cylinders?
tab <- table(mtcars$am, mtcars$cyl)
chisq.test(tab) # warns: expected counts are small
fisher.test(tab) # exact alternative for small tables
# Do iris species differ in sepal length?
fit <- aov(Sepal.Length ~ Species, data = iris)
summary(fit) # very small p-value
TukeyHSD(fit) # which pairs differ, with adjusted intervals
# Correlation between weight and mpg
cor.test(mtcars$wt, mtcars$mpg) # r about -0.87
# Adjusting for multiple tests
p.adjust(c(0.01, 0.04, 0.03), method = "holm")
Interpreting Results
p-value: probability of results at least this extreme if the null were true.
Confidence interval: plausible range for the effect; more informative than p alone.
Effect size: difference in means, Cohen’s d, odds ratio, correlation.
Assumptions: independence, approximate normality for t-tests (robust with larger samples), equal variances only for the classic Student test (var.equal = TRUE).
Common Pitfalls
Treating p > 0.05 as proof of no effect.
Running many tests without adjusting (p.adjust).
Ignoring assumptions or the warning from chisq.test() about small expected counts.
Paired data analysed as independent groups.
Interview Tips
Choose the test from the data type and design, then interpret the p-value and interval correctly.
Mention that t.test() uses Welch’s test by default.
22
What are common statistical distributions in R?
KnownRevise
R provides common probability distributions through a consistent family of four functions per distribution: d* for the density or probability mass (dnorm), p* for the cumulative probability (pnorm), q* for quantiles (qnorm), and r* for random numbers (rnorm); the most used are the normal, binomial, Poisson, uniform, exponential, t, chi-squared, F, beta and gamma distributions. They are used to compute probabilities and critical values, simulate data, check model assumptions and build statistical tests.
Read full answerHide full answer
Applies toR 4.4+ (stats)
The Four-Function Pattern
Prefix
Meaning
Example
d
Density / probability mass
dbinom(3, size = 10, prob = 0.5)
p
Cumulative probability P(X ≤ x)
pnorm(1.96) is about 0.975
q
Quantile (inverse of p)
qnorm(0.975) is about 1.96
r
Random draws
rnorm(100, mean = 0, sd = 1)
Common Distributions
Distribution
Use
Functions
Normal
Measurements, errors
dnorm, pnorm, qnorm, rnorm
Binomial
Successes in n trials
dbinom, pbinom, …
Poisson
Counts per interval
dpois, ppois, …
Uniform
Equal likelihood
dunif, runif
Exponential
Waiting times
dexp, rexp
t
Small-sample means
dt, pt, qt
Chi-squared
Variances, contingency tests
dchisq, pchisq
F
Ratio of variances, ANOVA
df, pf
Beta
Probabilities, proportions
dbeta, rbeta
Gamma
Positive skewed data
dgamma, rgamma
Negative binomial
Overdispersed counts
dnbinom
Code Example
pnorm(1.96) # 0.9750021
qnorm(0.975) # 1.959964
pnorm(110, mean = 100, sd = 15) - pnorm(90, mean = 100, sd = 15) # about 0.495
dbinom(3, size = 10, prob = 0.5) # 0.1171875 (= 120 / 1024)
pbinom(3, size = 10, prob = 0.5) # 0.171875 (P(X <= 3) = 176 / 1024)
dpois(2, lambda = 4) # about 0.1465
1 - ppois(6, lambda = 4) # P(X > 6) about 0.111
qt(0.975, df = 10) # 2.228139: critical t value for a 95% CI
set.seed(42)
x <- rnorm(1000, mean = 50, sd = 10)
c(mean = mean(x), sd = sd(x)) # close to 50 and 10
hist(x, breaks = 30, freq = FALSE)
curve(dnorm(x, 50, 10), add = TRUE, lwd = 2) # overlay the theoretical density
qqnorm(x); qqline(x) # check normality visually
Choosing a Distribution
Continuous and symmetric: normal (or t with heavy tails).
Counts: Poisson, or negative binomial when the variance exceeds the mean.
Yes/no out of n: binomial.
Positive and skewed: log-normal, gamma, exponential.
Fit candidates with MASS::fitdistr() or fitdistrplus and compare.
Common Pitfalls
Using dnorm() as a probability for continuous variables; densities aren’t probabilities.
Explain the d/p/q/r naming convention; it applies to every distribution.
Show one calculation, such as a critical value with qnorm or qt.
23
How do you conduct time series analysis in R?
KnownRevise
Time series analysis in R starts by storing the data as a time series object (base ts for regular series, or a tsibble in the tidyverts ecosystem), plotting it to see trend and seasonality, decomposing it (decompose(), stl(), or feasts::STL()), checking stationarity and autocorrelation (ACF and PACF plots, unit-root tests), fitting models such as exponential smoothing (ETS) and ARIMA (forecast::auto.arima() or fable::ARIMA()), checking the residuals, and forecasting with prediction intervals evaluated on a hold-out period. The forecast package and the newer fable framework cover most needs.
Read full answerHide full answer
Applies toR 4.4+, forecast 8.23, fable 0.4 / tsibble 1.1
Steps
Step
Base / forecast
tidyverts
Create series
ts(x, start = c(1949, 1), frequency = 12)
as_tsibble()
Plot
plot(), autoplot()
autoplot()
Decompose
decompose(), stl()
model(STL())
Autocorrelation
acf(), pacf()
ACF(), PACF()
Stationarity
tseries::adf.test(), forecast::ndiffs()
features(unitroot_kpss)
Models
ets(), auto.arima()
ETS(), ARIMA()
Residual checks
checkresiduals()
gg_tsresiduals()
Forecast
forecast(fit, h = 12)
forecast(h = "1 year")
Accuracy
accuracy(fc, test)
accuracy()
Code Example
library(forecast)
air <- log(AirPassengers) # monthly 1949-1960; log stabilizes variance
plot(stl(air, s.window = "periodic")) # trend, seasonal and remainder components
train <- window(air, end = c(1958, 12))
test <- window(air, start = c(1959, 1))
fit_arima <- auto.arima(train) # chooses (p,d,q)(P,D,Q)[12] by AICc
fit_ets <- ets(train)
fc_arima <- forecast(fit_arima, h = length(test))
fc_ets <- forecast(fit_ets, h = length(test))
accuracy(fc_arima, test) # RMSE, MAE, MAPE on the test period
accuracy(fc_ets, test)
checkresiduals(fit_arima) # residuals should look like white noise
autoplot(fc_arima) + autolayer(test, series = "actual")
exp(fc_arima$mean[1:3]) # back-transform the first forecasts to passengers
Key Concepts
Trend and seasonality: long-run movement and repeating patterns.
Stationarity: constant mean and variance; achieved by differencing (diff()), often needed for ARIMA.
ACF/PACF: help identify AR and MA orders.
Prediction intervals: widen with the horizon.
Common Pitfalls
Random train/test splits; always hold out the most recent period.
Forgetting to back-transform log forecasts.
Ignoring residual autocorrelation (the model missed structure).
Interview Tips
Walk through plot → decompose → model → check residuals → forecast and evaluate.
Mention auto.arima(), ETS and the tidyverts packages.
24
What is the role of the glm() function in R?
KnownRevise
glm() fits generalized linear models, which extend linear regression to responses that are not normally distributed by combining a linear predictor with a link function and an error distribution chosen with the family argument: binomial for yes/no outcomes (logistic regression), poisson for counts, Gamma for positive skewed data, and gaussian for ordinary linear regression. It estimates coefficients by maximum likelihood (iteratively reweighted least squares) and works with the same tools as lm(): summary(), predict(), anova(), confint() and AIC().
Read full answerHide full answer
Applies toR 4.4+ (stats), broom 1.0+
Families and Links
Family
Response
Default link
Typical use
gaussian
Continuous
identity
Linear regression
binomial
0/1 or proportions
logit
Classification, probabilities
poisson
Counts
log
Event counts, rates (with offset)
quasipoisson
Overdispersed counts
log
Variance larger than mean
Gamma
Positive, skewed
inverse
Costs, durations
inverse.gaussian
Positive, skewed
1/μ²
Rarely used
For overdispersed counts, MASS::glm.nb() fits negative binomial regression.
Code Example
# Logistic regression: probability of a manual transmission
logit <- glm(am ~ wt + hp, data = mtcars, family = binomial)
summary(logit) # coefficients on the log-odds scale
exp(coef(logit)) # odds ratios
predict(logit, data.frame(wt = 2.5, hp = 120), type = "response") # a probability
# Poisson regression: counts of warp breaks by wool and tension
pois <- glm(breaks ~ wool + tension, data = warpbreaks, family = poisson)
summary(pois)
exp(coef(pois)) # rate ratios
deviance(pois) / df.residual(pois) # well above 1 signals overdispersion
quasi <- glm(breaks ~ wool + tension, data = warpbreaks, family = quasipoisson)
summary(quasi)$dispersion # estimated dispersion
# Model comparison
logit_small <- glm(am ~ wt, data = mtcars, family = binomial)
anova(logit_small, logit, test = "Chisq") # likelihood-ratio test
AIC(logit_small, logit)
Interpreting Coefficients
Family
Coefficient meaning
binomial (logit)
Change in log-odds; exp() gives an odds ratio
poisson (log)
Change in log count; exp() gives a rate ratio
gaussian
Change in the mean response
Common Pitfalls
Forgetting type = "response" in predict(), which returns the linear predictor (log-odds).
Ignoring overdispersion in Poisson models, which understates standard errors.
Perfect separation in logistic regression (“fitted probabilities numerically 0 or 1”); consider brglm2 or regularization.
Reading R² from glm(); use deviance, AIC or pseudo-R² instead.
Interview Tips
Explain family + link and give logistic and Poisson examples.
Mention odds ratios and type = "response".
25
What is multicollinearity and how do you address it in R?
KnownRevise
Multicollinearity means two or more predictors in a regression are strongly correlated, so the model can’t separate their individual effects: coefficients become unstable, their standard errors inflate, signs can flip, and individual p-values look insignificant even when the predictors jointly matter, although predictions on similar data can still be fine. In R you detect it with correlation matrices and variance inflation factors (car::vif()), and address it by removing or combining redundant predictors, centering variables used in interactions or polynomials, using principal components, or using regularized regression such as ridge or the elastic net with glmnet.
Read full answerHide full answer
Applies toR 4.4+, car 3.1+, glmnet 4.1+
Detection
Method
Rule of thumb
Correlation matrix (cor())
Pairwise |r| above about 0.8 is a warning sign
VIF (car::vif())
Above 5 is concerning, above 10 serious
Condition number (kappa())
Large values (over about 30) suggest collinearity
Unstable coefficients
Big changes when adding or removing variables
Remedies
Remedy
When
Drop one of the correlated predictors
Redundant measures (e.g. two weight variables)
Combine them (average, index)
Several measures of one concept
Center before interactions/polynomials
Structural collinearity from x and x^2
Principal components regression
Many correlated predictors
Ridge or elastic net (glmnet)
Prediction focus; shrinks correlated coefficients together
Collect more data
Sometimes reduces variance
Code Example
library(car)
round(cor(mtcars[, c("disp", "hp", "wt", "cyl")]), 2) # disp, cyl and wt strongly correlated
fit <- lm(mpg ~ disp + hp + wt + cyl, data = mtcars)
vif(fit) # disp has the highest VIF of the four
fit2 <- lm(mpg ~ hp + wt, data = mtcars) # drop redundant size measures
vif(fit2) # much lower VIFs
summary(fit2)$coefficients
# Ridge regression keeps all predictors but shrinks them
library(glmnet)
x <- scale(as.matrix(mtcars[, c("disp", "hp", "wt", "cyl")]))
set.seed(42)
cv <- cv.glmnet(x, mtcars$mpg, alpha = 0) # alpha = 0 is ridge
coef(cv, s = "lambda.min")
# Structural collinearity: center before squaring
mtcars$wt_c <- mtcars$wt - mean(mtcars$wt)
vif(lm(mpg ~ wt_c + I(wt_c^2), data = mtcars)) # lower than with raw wt and wt^2
When It Matters
Inference (which variables matter, coefficient interpretation): yes, address it.
Prediction on data with the same correlation structure: often not a problem.
Common Pitfalls
Dropping variables only by p-value when they are jointly important.
Using VIF thresholds mechanically; consider the research question.
Interpreting ridge coefficients like unbiased OLS estimates.
Interview Tips
Define the consequences (unstable estimates, inflated SEs) and VIF.
Distinguish inference from prediction goals.
26
How does k-means clustering work and how do you implement it in R?
KnownRevise
k-means clustering partitions observations into k groups by repeatedly assigning each point to the nearest of k cluster centres (centroids) and then moving each centre to the mean of its assigned points, until assignments stop changing; it minimizes the total within-cluster sum of squared distances. In R it is run with the base kmeans() function on scaled numeric data, using several random starts (nstart) because results depend on initialization, and the number of clusters is chosen with the elbow method, silhouette widths or the gap statistic (for example via factoextra).
Read full answerHide full answer
Applies toR 4.4+ (stats), cluster 2.1+, factoextra 1.0.7
Algorithm (Lloyd’s)
Choose k and initial centres (random points; R’s default is the Hartigan-Wong algorithm).
Assign each point to its nearest centre (Euclidean distance).
Recompute each centre as the mean of its points.
Repeat steps 2-3 until assignments stop changing or the iteration limit is reached.
Code Example
library(cluster)
X <- scale(iris[, 1:4]) # k-means uses distances: scale first
set.seed(42)
km <- kmeans(X, centers = 3, nstart = 25) # 25 random starts, keep the best
km$size # cluster sizes
km$centers # centres on the scaled data
km$tot.withinss # total within-cluster sum of squares
table(cluster = km$cluster, species = iris$Species) # compare with known species
# Elbow method: within-cluster SS for k = 1..8
set.seed(42)
wss <- sapply(1:8, function(k) kmeans(X, centers = k, nstart = 25)$tot.withinss)
plot(1:8, wss, type = "b", xlab = "k", ylab = "Total within-cluster SS")
# Average silhouette width (higher is better)
sil <- silhouette(km$cluster, dist(X))
mean(sil[, "sil_width"])
# Optional visual helpers
# factoextra::fviz_nbclust(X, kmeans, method = "silhouette")
# factoextra::fviz_cluster(km, data = X)
Strengths and Limitations
Strengths
Limitations
Simple, fast, scales to large data
k must be chosen
Easy to interpret centres
Assumes roughly spherical, similar-size clusters
Works well with scaled numeric features
Sensitive to outliers and initialization
Numeric data only (use k-modes / PAM with Gower for mixed types)
Common Pitfalls
Not scaling variables, so large-range features dominate.
nstart = 1, giving a poor local optimum.
Forcing k-means on non-spherical clusters; try DBSCAN or hierarchical clustering.
Interview Tips
Explain assign-update iterations and the objective.
Mention nstart, scaling and choosing k.
27
How is the e1071 package used for machine learning in R?
KnownRevise
e1071 is a long-standing CRAN package from TU Wien’s statistics department that provides several machine learning and statistics tools, most notably svm(), an interface to the LIBSVM library for support vector machines (classification, regression and novelty detection), together with naiveBayes(), tune() for grid-search hyperparameter tuning with cross-validation, fuzzy c-means clustering (cmeans()), and utilities such as skewness() and kurtosis(). It is often used directly or as the engine behind caret and tidymodels SVM models.
Read full answerHide full answer
Applies toR 4.4+, e1071 1.7
Main Functions
Function
Purpose
svm()
Support vector machines: C-classification, nu-classification, eps-regression, one-classification
tune(), tune.svm()
Grid search with cross-validation
naiveBayes()
Naive Bayes classifier
cmeans()
Fuzzy c-means clustering
skewness(), kurtosis()
Distribution shape
bclust()
Bagged clustering
hamming.distance()
Distance for binary vectors
Code Example
library(e1071)
set.seed(42)
idx <- sample(nrow(iris), 120)
train <- iris[idx, ]
test <- iris[-idx, ]
# SVM with a radial kernel (features are scaled by default: scale = TRUE)
model <- svm(Species ~ ., data = train, kernel = "radial", cost = 1, gamma = 0.25,
probability = TRUE)
pred <- predict(model, test)
mean(pred == test$Species) # test accuracy
table(predicted = pred, actual = test$Species)
# Tune cost and gamma with 10-fold cross-validation
set.seed(42)
tuned <- tune.svm(Species ~ ., data = train, cost = 2^(-1:4), gamma = 2^(-4:0))
tuned$best.parameters
best <- tuned$best.model
# Probabilities
attr(predict(model, test[1:3, ], probability = TRUE), "probabilities")
# Naive Bayes
nb <- naiveBayes(Species ~ ., data = train)
mean(predict(nb, test) == test$Species)
Common Pitfalls
Forgetting probability = TRUE when fitting if you need probabilities later.
Large data: kernel SVMs scale poorly with many rows; consider linear models or LiblineaR.
Untuned cost and gamma, which can underfit or overfit badly.
Interview Tips
Name svm() (LIBSVM) and tune() as the main uses.
Mention that caret’s "svmRadial" uses kernlab, while tidymodels can use kernlab or LiblineaR, as alternatives to e1071.
28
How do you implement a decision tree in R?
KnownRevise
Decision trees in R are usually built with the rpart package, which grows a CART tree with rpart(y ~ ., data, method = "class") for classification or method = "anova" for regression, controls growth with rpart.control() (cp, minsplit, maxdepth), and prunes it back with prune() using the complexity parameter that minimizes cross-validated error from printcp(); rpart.plot draws readable diagrams. Alternatives are party::ctree() (conditional inference trees with statistical stopping rules) and tidymodels’ decision_tree() with the rpart engine.
Complexity parameter: a split must improve fit by this fraction
minsplit
Minimum observations to attempt a split (default 20)
minbucket
Minimum observations in a leaf
maxdepth
Maximum depth
xval
Number of cross-validation folds (default 10)
Code Example
library(rpart)
library(rpart.plot)
set.seed(42)
idx <- sample(nrow(iris), 105)
train <- iris[idx, ]
test <- iris[-idx, ]
tree <- rpart(Species ~ ., data = train, method = "class",
control = rpart.control(cp = 0.001, minsplit = 10))
printcp(tree) # cp table with cross-validated error (xerror)
best_cp <- tree$cptable[which.min(tree$cptable[, "xerror"]), "CP"]
pruned <- prune(tree, cp = best_cp)
rpart.plot(pruned, type = 4, extra = 104) # class, probabilities and % of observations
pred <- predict(pruned, test, type = "class")
mean(pred == test$Species) # test accuracy
pruned$variable.importance
# Regression tree
reg <- rpart(mpg ~ ., data = mtcars, method = "anova", control = rpart.control(minsplit = 5))
predict(reg, mtcars[1:3, ])
Reading the Tree
Each node shows the predicted class, class probabilities and share of observations.
Splits read as rules, e.g. “Petal.Length < 2.5 → setosa”.
variable.importance sums the improvement of each variable across splits, including surrogate splits.
Common Pitfalls
Overfitting with a very small cp; prune with cross-validation.
Instability: small data changes can change the tree; use random forests for accuracy.
Using predict() without type = "class", which returns probabilities.
Interview Tips
Explain cp and pruning by cross-validated error.
Mention rpart.plot and that rpart handles missing values with surrogate splits.
29
What are the differences between random forest and gradient boosting in R?
KnownRevise
Random forests (in R, ranger or randomForest) build many deep trees independently on bootstrap samples with random feature subsets and average them, reducing variance; gradient boosting (in R, xgboost, lightgbm or gbm) builds shallow trees one after another, each fitted to the errors of the current model with a small learning rate, reducing bias. Random forests are robust and need little tuning, while well-tuned boosting is often more accurate on tabular data but needs careful tuning and early stopping; both are available through tidymodels’ rand_forest() and boost_tree().
mtcars is tiny, so the comparison here only illustrates the code; differences appear on larger data.
Common Pitfalls
Boosting without early stopping or validation.
Comparing a tuned model with an untuned one.
Forgetting that xgboost needs numeric matrices (dummy-encode factors).
Interview Tips
Frame it as variance vs bias reduction and parallel vs sequential.
Name the R packages and their key parameters.
30
How do you train a support vector machine in R?
KnownRevise
A support vector machine (SVM) in R is usually trained with e1071::svm() (LIBSVM) or kernlab::ksvm(), choosing a kernel (linear, radial, polynomial), scaling the features (done by default in svm()), and tuning the cost parameter C and kernel parameters such as gamma with cross-validation, for example with tune.svm(), caret’s train(method = "svmRadial") or tidymodels’ svm_rbf() with tune_grid(). The SVM finds the boundary with the widest margin between classes; the kernel lets it model non-linear boundaries.
Read full answerHide full answer
Applies toR 4.4+, e1071 1.7, kernlab 0.9-33, tidymodels 1.2
Penalty for misclassification: high = narrow margin, risk of overfitting
gamma
RBF width: high = very local, wiggly boundary
degree
Polynomial kernel degree
class.weights
Weights for imbalanced classes
probability
Fit Platt scaling to produce probabilities
Code Example
library(tidymodels)
set.seed(42)
split <- initial_split(iris, prop = 0.8, strata = Species)
train <- training(split)
test <- testing(split)
rec <- recipe(Species ~ ., data = train) |> step_normalize(all_numeric_predictors())
svm_spec <- svm_rbf(cost = tune(), rbf_sigma = tune()) |>
set_engine("kernlab") |>
set_mode("classification")
wf <- workflow() |> add_recipe(rec) |> add_model(svm_spec)
set.seed(42)
folds <- vfold_cv(train, v = 5, strata = Species)
grid <- grid_regular(cost(), rbf_sigma(), levels = 5)
res <- tune_grid(wf, resamples = folds, grid = grid, metrics = metric_set(accuracy, roc_auc))
show_best(res, metric = "accuracy", n = 3)
final <- finalize_workflow(wf, select_best(res, metric = "accuracy")) |> last_fit(split)
collect_metrics(final) # test accuracy and ROC AUC
# The same with e1071 directly
library(e1071)
fit <- svm(Species ~ ., data = train, kernel = "radial", cost = 4, gamma = 0.25)
mean(predict(fit, test) == test$Species)
Choosing a Kernel
Kernel
When
Linear
Many features (text), roughly linear separation, large data
Radial (RBF)
Default for non-linear problems
Polynomial
Known polynomial interactions
Common Pitfalls
Not scaling features (kernlab via recipes needs explicit normalization).
Tuning C and gamma on the test set.
Kernel SVMs on very large datasets (training time grows faster than linearly with rows).
Interview Tips
Explain the margin, C and gamma in one sentence each.
Mention cross-validated tuning with tune.svm() or tidymodels.
31
How is a confusion matrix used in R?
KnownRevise
A confusion matrix is a table that compares a classifier’s predicted classes with the actual classes, counting true positives, true negatives, false positives and false negatives (or every predicted-actual pair for multi-class problems), and it is the basis for metrics such as accuracy, precision, recall (sensitivity), specificity and F1. In R you can build one with base table(predicted, actual), with caret::confusionMatrix(), which also reports accuracy with a confidence interval, kappa, sensitivity and specificity, or with yardstick::conf_mat() in tidymodels.
Read full answerHide full answer
Applies toR 4.4+, caret 6.0-94, yardstick 1.3+
Layout (Binary)
Actual positive
Actual negative
Predicted positive
TP
FP
Predicted negative
FN
TN
Derived Metrics
Metric
Formula
Question answered
Accuracy
(TP + TN) / total
How often correct overall?
Precision
TP / (TP + FP)
When it says positive, how often right?
Recall / sensitivity
TP / (TP + FN)
How many positives were found?
Specificity
TN / (TN + FP)
How many negatives were correctly rejected?
F1
2 · precision · recall / (precision + recall)
Balance of precision and recall
Kappa
Agreement beyond chance
Useful with imbalance
Code Example
actual <- factor(c("yes", "yes", "yes", "no", "no", "no", "no", "yes", "no", "no"),
levels = c("yes", "no"))
predicted <- factor(c("yes", "no", "yes", "no", "yes", "no", "no", "yes", "no", "no"),
levels = c("yes", "no"))
table(Predicted = predicted, Actual = actual)
# Actual
# Predicted yes no
# yes 3 1
# no 1 5
library(caret)
cm <- confusionMatrix(predicted, actual, positive = "yes", mode = "everything")
cm$overall["Accuracy"] # 0.8
cm$byClass[c("Sensitivity", "Specificity", "Precision", "F1")]
# Sensitivity 0.75, Specificity 0.8333, Precision 0.75, F1 0.75
library(yardstick)
df <- data.frame(truth = actual, estimate = predicted)
conf_mat(df, truth, estimate)
metrics <- metric_set(accuracy, precision, recall, f_meas)
metrics(df, truth = truth, estimate = estimate) # yardstick treats the first level ("yes") as the event
Multi-Class
Rows and columns for every class; the diagonal holds correct predictions.
Per-class precision and recall, averaged as macro (equal weight per class) or weighted by class size.
Common Pitfalls
The positive class: caret and yardstick use the first factor level by default; set positive = or event_level =.
Accuracy on imbalanced data hides poor minority recall.
Swapping the predicted and actual arguments, which transposes the matrix.
Interview Tips
Compute precision and recall by hand from a 2×2 table.
Mention the positive-class level issue in R.
32
How do you calculate ROC and AUC in R?
KnownRevise
An ROC curve plots the true positive rate against the false positive rate as the classification threshold varies, and the AUC (area under it) summarizes ranking quality: 1.0 is perfect, 0.5 is random guessing, and it equals the probability that a randomly chosen positive gets a higher score than a randomly chosen negative. In R you compute them from predicted probabilities with pROC::roc() and auc() (with confidence intervals via ci.auc()), or yardstick::roc_curve() and roc_auc() in tidymodels; ROCR is an older alternative.
Read full answerHide full answer
Applies toR 4.4+, pROC 1.18, yardstick 1.3+
Code Example
library(pROC)
# Logistic regression for manual (1) vs automatic (0) transmission
set.seed(42)
idx <- sample(nrow(mtcars), 22)
train <- mtcars[idx, ]
test <- mtcars[-idx, ]
fit <- glm(am ~ wt + hp, data = train, family = binomial)
prob <- predict(fit, test, type = "response") # probabilities, not classes
roc_obj <- roc(response = test$am, predictor = prob, levels = c(0, 1), direction = "<")
auc(roc_obj) # area under the curve
ci.auc(roc_obj) # DeLong confidence interval
plot(roc_obj, print.auc = TRUE, legacy.axes = TRUE) # x axis as 1 - specificity
coords(roc_obj, "best", ret = c("threshold", "sensitivity", "specificity")) # Youden's J
# Tiny hand-checkable example
scores <- c(0.9, 0.8, 0.7, 0.4, 0.3, 0.2)
labels <- c(1, 1, 0, 1, 0, 0)
auc(roc(labels, scores, levels = c(0, 1), direction = "<", quiet = TRUE))
# 8 of 9 positive-negative pairs ranked correctly: AUC = 0.889
# tidymodels / yardstick
library(yardstick)
df <- data.frame(truth = factor(labels, levels = c(1, 0)), .pred_1 = scores)
roc_auc(df, truth, .pred_1) # 0.889; first level "1" is the event
roc_curve(df, truth, .pred_1) # thresholds, sensitivity, specificity
Interpreting AUC
AUC
Rough meaning
0.5
No better than random
0.7-0.8
Acceptable
0.8-0.9
Good
> 0.9
Excellent (check for leakage)
ROC vs Precision-Recall
ROC AUC can look high on very imbalanced data even when precision is poor.
Use precision-recall curves (yardstick::pr_auc(), PRROC) when positives are rare.
Common Pitfalls
Passing predicted classes instead of probabilities.
Wrong event level or direction, which flips AUC to 1 − AUC.
Choosing the threshold on the test set.
Interview Tips
Explain AUC as the probability of ranking a positive above a negative.
Mention levels and direction in pROC to avoid silent flips.
33
What cross-validation techniques are available in R?
KnownRevise
R supports all the common cross-validation schemes: k-fold and repeated k-fold, stratified folds, leave-one-out (LOOCV), the bootstrap, Monte Carlo (repeated random) splits, group-based folds that keep related rows together, time-series rolling-origin splits, and nested cross-validation. They are available in tidymodels’ rsample (vfold_cv(), loo_cv(), bootstraps(), mc_cv(), group_vfold_cv(), rolling_origin(), nested_cv()) and in caret through trainControl(method = ...), and resampling results are then summarized across folds to estimate performance and tune models.
Read full answerHide full answer
Applies toR 4.4+, rsample 1.2+, caret 6.0-94
Techniques
Technique
rsample
caret trainControl(method = )
Use
k-fold
vfold_cv(v = 10)
"cv"
Default choice
Repeated k-fold
vfold_cv(v = 10, repeats = 3)
"repeatedcv"
More stable estimates
Stratified
vfold_cv(strata = y)
Stratified by default for factors
Classification, imbalance
Leave-one-out
loo_cv()
"LOOCV"
Very small data
Bootstrap
bootstraps(times = 25)
"boot"
Variance estimates
Monte Carlo
mc_cv(prop = 0.8, times = 25)
"LGOCV"
Random repeated splits
Grouped
group_vfold_cv(group = id)
index = custom folds
Repeated measures, patients
Time series
rolling_origin(), sliding_period()
"timeslice"
Forecasting
Nested
nested_cv()
(manual)
Unbiased tuning + evaluation
Code Example
library(tidymodels)
set.seed(42)
folds <- vfold_cv(mtcars, v = 5, repeats = 2)
lm_wf <- workflow() |> add_formula(mpg ~ wt + hp) |> add_model(linear_reg())
res <- fit_resamples(lm_wf, resamples = folds, metrics = metric_set(rmse, rsq))
collect_metrics(res) # mean RMSE and R-squared across 10 fits
# Grouped CV: all rows of one group stay in the same fold
df <- transform(mtcars, brand = sub(" .*", "", rownames(mtcars)))
set.seed(42)
gfolds <- group_vfold_cv(df, group = brand, v = 5)
# Rolling origin for time series
ts_df <- data.frame(t = 1:48, y = cumsum(rnorm(48)))
roll <- rolling_origin(ts_df, initial = 24, assess = 6, skip = 5, cumulative = TRUE)
nrow(roll) # number of resamples
# caret equivalent
library(caret)
ctrl <- trainControl(method = "repeatedcv", number = 5, repeats = 2)
set.seed(42)
train(mpg ~ wt + hp, data = mtcars, method = "lm", trControl = ctrl)$results
Choosing a Scheme
Independent rows: 5- or 10-fold, repeated for small data.
Classification: stratify by the outcome.
Several rows per entity: grouped folds.
Time-ordered data: rolling origin, never shuffled folds.
Reporting after tuning: nested CV or a separate test set.
Common Pitfalls
Preprocessing outside the resampling loop (leakage); put it in a recipe or caret’s preProcess.
Shuffled folds on time series.
Forgetting set.seed(), so folds differ between models.
Interview Tips
Match the scheme to the data structure.
Mention nested CV for unbiased estimates after tuning.
34
How do you tune hyperparameters in R?
KnownRevise
Hyperparameters in R are tuned by evaluating candidate settings with resampling and picking the best: in tidymodels you mark parameters with tune() in a model specification, create cross-validation folds with rsample, and search with tune_grid() (regular or random grids) or tune_bayes() (Bayesian optimization), then finalize the workflow with select_best(); in caret you pass a tuneGrid or tuneLength to train() with a trainControl() resampling scheme. Racing methods (finetune::tune_race_anova()) drop poor candidates early to save time.
Read full answerHide full answer
Applies toR 4.4+, tidymodels 1.2 (tune, dials, finetune), caret 6.0-94
Search Strategies
Strategy
tidymodels
caret
Regular grid
grid_regular()
tuneGrid = expand.grid(...)
Random / space-filling
grid_random(), grid_space_filling()
search = "random" in trainControl()
Bayesian optimization
tune_bayes()
—
Racing
finetune::tune_race_anova()
adaptive_cv
Simulated annealing
finetune::tune_sim_anneal()
—
Code Example
library(tidymodels)
set.seed(42)
split <- initial_split(iris, prop = 0.8, strata = Species)
train <- training(split)
folds <- vfold_cv(train, v = 5, strata = Species)
rf_spec <- rand_forest(mtry = tune(), min_n = tune(), trees = 500) |>
set_engine("ranger") |>
set_mode("classification")
wf <- workflow() |> add_formula(Species ~ .) |> add_model(rf_spec)
grid <- grid_regular(mtry(range = c(1, 4)), min_n(range = c(2, 20)), levels = 4) # 16 candidates
set.seed(42)
res <- tune_grid(wf, resamples = folds, grid = grid, metrics = metric_set(accuracy, roc_auc))
show_best(res, metric = "roc_auc", n = 3)
autoplot(res) # performance across the grid
best <- select_best(res, metric = "roc_auc") # or select_by_one_std_err() for a simpler model
final_fit <- finalize_workflow(wf, best) |> last_fit(split)
collect_metrics(final_fit) # performance on the held-out test set
# Bayesian optimization continuing from the grid results
set.seed(42)
bayes <- tune_bayes(wf, resamples = folds, initial = res, iter = 10,
param_info = extract_parameter_set_dials(wf) |>
update(mtry = mtry(c(1, 4))),
metrics = metric_set(roc_auc))
What is overfitting and how do you prevent it in R models?
KnownRevise
Overfitting happens when a model learns the noise and quirks of its training data instead of the general pattern, so it performs very well on training data but poorly on new data; it is detected by a large gap between training and validation or test performance. In R you prevent it by evaluating with cross-validation or a test set (rsample, caret), keeping models simple (fewer features, pruned trees with rpart’s cp), adding regularization (glmnet ridge and lasso, min_n in trees, weight decay or dropout in neural networks), using ensembles such as random forests, stopping boosting early, and getting more data.
Read full answerHide full answer
Applies toR 4.4+, glmnet 4.1+, rpart 4.1.23, tidymodels 1.2
Remedies
Remedy
R tools
Honest evaluation
initial_split(), vfold_cv(), caret trainControl()
Simpler model
Fewer predictors, lower polynomial degree
Regularization
glmnet (alpha = 0 ridge, 1 lasso), penalty in parsnip
Tree pruning
rpartcp, prune(), min_n, tree_depth
Early stopping
xgboost early_stopping_rounds, keras callbacks
Ensembles
ranger, bagging
Feature selection inside CV
recipes filters, caret rfe()
More / augmented data
Collect more, data augmentation for images
Code Example
set.seed(42)
idx <- sample(nrow(mtcars), 22)
train <- mtcars[idx, ]
test <- mtcars[-idx, ]
rmse <- function(obs, pred) sqrt(mean((obs - pred)^2))
# Polynomial degree: training error keeps falling, test error rises again
for (d in c(1, 3, 6)) {
fit <- lm(mpg ~ poly(hp, d), data = train)
cat(sprintf("degree %d: train %.2f test %.2f\n", d,
rmse(train$mpg, predict(fit)), rmse(test$mpg, predict(fit, test))))
}
# Lasso: cross-validation chooses how strongly to shrink coefficients
library(glmnet)
x_train <- as.matrix(train[, -1])
x_test <- as.matrix(test[, -1])
set.seed(42)
cv <- cv.glmnet(x_train, train$mpg, alpha = 1, nfolds = 5)
coef(cv, s = "lambda.1se") # several coefficients shrunk to exactly 0
rmse(test$mpg, predict(cv, x_test, s = "lambda.1se"))
# Pruned decision tree
library(rpart)
tree <- rpart(mpg ~ ., data = train, control = rpart.control(cp = 0, minsplit = 4))
pruned <- prune(tree, cp = tree$cptable[which.min(tree$cptable[, "xerror"]), "CP"])
c(full = rmse(test$mpg, predict(tree, test)), pruned = rmse(test$mpg, predict(pruned, test)))
Diagnosing
Plot learning curves (performance vs training size) and validation curves (performance vs complexity).
Large train-test gap = overfitting; both errors high = underfitting.
Common Pitfalls
Reporting training accuracy.
Data leakage that hides overfitting (preprocessing on all data, duplicate rows).
Repeatedly checking the test set while tuning.
Interview Tips
Define overfitting with the train-test gap and the bias-variance trade-off.
Give one R-specific remedy per model family.
36
How do you create a plot with ggplot2?
KnownRevise
A ggplot2 plot is built in layers following the grammar of graphics: start with ggplot(data, aes(x = ..., y = ...)) to map variables to aesthetics, add one or more geometric layers such as geom_point(), geom_line() or geom_bar() with +, and refine the result with scales, facets, labels (labs()) and themes. Each component is independent, so the same data and mappings can be shown in many ways by swapping layers, and finished plots are saved with ggsave().
library(ggplot2)
p <- ggplot(mtcars, aes(x = wt, y = mpg, colour = factor(cyl))) +
geom_point(size = 3, alpha = 0.8) +
geom_smooth(aes(group = 1), method = "lm", se = TRUE, colour = "grey30") + # one overall trend
scale_colour_brewer(palette = "Dark2") +
labs(title = "Heavier cars use more fuel",
x = "Weight (1000 lbs)", y = "Miles per gallon", colour = "Cylinders") +
theme_minimal(base_size = 12) +
theme(legend.position = "bottom")
p
ggsave("mpg_vs_weight.png", p, width = 7, height = 5, dpi = 300)
# Other common plots
ggplot(iris, aes(Species, Sepal.Length, fill = Species)) + geom_boxplot(show.legend = FALSE)
ggplot(mtcars, aes(mpg)) + geom_histogram(binwidth = 2, fill = "steelblue", colour = "white")
ggplot(mtcars, aes(factor(cyl))) + geom_bar() + labs(x = "Cylinders", y = "Count")
Aesthetics: Mapping vs Setting
Mapped inside aes(): varies with data (aes(colour = cyl)) and gets a legend.
Set outside aes(): fixed value (geom_point(colour = "red")).
Common Pitfalls
Putting a fixed colour inside aes() (aes(colour = "red")), which creates a legend named “red”.
Using %>%/|> between layers instead of +.
Numeric variables mapped to colour when they should be discrete; wrap in factor().
geom_bar() counts rows; use geom_col() for pre-computed values.
Interview Tips
Explain the layers: data, aesthetics, geoms, then scales/facets/themes.
Mention mapping vs setting aesthetics.
37
What is the purpose of facets in ggplot2?
KnownRevise
Facets in ggplot2 split one plot into a grid of small panels, one for each value (or combination of values) of one or two categorical variables, all with the same axes and geoms, so you can compare patterns across groups side by side instead of overloading one panel with colours.facet_wrap(~ var) wraps panels for one variable into rows and columns, and facet_grid(rows = vars(a), cols = vars(b)) makes a matrix for two variables; the scales argument lets axes vary by panel when ranges differ a lot.
Read full answerHide full answer
Applies toR 4.4+, ggplot2 3.5+
facet_wrap vs facet_grid
Function
Layout
Best for
facet_wrap(~ class)
Wraps panels into a 2-D layout
One variable with many levels
facet_grid(rows = vars(a), cols = vars(b))
Rows × columns matrix
Two variables, or aligned comparisons
facet_grid(a ~ b)
Formula form of the same
Key Arguments
Argument
Effect
scales = "free", "free_x", "free_y"
Axes vary per panel (comparisons get harder)
ncol, nrow
Layout of facet_wrap()
labeller = label_both
Panel labels like “cyl: 4”
space = "free"
Panel size proportional to data range (facet_grid)
margins = TRUE
Add an “all” panel (facet_grid)
Code Example
library(ggplot2)
# One variable: a panel per number of cylinders
ggplot(mtcars, aes(wt, mpg)) +
geom_point() +
geom_smooth(method = "lm", se = FALSE) +
facet_wrap(~ cyl, labeller = label_both)
# Two variables: transmission (rows) by gears (columns)
ggplot(mtcars, aes(hp, mpg)) +
geom_point() +
facet_grid(rows = vars(am), cols = vars(gear), labeller = label_both)
# Free y scales when ranges differ by group
ggplot(iris, aes(Sepal.Length)) +
geom_histogram(bins = 15) +
facet_wrap(~ Species, ncol = 1, scales = "free_y")
# Show the whole data in grey behind each panel for context
ggplot(mtcars, aes(wt, mpg)) +
geom_point(data = transform(mtcars, cyl = NULL), colour = "grey80") +
geom_point(colour = "steelblue") +
facet_wrap(~ cyl)
When to Use Facets
Comparing a relationship across groups (regions, segments, experiments).
Too many groups for one panel’s colour legend.
Showing distributions per category.
Common Pitfalls
Too many panels to read; filter or group levels first.
Free scales that make panels look similar when the values differ a lot; label clearly.
Faceting by a continuous variable; bin it first with cut() or cut_number().
Interview Tips
Contrast facet_wrap() and facet_grid().
Mention scales = "free" and its trade-off.
38
How do you create interactive visualizations in R?
KnownRevise
Interactive visualizations in R are created with htmlwidgets-based packages that render in the browser, RStudio viewer, R Markdown or Quarto: plotly turns ggplot2 plots into interactive charts with ggplotly() or builds them directly with plot_ly(), leaflet makes interactive maps, DT makes searchable tables, dygraphs handles time series, visNetwork draws networks, and ggiraph adds tooltips to ggplot2; for full applications with inputs and reactive updates, shiny (and bslib for layout) builds web apps. Static ggplot2 is best for reports, interactivity for exploration and dashboards.
Exploration and HTML reports: plotly, DT, leaflet in Quarto or R Markdown.
Dashboards with user inputs: Shiny (or Quarto dashboards with shiny runtime).
Static publications: plain ggplot2.
Common Pitfalls
Widgets don’t render in PDF or Word output; use static plots there.
Very large data in widgets makes pages slow; aggregate or sample.
ggplotly() doesn’t support every ggplot2 feature perfectly; check the result.
Interview Tips
Name plotly, leaflet, DT and Shiny with one use each.
Explain htmlwidgets vs Shiny (no server vs reactive server).
39
How do you use and customize themes in ggplot2?
KnownRevise
Themes in ggplot2 control everything in a plot that isn’t data, such as fonts, background, grid lines, axis text, legend position and panel borders; you apply a complete built-in theme like theme_minimal(), theme_bw() or theme_classic() and then adjust individual elements with theme(), using element functions element_text(), element_line(), element_rect() and element_blank(). A custom theme can be saved as a function and reused, or set for a whole session with theme_set(), which keeps a team’s charts consistent.
Read full answerHide full answer
Applies toR 4.4+, ggplot2 3.5+
Built-in Themes
Theme
Look
theme_grey()
Default: grey panel, white grid
theme_bw()
White panel, grey grid, border
theme_minimal()
No background or border, light grid
theme_classic()
Axis lines, no grid (journal style)
theme_light(), theme_dark()
Light or dark panels
theme_void()
Nothing but data (maps, diagrams)
Extension packages: ggthemes (styles of well-known publications), hrbrthemes, bbplot.
library(ggplot2)
base <- ggplot(mtcars, aes(wt, mpg, colour = factor(cyl))) +
geom_point(size = 3) +
labs(title = "Fuel economy by weight", x = "Weight (1000 lbs)", y = "MPG", colour = "Cylinders")
base + theme_minimal()
base + theme_bw(base_size = 13) +
theme(plot.title = element_text(face = "bold", hjust = 0),
panel.grid.minor = element_blank(),
legend.position = "bottom",
axis.title = element_text(colour = "grey30"))
# A reusable house theme
theme_company <- function(base_size = 12) {
theme_minimal(base_size = base_size) %+replace%
theme(plot.title = element_text(face = "bold", size = rel(1.3), hjust = 0,
margin = margin(b = 8)),
panel.grid.minor = element_blank(),
panel.grid.major.x = element_blank(),
legend.position = "top",
legend.justification = "left")
}
base + theme_company()
theme_set(theme_company()) # default for all later plots in the session
Common Pitfalls
Calling a complete theme after theme(), which overwrites your tweaks; add complete themes first.
Using theme() to change data colours; use scale_* functions instead.
Tiny fonts in saved images; set base_size and check with ggsave() dimensions.
Interview Tips
Explain complete themes vs theme() tweaks and element functions.
Mention theme_set() and a reusable theme function for consistency.
40
What is the best way to visualize high-dimensional data in R?
KnownRevise
There is no single best method, but the usual approach is to reduce the data to two or three dimensions and plot it: principal component analysis (prcomp()) for linear structure and an interpretable view of variables (biplots), and non-linear embeddings such as t-SNE (Rtsne) or UMAP (uwot) for clusters and local structure; alongside these, pairwise scatterplot matrices (GGally::ggpairs()), correlation heatmaps (corrplot), heatmaps with clustering (pheatmap), and parallel coordinate plots show the original variables directly. Scale the data first, and remember that t-SNE and UMAP distances between clusters are not meaningful.
Interpretability: PCA (axes are combinations of known variables).
Clusters in complex data (single-cell, embeddings): UMAP or t-SNE.
Few variables: scatterplot matrix.
Variable relationships: correlation heatmap.
Common Pitfalls
Not scaling variables before PCA, t-SNE or UMAP.
Reading cluster sizes or distances between clusters in t-SNE/UMAP as meaningful.
Drawing conclusions from one t-SNE run; try several perplexities and seeds.
Interview Tips
Mention PCA first for interpretability, then UMAP/t-SNE for structure.
Warn about the limits of non-linear embeddings.
41
How do you load a CSV file in R and summarize its statistics?
KnownRevise
To load a CSV file in R and summarize it, read it with readr::read_csv() (or base read.csv()), check its structure with glimpse() or str(), get quick per-column statistics with summary(), and build richer summaries with dplyr, such as counts, means, medians, standard deviations and missing values per column or per group; packages like skimr produce a full overview in one call. Always confirm that column types were read correctly, because numbers stored as text or dates read as strings change every summary.
How do you filter and sort a data frame with dplyr on several conditions?
KnownRevise
With dplyr you filter rows on several conditions with filter(), where comma-separated conditions are combined with AND and | expresses OR, using helpers such as %in%, between() and !is.na(); you then sort with arrange(), listing columns in priority order and wrapping a column in desc() for descending order. Chained with the pipe, the steps read top to bottom like the question you are asking of the data.
Read full answerHide full answer
Applies toR 4.4+, dplyr 1.1+
Condition Building Blocks
Need
Code
AND
filter(df, a > 1, b == "x") or a > 1 & b == "x"
OR
filter(df, a > 1 | b == "x")
In a set
cyl %in% c(4, 6)
Range (inclusive)
between(mpg, 20, 30)
Not missing
!is.na(Ozone)
Text match
grepl("Merc", name) or stringr::str_detect(name, "^Merc")
Per-group condition
group_by(g) |> filter(x == max(x))
Across columns
filter(if_all(c(a, b), ~ .x > 0)), if_any()
Code Example
library(dplyr)
library(tibble)
cars <- mtcars |> rownames_to_column("model")
# Efficient, powerful manual cars, sorted by mpg (desc) then hp (asc)
cars |>
filter(am == 1, # manual
mpg > 20,
hp >= 90 | cyl == 6) |> # either condition
arrange(desc(mpg), hp) |>
select(model, mpg, hp, cyl)
# Lotus Europa 30.4, Porsche 914-2 26.0, Datsun 710 22.8, Volvo 142E 21.4, Mazda RX4 21.0, Mazda RX4 Wag 21.0
# Set membership, ranges and missing values
airquality |>
filter(Month %in% c(7, 8), between(Temp, 80, 90), !is.na(Ozone)) |>
arrange(Month, desc(Ozone)) |>
head()
# Best car per cylinder group, sorted
cars |>
group_by(cyl) |>
filter(mpg == max(mpg)) |>
ungroup() |>
arrange(cyl)
# Conditions on several columns at once
iris |> filter(if_all(starts_with("Sepal"), ~ .x > 3)) |> nrow() # rows where both sepal measures > 3
Common Pitfalls
filter(x == NA) returns nothing; use is.na(x).
== with floating point values; use between() or near().
Forgetting ungroup() after grouped filtering.
Using && or || (scalar operators) instead of & and |.
Interview Tips
Explain that commas mean AND and that filter() drops NA rows.
Show desc() and multiple sort keys.
43
How do you write an R function for Euclidean distance between two points?
KnownRevise
The Euclidean distance between two points is the square root of the sum of squared differences of their coordinates, sqrt(Σ (a_i − b_i)²), and in R it is a one-line vectorized function, sqrt(sum((a - b)^2)), that works for points in any number of dimensions; good practice adds input checks for numeric vectors of equal length. For many points at once, the built-in dist() returns all pairwise distances, so you rarely need a loop.
Read full answerHide full answer
Applies toR 4.4+ (base and stats)
Code Example
euclidean_distance <- function(a, b) {
# Check the inputs: numeric vectors of the same length without missing values
stopifnot(is.numeric(a), is.numeric(b))
if (length(a) != length(b)) stop("Points must have the same number of dimensions.")
if (anyNA(a) || anyNA(b)) stop("Points must not contain missing values.")
sqrt(sum((a - b)^2))
}
euclidean_distance(c(0, 0), c(3, 4)) # 5
euclidean_distance(c(1, 2, 3), c(4, 6, 8)) # sqrt(9 + 16 + 25) = 7.071068
euclidean_distance(5, 2) # 3 (one dimension)
# Distance from one point to every row of a matrix (vectorized)
distances_to <- function(point, m) sqrt(rowSums(sweep(m, 2, point)^2))
m <- matrix(c(0, 0, 3, 4, 6, 8), ncol = 2, byrow = TRUE)
distances_to(c(0, 0), m) # 0 5 10
# All pairwise distances with base R
dist(m) # 5, 10 and 5 between the three points
as.matrix(dist(m))[1, 3] # 10
# Scaling matters when features use different units
scaled <- scale(mtcars[, c("mpg", "hp")])
euclidean_distance(scaled["Mazda RX4", ], scaled["Datsun 710", ])
Related Distances
Distance
Formula
R
Euclidean
sqrt(Σ (a − b)²)
dist(x, method = "euclidean")
Manhattan
Σ |a − b|
dist(x, method = "manhattan")
Minkowski
(Σ |a − b|^p)^(1/p)
dist(x, method = "minkowski", p = 3)
Cosine
1 − a·b / (‖a‖‖b‖)
1 - sum(a * b) / sqrt(sum(a^2) * sum(b^2))
Common Pitfalls
Unequal lengths: R recycles the shorter vector silently, giving wrong answers; check lengths.
Comparing features with different scales without standardizing.
Using loops over points instead of vectorized code or dist().
Interview Tips
Write the vectorized one-liner and add validation.
Mention dist() and scaling for real use (k-NN, clustering).
44
How do you write an R script that performs k-means clustering?
KnownRevise
A k-means clustering script in R loads and selects numeric features, scales them, chooses the number of clusters with the elbow method or average silhouette width, runs kmeans() with a fixed seed and many random starts (nstart), attaches the cluster labels to the data, profiles each cluster with grouped summaries, and visualizes the result, typically on the first two principal components. Scaling and multiple starts are the two settings that most affect result quality.
Read full answerHide full answer
Applies toR 4.4+, dplyr 1.1+, ggplot2 3.5+, cluster 2.1+
Code Example
library(dplyr)
library(ggplot2)
library(cluster)
# 1. Data: numeric features only
data <- USArrests # crime rates per US state
X <- scale(data) # 2. standardize
# 3. Choose k: elbow and silhouette
set.seed(42)
ks <- 2:8
wss <- sapply(ks, function(k) kmeans(X, centers = k, nstart = 25)$tot.withinss)
sil <- sapply(ks, function(k) {
km <- kmeans(X, centers = k, nstart = 25)
mean(silhouette(km$cluster, dist(X))[, "sil_width"])
})
print(data.frame(k = ks, wss = round(wss, 1), silhouette = round(sil, 3)))
best_k <- ks[which.max(sil)]
# 4. Final model
set.seed(42)
km <- kmeans(X, centers = best_k, nstart = 50, iter.max = 100)
data$cluster <- factor(km$cluster)
# 5. Profile clusters on the original scale
data |>
group_by(cluster) |>
summarise(across(Murder:Rape, mean), states = n())
# 6. Visualize on the first two principal components
pca <- prcomp(X)
plot_df <- data.frame(pca$x[, 1:2], cluster = data$cluster, state = rownames(data))
ggplot(plot_df, aes(PC1, PC2, colour = cluster, label = state)) +
geom_text(size = 3) +
labs(title = paste("k-means with k =", best_k), colour = "Cluster") +
theme_minimal()
Script Checklist
Step
Why
Numeric features only
k-means uses Euclidean distance
scale()
Prevents large-range variables dominating
set.seed() + nstart
Reproducible and avoids poor local minima
Elbow / silhouette
Choose k with evidence
Profile on original scale
Clusters people can interpret
PCA plot
2-D view of the clusters
Common Pitfalls
Choosing k only by eye from the elbow; combine with silhouette and domain sense.
Categorical variables coded as numbers; use other methods (k-prototypes, PAM with Gower distance).
Interpreting cluster numbers as ordered; they are arbitrary labels.
Interview Tips
Walk through scale → choose k → fit → profile → visualize.
Mention nstart and the arbitrary cluster labels.
45
How do you build a linear regression model in R and plot its summary?
KnownRevise
Build the model with lm(), inspect it with summary() for coefficients, p-values and R², and plot it in three useful ways: the fitted relationship (ggplot2 with geom_smooth(method = "lm") or predicted lines), a coefficient plot with confidence intervals from broom::tidy(conf.int = TRUE), and the diagnostic plots from plot(model) to check residual assumptions. Together they show what the model says, how certain it is, and whether its assumptions hold.
Read full answerHide full answer
Applies toR 4.4+, ggplot2 3.5+, broom 1.0+
Code Example
library(ggplot2)
library(broom)
model <- lm(mpg ~ wt + hp + factor(am), data = mtcars)
summary(model) # coefficients, standard errors, p-values, R-squared
# 1. Coefficient plot with 95% confidence intervals
coefs <- tidy(model, conf.int = TRUE)
coefs <- coefs[coefs$term != "(Intercept)", ]
ggplot(coefs, aes(estimate, reorder(term, estimate))) +
geom_vline(xintercept = 0, linetype = "dashed", colour = "grey50") +
geom_errorbarh(aes(xmin = conf.low, xmax = conf.high), height = 0.2) +
geom_point(size = 3) +
labs(x = "Effect on mpg (95% CI)", y = NULL, title = "Linear model coefficients") +
theme_minimal()
# 2. Fitted vs actual
aug <- augment(model) # adds .fitted, .resid, .hat, .cooksd
ggplot(aug, aes(.fitted, mpg)) +
geom_point() +
geom_abline(slope = 1, intercept = 0, linetype = "dashed") +
labs(x = "Fitted mpg", y = "Actual mpg")
# 3. Residual diagnostics
par(mfrow = c(2, 2))
plot(model) # residuals vs fitted, Q-Q, scale-location, leverage
par(mfrow = c(1, 1))
# 4. Model-level statistics
glance(model)[, c("r.squared", "adj.r.squared", "sigma", "AIC")]
# 5. Relationship for one predictor with the others held at typical values
grid <- data.frame(wt = seq(1.5, 5.5, by = 0.1), hp = median(mtcars$hp), am = 0)
grid$pred <- predict(model, grid)
ggplot(mtcars, aes(wt, mpg)) + geom_point() + geom_line(data = grid, aes(wt, pred), colour = "blue")
What the Plots Tell You
Plot
Question
Coefficient plot
Which predictors matter and in which direction, with uncertainty
Fitted vs actual
How well the model predicts overall
Residuals vs fitted
Non-linearity, unequal variance
Normal Q-Q
Normality of residuals
Residuals vs leverage
Influential observations (Cook’s distance)
Common Pitfalls
Comparing coefficients measured in different units as “importance”; standardize predictors first if needed.
Ignoring patterns in residual plots.
Interpreting the effect of one predictor without holding others constant.
Interview Tips
Show both coefficient and diagnostic plots.
Mention broom’s tidy(), glance() and augment().
46
How do you visualize a decision tree model in R?
KnownRevise
To visualize a decision tree in R, fit it with rpart() and draw it with rpart.plot::rpart.plot(), which shows split rules, predicted classes or values, probabilities and the share of observations in each node; alternatives are partykit (plot(as.party(tree))), which adds distributions in the leaves, and base plot() with text(). Prune the tree first using the complexity parameter so the diagram stays readable, and print the rules with rpart.rules() for a text version.
Read full answerHide full answer
Applies toR 4.4+, rpart 4.1.23, rpart.plot 3.1, partykit 1.2
Code Example
library(rpart)
library(rpart.plot)
set.seed(42)
tree <- rpart(Species ~ ., data = iris, method = "class",
control = rpart.control(cp = 0.01, minsplit = 10))
# Prune to the cp with the lowest cross-validated error
best_cp <- tree$cptable[which.min(tree$cptable[, "xerror"]), "CP"]
pruned <- prune(tree, cp = best_cp)
# Main diagram: class, class probabilities and % of observations per node
rpart.plot(pruned, type = 4, extra = 104, box.palette = "auto",
main = "Decision tree for iris species")
rpart.rules(pruned, cover = TRUE) # the same tree as readable rules
plotcp(tree) # cross-validated error vs cp (pruning choice)
# partykit: leaves show class distributions as bar charts
library(partykit)
plot(as.party(pruned))
# Regression tree for mpg
reg_tree <- rpart(mpg ~ wt + hp + cyl, data = mtcars, control = rpart.control(minsplit = 6))
rpart.plot(reg_tree, type = 2, extra = 101, digits = 3) # mean mpg and n per node
# Save to a file
png("iris_tree.png", width = 1600, height = 1000, res = 200)
rpart.plot(pruned, type = 4, extra = 104)
dev.off()
Useful rpart.plot Options
Argument
Effect
type
Layout of labels (0-5); 4 labels both branches
extra
Node info: 104 = class probabilities + percentage (classification), 101 = n + percentage (regression)
box.palette
Node colours by class or value
fallen.leaves
Put leaves at the bottom (default TRUE)
tweak
Scale the text size
digits
Rounding of node values
Common Pitfalls
Plotting an unpruned tree that is too big to read.
extra values meant for classification used on regression trees.
Presenting one tree as definitive; trees change with small data changes.
Interview Tips
Mention rpart.plot and rpart.rules() for diagrams and text.
Explain pruning with plotcp() before plotting.
47
How do you write R code for grid search hyperparameter tuning?
KnownRevise
Grid search in R evaluates every combination of a set of hyperparameter values with cross-validation and keeps the best one: in tidymodels you mark parameters with tune(), build a grid with grid_regular() or expand_grid(), run tune_grid() over vfold_cv() folds, and finalize with select_best() and last_fit(); in caret you pass tuneGrid = expand.grid(...) to train(); and you can also write the loop yourself to see exactly what happens.
Read full answerHide full answer
Applies toR 4.4+, tidymodels 1.2 (tune, dials, rsample), ranger 0.16
Prefer random or Bayesian search for many parameters
Grids grow exponentially
Same folds for every candidate
Fair comparison
Consider the one-standard-error rule
select_by_one_std_err() picks a simpler model
Common Pitfalls
Tuning on the test set.
Preprocessing outside the resampling loop.
Grids with values outside sensible ranges (e.g. mtry above the number of predictors).
Interview Tips
Explain the full loop: grid × folds → mean score → best → final test.
Mention random and Bayesian alternatives when the grid is large.
48
How do you write an R function for accuracy, precision, recall and F1?
KnownRevise
A classification metrics function in R builds the confusion counts from the true and predicted labels for a chosen positive class, true positives, false positives, false negatives and true negatives, and computes accuracy = (TP + TN) / total, precision = TP / (TP + FP), recall = TP / (TP + FN), and F1 = 2 · precision · recall / (precision + recall), returning NA where a denominator is zero. For multi-class problems, compute the per-class values and average them (macro or weighted); yardstick and caret provide tested versions.
Choosing the wrong positive class; always pass it explicitly.
Division by zero when the model never predicts the positive class.
Averaging per-class F1 (macro) vs computing F1 from pooled counts (micro) without saying which.
Accuracy alone on imbalanced data.
Interview Tips
Derive each metric from the confusion counts.
Mention macro vs micro averaging for multi-class problems.
49
How do you add a regression line with confidence band in ggplot2?
KnownRevise
In ggplot2, a scatter plot with a regression line and confidence interval is ggplot(df, aes(x, y)) + geom_point() + geom_smooth(method = "lm", se = TRUE): geom_smooth() fits a linear model to the plotted data and draws the fitted line with a shaded 95% confidence band for the mean response, and level changes the confidence level. Map a grouping variable to colour to get one line per group, use formula = y ~ poly(x, 2) for curved fits, and add a prediction band yourself from predict(..., interval = "prediction") when you want the range for individual points.
Read full answerHide full answer
Applies toR 4.4+, ggplot2 3.5+
Code Example
library(ggplot2)
# Basic scatter + linear fit + 95% confidence band
ggplot(mtcars, aes(wt, mpg)) +
geom_point(size = 2.5, alpha = 0.8) +
geom_smooth(method = "lm", formula = y ~ x, se = TRUE, level = 0.95,
colour = "steelblue", fill = "steelblue", alpha = 0.2) +
labs(x = "Weight (1000 lbs)", y = "Miles per gallon",
title = "Fuel economy falls with weight",
subtitle = "Linear fit with 95% confidence interval") +
theme_minimal()
# One line per group
ggplot(mtcars, aes(hp, mpg, colour = factor(am, labels = c("automatic", "manual")))) +
geom_point() +
geom_smooth(method = "lm", formula = y ~ x) +
labs(colour = "Transmission")
# Quadratic fit
ggplot(mtcars, aes(hp, mpg)) +
geom_point() +
geom_smooth(method = "lm", formula = y ~ poly(x, 2))
# Add a 95% prediction band (where individual cars are expected to fall)
fit <- lm(mpg ~ wt, data = mtcars)
grid <- data.frame(wt = seq(min(mtcars$wt), max(mtcars$wt), length.out = 100))
band <- cbind(grid, predict(fit, grid, interval = "prediction"))
ggplot(mtcars, aes(wt, mpg)) +
geom_ribbon(data = band, aes(x = wt, ymin = lwr, ymax = upr), inherit.aes = FALSE,
fill = "grey85") +
geom_smooth(method = "lm", formula = y ~ x, colour = "black") +
geom_point()
Confidence vs Prediction Band
Band
Meaning
Width
Confidence (geom_smooth se = TRUE)
Where the mean mpg at each weight lies
Narrower
Prediction
Where an individual new car’s mpg lies
Wider
Common Pitfalls
Not setting method = "lm": the default is LOESS for small data (and a GAM for larger), which isn’t a straight line.
Expecting the plotted line to match a multi-variable model; geom_smooth() uses only x and y.
Extrapolating the band beyond the data range.
Interview Tips
Explain what the shaded band represents.
Mention per-group fits and the difference between confidence and prediction intervals.
50
How do you split data and train a random forest in R?
KnownRevise
A complete random forest script in R sets a seed, splits the data into training and test sets with stratification (rsample::initial_split() or caret::createDataPartition()), trains the forest on the training set only with ranger or randomForest (or tidymodels’ rand_forest()), checks the out-of-bag error, predicts on the test set, and reports metrics such as accuracy and a confusion matrix plus variable importance. The test set is used once, at the end, to estimate performance on new data.
Read full answerHide full answer
Applies toR 4.4+, rsample 1.2+, ranger 0.16, yardstick 1.3+
library(tidymodels)
set.seed(42)
split <- initial_split(iris, prop = 0.75, strata = Species)
wf <- workflow() |>
add_formula(Species ~ .) |>
add_model(rand_forest(trees = 500) |> set_engine("ranger") |> set_mode("classification"))
final <- last_fit(wf, split) # fits on training, evaluates on test
collect_metrics(final)
Key Settings
ranger
randomForest
Meaning
num.trees
ntree
Number of trees
mtry
mtry
Predictors per split (default floor(sqrt(p)) for classification)
min.node.size
nodesize
Minimum leaf size
importance
importance = TRUE
Variable importance
Common Pitfalls
Evaluating on the training data.
Forgetting set.seed() (or ranger’s seed), so results change each run.
Character outcomes: convert to factors for classification.
Interview Tips
Show split → fit → OOB → test metrics → importance.
Mention ranger as the fast modern implementation.
51
How would you forecast next quarter's sales in R?
KnownRevise
To forecast next quarter’s sales in R, aggregate the sales history to a regular frequency (for example monthly), plot it to understand trend and seasonality, hold out the most recent months for testing, fit several candidate models such as seasonal naive, ETS (exponential smoothing) and ARIMA with the fable or forecast package, compare their accuracy on the hold-out period, then refit the best model on all data and forecast the next three months with prediction intervals, summing them for the quarter. Add known drivers such as promotions, prices or holidays as regressors when they help, and present the forecast as a range rather than a single number.
Prepare: aggregate transactions to monthly totals; fill gaps; handle outliers and calendar effects.
Explore: plot, STL decomposition, seasonal plots.
Split: last 6-12 months as the test period (or time-series cross-validation).
Model: benchmarks (naive, seasonal naive) plus ETS, ARIMA, and regression with drivers.
Evaluate: RMSE, MAE, MAPE or MASE on the test period.
Forecast: refit on all data, forecast 3 months, aggregate to the quarter with intervals.
Communicate: point forecast, 80%/95% intervals, assumptions.
Code Example
library(dplyr)
library(tsibble)
library(fable)
library(feasts)
# Example monthly sales with trend and seasonality (replace with your data)
set.seed(42)
months <- yearmonth("2021 Jan") + 0:59
sales <- tsibble(
month = months,
sales = round(1000 + 10 * (0:59) + 150 * sin(2 * pi * (0:59) / 12) + rnorm(60, 0, 40)),
index = month
)
sales |> model(STL(sales ~ season(window = "periodic"))) |> components() |> autoplot()
train <- sales |> filter(month < yearmonth("2025 Jan"))
fits <- train |>
model(snaive = SNAIVE(sales),
ets = ETS(sales),
arima = ARIMA(sales))
fits |> forecast(h = 12) |> accuracy(sales) |> arrange(RMSE) # compare on 2025
best <- sales |> model(ets = ETS(sales)) # refit the chosen model on all data
fc <- best |> forecast(h = 3) # next quarter (3 months)
fc |> hilo(level = 95)
# Quarter total with an interval from simulated future paths
sims <- best |> generate(h = 3, times = 1000, bootstrap = TRUE)
totals <- sims |> as_tibble() |> group_by(.rep) |> summarise(total = sum(.sim))
quantile(totals$total, c(0.025, 0.5, 0.975))
Model Options
Model
Good for
Seasonal naive
Benchmark: same month last year
ETS
Trend and seasonality, smooth changes
ARIMA
Autocorrelation patterns, with seasonal terms
Regression with ARIMA errors
Drivers like price, promotions, holidays
Prophet (fable.prophet)
Multiple seasonalities, holiday effects
Common Pitfalls
Summing the upper bounds of three monthly intervals to get a quarterly interval; simulate instead.
Random train/test splits.
Forgetting structural changes (new stores, pricing changes).
Interview Tips
Always compare against a naive benchmark.
Explain how you would report uncertainty for the quarter.
52
How would you identify customer segments for market analysis in R?
KnownRevise
Customer segmentation in R starts by building one row per customer with meaningful features, often RFM (recency, frequency, monetary value) plus product mix, channel and demographics, then transforming and scaling them, choosing a clustering method (k-means for numeric features, PAM with Gower distance for mixed types, hierarchical clustering or Gaussian mixtures), selecting the number of segments with silhouette or BIC, profiling and naming the segments, and validating that they are stable and actionable. The output should be a small set of segments that marketing can recognise and target differently.
Read full answerHide full answer
Applies toR 4.4+, dplyr 1.1+, cluster 2.1+, mclust 6.1
Steps
Customer table: aggregate transactions to recency, frequency, monetary value, average basket, category shares, tenure.
Transform: log skewed money and counts; scale numeric features.
Cluster: k-means, PAM (Gower distance for categorical + numeric), hierarchical, or mclust.
Choose k: silhouette, elbow, BIC (mclust), and business usability (4-7 segments is common).
Profile: segment sizes and averages; give names like “loyal high spenders” or “lapsed”.
Validate: stability across samples and seeds; differences in outcomes such as churn or response.
Clustering raw skewed spend, so a few big customers dominate.
Too many segments to act on.
Segments that don’t differ in outcomes the business cares about.
Interview Tips
Mention RFM, scaling, choosing k and naming segments.
Tie segments to actions and measurement.
53
How would you use R to detect potentially fraudulent transactions?
KnownRevise
In R, fraud detection combines supervised models trained on labelled past fraud, such as logistic regression, random forests (ranger) or gradient boosting (xgboost) with class weights or resampling for the rare fraud class, and unsupervised anomaly detection for new patterns, such as isolation forests (isotree) or distance-based outlier scores, on engineered behavioural features like amount relative to the customer’s usual spend, transaction velocity, new merchants or devices, and unusual times or locations. Models are validated on later time periods with precision-recall metrics, and alert thresholds are set from investigation capacity and the cost of missed fraud.
themis::step_downsample(), step_smote(), class weights
Supervised models
logistic_reg(), rand_forest(), boost_tree()
Anomaly detection
isotree::isolation.forest(), Mahalanobis distance
Time-based validation
rsample::initial_time_split(), sliding_period()
Metrics
yardstick::pr_auc(), precision and recall at a threshold
Code Example
library(dplyr)
library(tidymodels)
library(themis)
set.seed(42)
n <- 20000
tx <- tibble(
customer = sample(1:2000, n, replace = TRUE),
time = sort(as.POSIXct("2026-01-01", tz = "UTC") + runif(n, 0, 300 * 86400)),
amount = rlnorm(n, 3.5, 0.8),
foreign = rbinom(n, 1, 0.03)
) |>
arrange(customer, time) |>
group_by(customer) |>
mutate(avg_amount = lag(cummean(amount)), # customer's past average
amount_ratio = amount / coalesce(avg_amount, amount),
hours_since_last = as.numeric(difftime(time, lag(time), units = "hours"))) |>
ungroup() |>
mutate(hours_since_last = coalesce(hours_since_last, 1000),
risk = -6 + 1.2 * log(amount_ratio + 0.1) + 2 * foreign - 0.002 * hours_since_last,
fraud = factor(rbinom(n, 1, plogis(risk)), levels = c(1, 0))) |>
arrange(time)
split <- initial_time_split(tx, prop = 0.8) # train on earlier transactions
rec <- recipe(fraud ~ amount_ratio + hours_since_last + foreign + amount, data = training(split)) |>
step_log(amount) |>
step_downsample(fraud, under_ratio = 5) # applied to training data only
model <- boost_tree(trees = 300, learn_rate = 0.05) |> set_engine("xgboost") |> set_mode("classification")
fit <- workflow() |> add_recipe(rec) |> add_model(model) |> fit(training(split))
preds <- augment(fit, testing(split))
pr_auc(preds, truth = fraud, .pred_1) # fraud is the first level
roc_auc(preds, truth = fraud, .pred_1)
# Unsupervised score for unusual transactions
library(isotree)
iso <- isolation.forest(training(split)[, c("amount_ratio", "hours_since_last")], ntrees = 200)
testing(split) |> mutate(anomaly = predict(iso, testing(split)[, c("amount_ratio", "hours_since_last")])) |>
slice_max(anomaly, n = 5)
Common Pitfalls
Random splits that let the model see the future.
Features using information only available after the transaction (e.g. chargeback status).
Reporting accuracy instead of PR AUC and alert precision.
Resampling before splitting.
Interview Tips
Combine supervised models with anomaly detection.
Explain time-based validation and threshold choice from business costs.
54
How would you build a recommendation system in R?
KnownRevise
A recommendation system in R can be built with the recommenderlab package, which takes a user-item rating matrix (realRatingMatrix or binaryRatingMatrix) and provides popularity, user-based and item-based collaborative filtering, SVD-style matrix factorization and association-rule recommenders with a shared evaluation framework; for larger or implicit data, matrix factorization with rsparse (weighted ALS) or recosystem, or Spark ALS through sparklyr, is used. Evaluate with held-out ratings using top-N precision and recall or RMSE, compare against a popularity baseline, and add content features for new items.
Read full answerHide full answer
Applies toR 4.4+, recommenderlab 1.0.6, recosystem 0.5
Explicit vs implicit: ratings use realRatingMatrix; clicks use binaryRatingMatrix or weighted ALS.
Cold start: popularity and content-based features for new users and items.
Scale: recommenderlab is for small to medium data; use recosystem, rsparse or Spark for millions of ratings.
Serving: precompute top-N lists and expose them with plumber or a database.
Common Pitfalls
Evaluating only with RMSE when the product shows top-N lists.
No popularity baseline.
Leaking future ratings into training; split by time where possible.
Interview Tips
Name recommenderlab’s methods and its evaluation scheme.
Mention scaling options and cold start handling.
55
How do you use R for sentiment analysis of customer feedback?
KnownRevise
R analyses customer feedback text with the tidytext approach: tokenize the comments into words with unnest_tokens(), remove stop words, and either score sentiment with lexicons such as Bing, AFINN or NRC (joined with get_sentiments()), or train a supervised classifier on labelled feedback with tidymodels and the textrecipes package (tokenize, TF-IDF, then a lasso or random forest model). Lexicon methods are quick and need no labels but miss context and negation; supervised models or transformer models via text or reticulate are more accurate when labelled data exists.
library(dplyr)
library(tidytext)
feedback <- tibble(
id = 1:5,
text = c("The delivery was fast and the product is excellent",
"Terrible support, I waited two weeks for a refund",
"Good quality but the price is too high",
"Not happy with the broken packaging",
"Love it, great value and friendly staff")
)
words <- feedback |>
unnest_tokens(word, text) |> # lowercase, one word per row
anti_join(stop_words, by = "word")
bing <- get_sentiments("bing") # positive / negative word list
scores <- words |>
inner_join(bing, by = "word") |>
count(id, sentiment) |>
tidyr::pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) |>
mutate(score = positive - negative)
feedback |> left_join(scores, by = "id") |> select(id, score, text)
# Note: "Not happy" in comment 4 may be scored positive by a word-level lexicon
# Most frequent negative words
words |> inner_join(bing, by = "word") |> filter(sentiment == "negative") |> count(word, sort = TRUE)
# Supervised model when labels exist
library(tidymodels)
library(textrecipes)
labelled <- feedback |> mutate(label = factor(c("pos", "neg", "pos", "neg", "pos")))
rec <- recipe(label ~ text, data = labelled) |>
step_tokenize(text) |>
step_stopwords(text) |>
step_tokenfilter(text, max_tokens = 500) |>
step_tfidf(text)
spec <- logistic_reg(penalty = 0.01, mixture = 1) |> set_engine("glmnet")
wf <- workflow() |> add_recipe(rec) |> add_model(spec)
# fit(wf, data = real_labelled_feedback) # needs hundreds of labelled comments
Common Pitfalls
Negation and sarcasm with word-level lexicons; use sentimentr or supervised models.
Removing stop words like “not” that change meaning.
Lexicons built for other domains (e.g. “cheap” can be positive or negative).
Too few labelled examples for supervised models.
Interview Tips
Start with the tidytext lexicon approach and explain its limits.
Mention textrecipes for supervised models and topic models for themes.
56
What are the trending R packages for advanced machine learning?
KnownRevise
The trending R packages for advanced machine learning centre on the tidymodels ecosystem (parsnip, recipes, tune, workflows, stacks, finetune) and mlr3 for unified, modular modelling; fast tree ensembles through ranger, xgboost and lightgbm (with bonsai); deep learning with torch for R and keras3; model explanation with DALEX, iml, vip and shapviz; probabilistic and Bayesian modelling with brms and Stan; and deployment and MLOps with vetiver, plumber and pins. Interfaces such as h2o and sparklyr add scalable AutoML and distributed training.
Read full answerHide full answer
Applies toR 4.4+ (package versions current in 2026)
library(tidymodels)
library(bonsai) # lightgbm engine for boost_tree()
set.seed(42)
split <- initial_split(mtcars, prop = 0.8)
spec <- boost_tree(trees = 300, learn_rate = 0.05, tree_depth = 3) |>
set_engine("lightgbm") |>
set_mode("regression")
fit <- workflow() |> add_formula(mpg ~ .) |> add_model(spec) |> fit(training(split))
# Explanation with DALEX
library(DALEX)
explainer <- explain_tidymodels(fit, data = training(split)[, -1], y = training(split)$mpg,
label = "lightgbm", verbose = FALSE)
model_parts(explainer) # permutation importance
predict_parts(explainer, new_observation = testing(split)[1, -1]) # break-down for one car
# Versioned deployment with vetiver
library(vetiver)
v <- vetiver_model(fit, "mpg_lightgbm")
# vetiver_pin_write(pins::board_folder("models"), v); vetiver_api(plumber::pr(), v)
How to Choose
Standard tabular ML: tidymodels or mlr3 with xgboost/lightgbm/ranger.
Explainability requirements: DALEX, shapviz.
Deep learning in R: torch (no Python needed).
Production: vetiver + plumber, pipelines with targets.
Common Pitfalls
Chasing new packages without checking maintenance and CRAN status.
Mixing frameworks (caret, tidymodels, mlr3) in one pipeline.
Interview Tips
Name a package per area and say why you’d use it.
Mention vetiver for MLOps, which many candidates don’t know.
57
How does R handle big data and what are the best practices?
KnownRevise
By default R holds data in memory, so big data is handled by choosing tools that avoid loading everything at once or that push computation elsewhere: data.table for fast in-memory work, arrow and DuckDB for querying files larger than memory (Parquet datasets with dplyr syntax), dbplyr to translate dplyr code into SQL that runs in the database, sparklyr for Spark clusters, and parallel processing with future, furrr or parallel. Best practices are to filter and aggregate before collecting into R, store data in columnar formats such as Parquet, use efficient types, and profile memory and time.
library(dplyr)
library(arrow)
# Write a partitioned Parquet dataset (example data)
dir <- file.path(tempdir(), "flights_ds")
big <- tibble(year = rep(2024:2026, each = 100000),
carrier = sample(c("AA", "BA", "LH"), 300000, replace = TRUE),
delay = rnorm(300000, 10, 20))
write_dataset(big, dir, partitioning = "year", format = "parquet")
# Query lazily: only the needed columns and partitions are read
ds <- open_dataset(dir)
ds |>
filter(year == 2026, delay > 0) |>
group_by(carrier) |>
summarise(mean_delay = mean(delay), n = n()) |>
collect() # computation happens before collect()
# The same with DuckDB on the Parquet files
library(duckdb)
con <- dbConnect(duckdb())
tbl(con, sql(paste0("SELECT * FROM read_parquet('", dir, "/*/*.parquet', hive_partitioning = true)"))) |>
group_by(carrier) |>
summarise(mean_delay = mean(delay, na.rm = TRUE)) |>
collect()
dbDisconnect(con, shutdown = TRUE)
# Fast in-memory aggregation with data.table
library(data.table)
dt <- as.data.table(big)
dt[delay > 0, .(mean_delay = mean(delay), .N), by = .(year, carrier)]
Best Practices
Filter, select and aggregate in the database or Arrow before collect().
Store data as Parquet, partitioned by commonly filtered columns.
Use appropriate types (integers, factors, dates) and drop unused columns.
Avoid growing objects in loops; preallocate or vectorize.
Parallelize independent tasks (model fits, simulations) with future.
Profile with profvis and bench; check memory with lobstr::obj_size().
Common Pitfalls
read.csv() on multi-gigabyte files.
Calling collect() too early, pulling everything into memory.
Copies created by modifying large data frames repeatedly (data.table updates by reference with :=).
Interview Tips
Mention arrow/DuckDB for larger-than-memory data and dbplyr for databases.
Explain lazy evaluation and pushing computation to where the data lives.
58
What role does R play in reproducible research?
KnownRevise
R supports reproducible research by letting analysis code, results and narrative live together in literate documents (R Markdown and Quarto), so every number, table and figure is regenerated from the data when the document is rendered, and by providing tools to pin the computing environment and pipeline: renv records exact package versions, targets builds analysis pipelines that rerun only what changed, Git tracks code history, and Docker images (for example from the Rocker project) capture the whole system. Combined with scripted data cleaning, fixed random seeds and shared data, this lets others rerun the analysis and get the same results.
Read full answerHide full answer
Applies toR 4.4+, Quarto 1.5+, rmarkdown 2.28, renv 1.0, targets 1.7
Building Blocks
Need
Tool
Code + results + text in one document
Quarto (.qmd), R Markdown (.Rmd)
Exact package versions
renv (renv.lock)
Pipelines and caching
targets
Version control
Git, GitHub/GitLab
Whole environment
Docker (Rocker images)
Random results
set.seed()
Project structure
RStudio projects, here::here() for paths
Packaging analysis code
R package structure, testthat tests
Sharing data
pins, data repositories with DOIs
Code Example
A Quarto document (report.qmd) starts with a YAML header:
title: "Fuel economy analysis"
format: html
execute:
echo: true
Below the header, prose is mixed with R code chunks (opened with three backticks followed by {r}), and rendering reruns every chunk:
set.seed(42)
model <- lm(mpg ~ wt + hp, data = mtcars)
Inline code keeps numbers in the text in sync: a backtick, the letter r and an expression such as round(coef(model)["wt"], 2), closed with another backtick, prints the current weight coefficient inside a sentence when the document is rendered.
A targets pipeline (_targets.R):
library(targets)
tar_option_set(packages = c("dplyr", "ggplot2"))
list(
tar_target(raw, read.csv("data/cars.csv")),
tar_target(clean, dplyr::filter(raw, !is.na(mpg))),
tar_target(model, lm(mpg ~ wt + hp, data = clean)),
tar_target(report, quarto::quarto_render("report.qmd"), format = "file")
)
# tar_make() runs only the steps whose inputs changed; renv::snapshot() records package versions
Practices
Never edit data by hand; script every transformation from raw data.
Use relative paths (here::here()), not setwd() with absolute paths.
Record sessionInfo() in reports.
Keep raw data read-only and separate from derived data.
Common Pitfalls
Results copied manually into Word or slides, which go stale.
Mention Quarto/R Markdown, renv and targets as the core trio.
Explain inline code that keeps numbers in the text in sync with the analysis.
59
How can R be used together with other programming languages?
KnownRevise
R works with other languages through dedicated bridges: reticulate runs Python code and libraries inside R and converts objects between them, Rcpp lets you write fast C++ functions callable from R, .Call()/.C() interface with C and Fortran (which much of base R uses), DBI sends SQL to databases, JuliaCall and rJava reach Julia and Java, and Quarto documents can mix R, Python, SQL and other chunks. Across processes, languages exchange data through Arrow and Parquet, and services through REST APIs built with plumber or called with httr2.
# Python from R
library(reticulate)
np <- import("numpy")
np$mean(c(1, 2, 3, 4)) # 2.5 computed by NumPy
py_run_string("squares = [i ** 2 for i in range(5)]")
py$squares # list(0, 1, 4, 9, 16) back in R
# C++ from R
library(Rcpp)
cppFunction("
double sum_squares(NumericVector x) {
double total = 0;
for (int i = 0; i < x.size(); ++i) total += x[i] * x[i];
return total;
}")
sum_squares(c(1, 2, 3)) # 14
# SQL from R
library(DBI)
con <- dbConnect(RSQLite::SQLite(), ":memory:")
dbWriteTable(con, "cars", mtcars)
dbGetQuery(con, "SELECT cyl, COUNT(*) AS n FROM cars GROUP BY cyl") # 4: 11, 6: 7, 8: 14
dbDisconnect(con)
# Share data with Python or Spark through Parquet
# arrow::write_parquet(mtcars, "mtcars.parquet") # pandas.read_parquet("mtcars.parquet")
# Expose an R model as an API (plumber.R)
# #* @post /predict
# function(wt, hp) predict(model, data.frame(wt = as.numeric(wt), hp = as.numeric(hp)))
Common Pitfalls
reticulate using the wrong Python environment; set it with use_virtualenv() or RETICULATE_PYTHON.
Copying large objects between R and Python repeatedly; use Arrow.
Compilers missing for Rcpp on Windows (install Rtools).
Interview Tips
Mention reticulate and Rcpp first; they are the most used.
Explain data exchange with Arrow/Parquet and APIs with plumber.
60
What are R's limitations for machine learning and how can they be overcome?
KnownRevise
R’s main limitations for machine learning are that it keeps data in memory by default and is single-threaded for most base code, its deep learning and large-scale ML ecosystems are smaller than Python’s, production deployment and MLOps tooling are less widespread, and the many overlapping packages (caret, tidymodels, mlr3) and inconsistent interfaces can confuse newcomers. They can be overcome with larger-than-memory and distributed tools (arrow, DuckDB, data.table, sparklyr), parallel processing (future), compiled code (Rcpp), unified frameworks (tidymodels or mlr3), deep learning through torch for R or reticulate, and deployment with vetiver, plumber, Docker and Posit Connect.
Read full answerHide full answer
Applies toR 4.4+ (tidymodels 1.2, torch 0.13, vetiver 0.2)