Skip to content
Machine Learning & Data Science

R interview questions and answers

60 questions, 60 with code. About 2 h 50 min to read them all.

By TestLab · Updated

Plan for an interview
60 questions

Track what you know and what to revise.

Sign in to track progress

You can open a few full answers without signing in. Sign in to read all

1

Why is R important for data analysis and machine learning?

R is a language and environment built for statistics and data analysis, and it matters in data science because it offers an unmatched range of statistical methods (from linear and mixed models to survival analysis and Bayesian modelling), first-class data visualization with ggplot2, a fluent data-manipulation toolkit in the tidyverse, a unified machine learning interface in tidymodels and caret, and reproducible reporting with R Markdown and Quarto. Over 20,000 packages on CRAN, plus Bioconductor for genomics, mean that new statistical methods often appear in R first, which is why it is widely used in academia, biostatistics, pharma, finance and analytics teams.

Read full answer

Applies toR 4.4+, tidyverse 2.0, tidymodels 1.2

Strengths

Area What R offers
Statistics lm, glm, t.test, aov, mixed models (lme4), survival (survival), Bayesian (brms)
Visualization ggplot2’s grammar of graphics, plus plotly for interactivity
Data wrangling dplyr, tidyr, readr, data.table for speed
Machine learning tidymodels (recipes, parsnip, tune, yardstick), caret, ranger, xgboost
Reporting R Markdown and Quarto for documents, slides and dashboards
Apps Shiny for interactive web apps
Domain packages Bioconductor (genomics), forecasting (fable), econometrics

Typical Workflow

  1. Import data with readr::read_csv().

  2. Clean and transform with dplyr and tidyr.

  3. Explore with ggplot2 and summary statistics.

  4. Model with lm()/glm() or tidymodels.

  5. Report with Quarto or share a Shiny app.

Code Example

library(dplyr)
library(ggplot2)

# Summarise and model a built-in data set
mtcars |>
  group_by(cyl) |>
  summarise(mean_mpg = mean(mpg), n = n())
# cyl 4: mean_mpg 26.7 (n 11); cyl 6: 19.7 (n 7); cyl 8: 15.1 (n 14)

model <- lm(mpg ~ wt + hp, data = mtcars)
summary(model)   # coefficients, standard errors, p-values, R-squared

ggplot(mtcars, aes(wt, mpg, colour = factor(cyl))) +
  geom_point() +
  geom_smooth(method = "lm", se = TRUE) +
  labs(x = "Weight (1000 lbs)", y = "Miles per gallon", colour = "Cylinders")

R vs Python, Honestly

Aspect R Python
Classical statistics Excellent, very broad Good (statsmodels, SciPy)
Visualization ggplot2 is a standard Matplotlib, seaborn, plotly
Deep learning Possible (torch for R, keras3) but smaller Dominant (PyTorch, TensorFlow, JAX)
Production / engineering Plumber APIs, Shiny; smaller ecosystem Larger web and MLOps ecosystem
Community Statisticians, researchers Broad software and ML community

Common Pitfalls

  • Growing vectors in loops instead of vectorized operations.

  • Mixing up the native pipe |> (R 4.1+) and magrittr’s %>% in older code.

  • Ignoring factor levels when modelling categorical data.

Interview Tips

  • Lead with statistics, visualization and reporting as R’s strengths.

  • Acknowledge Python’s lead in deep learning and production, and mention that teams often use both (reticulate).

2

How do you install and load packages in R?

In R you install a package once from CRAN with install.packages("name"), which downloads it into your library, and load it in every session where you use it with library(name), which attaches its functions to the search path. Packages from other sources use helpers such as BiocManager::install() for Bioconductor and remotes::install_github() or pak::pak() for GitHub, and project-specific package versions are best managed with renv for reproducibility.

Read full answer

Applies toR 4.4+

Key Functions

Task Code
Install from CRAN install.packages("dplyr")
Install several install.packages(c("dplyr", "ggplot2"))
Load (attach) library(dplyr)
Check if installed and load requireNamespace("dplyr", quietly = TRUE)
Use one function without attaching dplyr::filter(df, x > 1)
Update packages update.packages(ask = FALSE)
Remove remove.packages("dplyr")
List installed installed.packages()[, "Package"]
Package version packageVersion("dplyr")
Bioconductor BiocManager::install("DESeq2")
GitHub remotes::install_github("user/repo") or pak::pak("user/repo")

Code Example

# Install only the packages that are missing
needed <- c("dplyr", "ggplot2", "readr")
missing <- needed[!vapply(needed, requireNamespace, logical(1), quietly = TRUE)]
if (length(missing) > 0) install.packages(missing)

library(dplyr)          # attach for this session
library(ggplot2)

packageVersion("dplyr") # e.g. '1.1.4'

# Call a function without attaching the package (avoids name clashes)
readr::read_csv(readr::readr_example("mtcars.csv"), show_col_types = FALSE)

# Reproducible project library
# renv::init()       # create a project-specific library and lockfile
# renv::snapshot()   # record exact package versions in renv.lock
# renv::restore()    # reinstall those versions on another machine

library() vs require()

  • library() stops with an error if the package is missing, which is what scripts should do.

  • require() returns FALSE with a warning; use requireNamespace() to test availability without attaching.

Name Conflicts

  • Loading dplyr masks stats::filter() and stats::lag(); use pkg::fun() to be explicit, or the conflicted package to force a choice.

Common Pitfalls

  • Putting install.packages() at the top of a shared script, which reinstalls every run.

  • Forgetting to load the package in a new session (“could not find function”).

  • Mixing package versions across team members; use renv.

Interview Tips

  • Explain install once vs load each session.

  • Mention renv for reproducibility and :: for namespacing.

3

What are the different data types in R?

R has six basic (atomic) data types: logical (TRUE/FALSE), integer (5L), double (numeric, 3.14), character ("text"), complex (2+3i) and raw (bytes), and it organises values into data structures: atomic vectors (one type), lists (any types), matrices and arrays (vectors with dimensions), factors (categorical data with levels), and data frames or tibbles (lists of equal-length columns). There are no scalars in R: a single value is a vector of length 1, and missing values are represented by NA, with typed versions for each type.

Read full answer

Applies toR 4.4+

Atomic Types

Type Example typeof() Note
Logical TRUE, FALSE, NA "logical" Use TRUE, not T
Integer 5L, 1:3 "integer" The L suffix makes an integer
Double 3.14, 5 "double" Default for numbers; class() is "numeric"
Character "hello" "character" Strings
Complex 2+3i "complex" Rarely used
Raw as.raw(255) "raw" Bytes

Data Structures

Structure Dimensions Types Example
Vector 1 One c(1, 2, 3)
List 1 Any list(a = 1, b = "x")
Matrix 2 One matrix(1:6, nrow = 2)
Array n One array(1:24, dim = c(2, 3, 4))
Factor 1 Integer codes + levels factor(c("low", "high"))
Data frame / tibble 2 One per column data.frame(x = 1:2, y = c("a", "b"))

Code Example

x <- c(1.5, 2, 3)
typeof(x)            # "double"
class(x)             # "numeric"
typeof(1:3)          # "integer"
typeof("a")          # "character"
typeof(TRUE)         # "logical"

# Vectors hold one type; mixing coerces (logical < integer < double < character)
c(1, "a", TRUE)      # "1" "a" "TRUE"

size <- factor(c("small", "large", "small"), levels = c("small", "large"))
levels(size)         # "small" "large"
as.integer(size)     # 1 2 1

df <- data.frame(id = 1:3, name = c("A", "B", "C"))
str(df)              # 'data.frame': 3 obs. of 2 variables: id int, name chr

l <- list(n = 1, s = "text", v = 1:3)
str(l)

NA_integer_; NA_real_; NA_character_   # typed missing values
is.na(c(1, NA, 3))   # FALSE TRUE FALSE

Special Values

  • NA: missing; NULL: absence of an object; NaN: not a number (0/0); Inf/-Inf: infinity (1/0).

Common Pitfalls

  • Expecting 5 to be an integer; it is a double unless written 5L.

  • Factors converted with as.numeric() give level codes, not the original numbers; use as.numeric(as.character(f)).

  • Using T and F, which can be reassigned.

Interview Tips

  • Distinguish types (typeof) from structures and classes (class).

  • Mention coercion order and the factor-to-number pitfall.

4

How do you convert data types in R?

R converts data types with the as.*() family, such as as.numeric(), as.integer(), as.character(), as.logical(), as.factor() and as.Date(), and checks types with the matching is.*() functions; conversions that fail produce NA with a warning instead of an error. R also coerces implicitly when you combine types in a vector, following the order logical → integer → double → character, and data frame columns can be converted together with dplyr’s mutate(across(...)).

Read full answer

Applies toR 4.4+, dplyr 1.1+

Common Conversions

From → To Function Example
Character → number as.numeric() as.numeric("3.5") is 3.5
Number → integer as.integer() as.integer(3.9) is 3 (truncates)
Anything → text as.character() as.character(42) is “42”
Text/number → logical as.logical() as.logical("TRUE"), as.logical(0) is FALSE
Vector → factor factor() / as.factor() factor(x, levels = c("lo", "hi"))
Factor → number as.numeric(as.character(f)) Avoids getting level codes
Text → date as.Date() as.Date("2026-11-11")
Text → date-time as.POSIXct() as.POSIXct("2026-11-11 10:30", tz = "UTC")
Matrix ↔ data frame as.data.frame(), as.matrix()
List → vector unlist()

Code Example

library(dplyr)

as.numeric(c("1", "2.5", "abc"))   # 1.0 2.5 NA, with a warning "NAs introduced by coercion"
as.integer(c(3.9, -3.9))           # 3 -3 (truncates toward zero)
as.logical(c("yes", "TRUE", "F"))  # NA TRUE FALSE

f <- factor(c("10", "20", "10"))
as.numeric(f)                      # 1 2 1   (level codes, usually not wanted)
as.numeric(as.character(f))        # 10 20 10

as.Date("11/11/2026", format = "%m/%d/%Y")   # "2026-11-11"

df <- data.frame(id = c("1", "2", "3"), score = c("90", "85", "x"), grade = c("A", "B", "A"))
df <- df |>
  mutate(across(c(id, score), as.numeric),   # several columns at once ("x" becomes NA)
         grade = factor(grade))
str(df)

# Checking types
is.numeric(df$score)   # TRUE
is.factor(df$grade)    # TRUE

Reading Data with the Right Types

  • readr::read_csv(file, col_types = cols(id = col_integer(), date = col_date())) sets types on import.

  • readr::parse_number("$1,234") extracts numbers from formatted text (1234).

Common Pitfalls

  • Converting factors directly to numbers.

  • Silent NAs from failed conversions; check sum(is.na(x)) after converting.

  • Dates in non-ISO formats without a format argument.

Interview Tips

  • Mention the coercion hierarchy and the factor pitfall.

  • Show mutate(across()) for converting many columns.

5

What is the difference between a list and a data frame in R?

A list is R’s most flexible container: an ordered collection whose elements can be of any type and any length, including other lists, while a data frame is a special list in which every element is a column vector of the same length, giving a rectangular table where each column has one type and each row is an observation. Lists suit nested or irregular data (model outputs, JSON, configuration); data frames and tibbles suit tabular data for analysis, modelling and plotting.

Read full answer

Applies toR 4.4+, tibble 3.2+

Comparison

Aspect List Data frame
Structure Any elements, any lengths Columns of equal length
Element types Mixed, nested One type per column (list-columns possible)
Dimensions 1 (length) 2 (rows × columns)
Indexing l[["a"]], l$a, l[1] df[rows, cols], df$col, df[["col"]]
Typical use Model results, JSON, mixed objects Tabular data, modelling, ggplot2
Created with list() data.frame(), tibble()
class() "list" "data.frame" (tibbles add "tbl_df", "tbl")

Code Example

# A list: mixed types and lengths
person <- list(name = "Ana", age = 31, skills = c("R", "SQL"), address = list(city = "Lisbon"))
person$skills            # "R" "SQL"
person[["address"]]$city # "Lisbon"
person["age"]            # a list of length 1 containing age
length(person)           # 4

# A data frame: equal-length columns
df <- data.frame(name = c("Ana", "Ben"), age = c(31, 28), stringsAsFactors = FALSE)
dim(df)                  # 2 2
df[df$age > 30, ]        # rows where age > 30
is.list(df)              # TRUE: a data frame is a list of columns

# This fails: columns must have the same length
# data.frame(a = 1:3, b = 1:2)   # error: arguments imply differing number of rows

# Tibble with a list-column (nested data in a table)
library(tibble)
tb <- tibble(id = 1:2, scores = list(c(90, 85), c(70)))
tb$scores[[1]]           # 90 85

Indexing Difference

  • l[1] returns a list containing the first element; l[[1]] returns the element itself.

  • df["age"] returns a one-column data frame; df[["age"]] and df$age return the vector.

When to Use Which

  • Data frame / tibble: observations × variables, analysis with dplyr and ggplot2.

  • List: results of functions (lm() returns a list-based object), heterogeneous collections, iterating with lapply/purrr::map.

Common Pitfalls

  • Confusing [ and [[ on lists.

  • Expecting a data frame to hold columns of different lengths.

  • df[, 1] drops to a vector for data frames but not for tibbles.

Interview Tips

  • State that a data frame is a list of equal-length vectors.

  • Explain [ vs [[ clearly.

6

How do you handle missing values in R?

In R, missing values are NA (with typed variants), detected with is.na(), counted with sum(is.na(x)) or colSums(is.na(df)), skipped in calculations with na.rm = TRUE, removed with na.omit(), complete.cases() or tidyr::drop_na(), and filled with tidyr::replace_na(), dplyr::coalesce(), grouped means or medians, or model-based imputation packages such as mice. The right method depends on why the data is missing (completely at random, at random, or not at random), and imputation for models should be learned on training data only, for example inside a tidymodels recipe.

Read full answer

Applies toR 4.4+, dplyr 1.1+, tidyr 1.3+, recipes 1.1+

Detecting

Task Code
Missing flags is.na(x)
Count per column colSums(is.na(df))
Rows with no missing values complete.cases(df)
Any missing anyNA(df)

Handling

Approach Code
Ignore in calculations mean(x, na.rm = TRUE)
Drop rows na.omit(df), tidyr::drop_na(df, col)
Constant fill tidyr::replace_na(df, list(col = 0))
First non-missing dplyr::coalesce(x, y, 0)
Mean/median (grouped) mutate(col = if_else(is.na(col), median(col, na.rm = TRUE), col))
Carry forward (time series) tidyr::fill(df, col, .direction = "down")
Model-based mice::mice(), missForest, recipes::step_impute_knn()

Code Example

library(dplyr)
library(tidyr)

colSums(is.na(airquality))           # Ozone 37, Solar.R 7 missing; other columns 0
mean(airquality$Ozone)               # NA
mean(airquality$Ozone, na.rm = TRUE) # about 42.1

nrow(drop_na(airquality))            # 111 complete rows out of 153

imputed <- airquality |>
  group_by(Month) |>
  mutate(Ozone = if_else(is.na(Ozone), median(Ozone, na.rm = TRUE), as.numeric(Ozone)),
         Solar.R_missing = is.na(Solar.R)) |>   # keep an indicator before filling
  ungroup() |>
  mutate(Solar.R = replace_na(Solar.R, median(Solar.R, na.rm = TRUE)))
anyNA(imputed$Ozone)                 # FALSE

# For modelling: impute inside a recipe so it is learned on training data only
library(recipes)
rec <- recipe(Ozone ~ ., data = airquality) |>
  step_impute_median(all_numeric_predictors())

Choosing a Method

  • Few missing, random: drop rows or simple imputation.

  • Many missing or informative missingness: indicators plus imputation, or models that handle NA.

  • Time series: carry forward or interpolate (zoo::na.approx).

  • Inference: multiple imputation (mice) to keep uncertainty honest.

Common Pitfalls

  • Comparing with x == NA (always NA); use is.na(x).

  • Imputing before splitting into train and test.

  • Forgetting that lm() silently drops rows with NA (na.action = na.omit).

Interview Tips

  • Mention na.rm, drop_na, replace_na and mice.

  • Discuss missingness mechanisms (MCAR, MAR, MNAR).

7

What is the apply family of functions in R?

The apply family is a set of base R functions that apply a function over the elements, rows, columns or groups of a data structure without writing an explicit loop: apply() works on matrix rows or columns, lapply() on each list element and returns a list, sapply() simplifies that result to a vector or matrix, vapply() does the same with a declared output type, mapply() iterates over several inputs in parallel, and tapply() applies a function to groups. They make code shorter and clearer; the tidyverse equivalent is purrr’s map() family.

Read full answer

Applies toR 4.4+, purrr 1.0+

The Functions

Function Input Output Example
apply(X, MARGIN, FUN) Matrix / array Vector, matrix Row means: apply(m, 1, mean)
lapply(X, FUN) List or vector List lapply(list_of_dfs, nrow)
sapply(X, FUN) List or vector Simplified vector/matrix sapply(mtcars, mean)
vapply(X, FUN, FUN.VALUE) List or vector Type-checked vector vapply(mtcars, mean, numeric(1))
mapply(FUN, ...) Several vectors Vector or list mapply(rep, 1:3, 3:1)
tapply(X, INDEX, FUN) Vector + grouping Array by group tapply(mtcars$mpg, mtcars$cyl, mean)
Map(), Reduce(), Filter() Functional helpers Reduce("+", 1:4) is 10

Code Example

m <- matrix(1:6, nrow = 2)       # 2 x 3: columns (1,2), (3,4), (5,6)
apply(m, 1, sum)                 # row sums: 9 12
apply(m, 2, max)                 # column max: 2 4 6

lapply(1:3, function(x) x^2)     # list(1, 4, 9)
sapply(1:3, function(x) x^2)     # 1 4 9
vapply(mtcars[, 1:3], mean, numeric(1))   # mpg, cyl, disp means; guaranteed numeric

round(tapply(mtcars$mpg, mtcars$cyl, mean), 1)   # 4: 26.7  6: 19.7  8: 15.1

mapply(function(x, y) x + y, 1:3, 4:6)   # 5 7 9
mapply(rep, 1:3, 3:1)                    # list(c(1, 1, 1), c(2, 2), 3)

# purrr equivalents
library(purrr)
map_dbl(mtcars[, 1:3], mean)
map2_dbl(1:3, 4:6, ~ .x + .y)            # 5 7 9

When to Use Which

  • vapply() in functions and packages (safe output type).

  • sapply() interactively.

  • apply() for matrices, not data frames (it converts them to a matrix, coercing types).

  • Vectorized functions (rowSums, colMeans) are faster than apply() for simple sums and means.

  • For grouped data frames, dplyr::group_by() + summarise() is usually clearer than tapply().

Common Pitfalls

  • sapply() returning a list unexpectedly when results have different lengths.

  • Using apply() on a data frame with character columns, which turns everything into text.

  • Assuming apply functions are always faster than loops; they are mainly clearer.

Interview Tips

  • Explain the output type of each function.

  • Mention vapply() for safety and purrr’s map_*() as the modern alternative.

8

How does variable scope work in R?

R uses lexical (static) scoping: when a function looks up a variable, it searches first in its own local environment, then in the environment where the function was defined (its enclosing environment), and so on up to the global environment, attached packages and base R. Assignments with <- inside a function create local variables that disappear when the function returns, the super-assignment operator <<- modifies a variable in an enclosing environment, and functions that capture their defining environment are closures, which makes function factories possible.

Read full answer

Applies toR 4.4+

Environments

Environment Contents
Local (execution) Arguments and variables created in the current function call
Enclosing Environment where the function was defined
Global (globalenv()) Objects created at the console or top of a script
Package / search path Attached packages, then base
Empty End of the chain

Rules

  • Name lookup: local → enclosing → … → global → search path.

  • Assignment: <- always creates or modifies in the current environment.

  • <<-: searches enclosing environments for the name and modifies it there (or creates it in global if not found).

  • Fresh start: each call gets a new local environment, so variables don’t persist between calls (unless stored in an enclosing environment).

  • Lazy evaluation: arguments are evaluated only when used.

Code Example

x <- 10                         # global

f <- function() {
  x <- 20                       # local, shadows the global x
  g <- function() x + 1         # g looks up x in f's environment
  g()
}
f()                             # 21
x                               # 10: global unchanged

h <- function() x * 2           # no local x: uses the global
h()                             # 20

# Closures: a function factory
make_counter <- function() {
  count <- 0
  function() {
    count <<- count + 1         # modifies count in the enclosing environment
    count
  }
}
counter <- make_counter()
counter()                       # 1
counter()                       # 2

# Inspecting environments
environment(counter)            # the factory's environment
exists("count")                 # FALSE in the global environment

Dynamic Lookup at Call Time

  • Lookup happens when the function runs, not when it is defined, so a global variable changed later affects the result. Pass values as arguments to avoid this.

Common Pitfalls

  • Functions that depend on global variables silently; pass everything as arguments.

  • Overusing <<-, which makes code hard to follow; prefer returning values.

  • Expecting a variable created inside a function to exist afterwards.

Interview Tips

  • Name it “lexical scoping” and walk through the lookup order.

  • Show a closure with <<- as the classic example.

9

How do you read and write data in R?

R reads and writes data with base functions such as read.csv()/write.csv() and readRDS()/saveRDS(), and with faster, more consistent packages: readr (read_csv(), write_csv()) for text files, readxl and writexl for Excel, data.table’s fread()/fwrite() for very large CSVs, arrow for Parquet and Feather, haven for SPSS, Stata and SAS files, jsonlite for JSON, and DBI with a driver for databases. Use .rds to save a single R object exactly as it is, and Parquet to share large tables with other tools.

Read full answer

Applies toR 4.4+, readr 2.1+, readxl 1.4+, arrow 16+, DBI 1.2+

Functions by Format

Format Read Write
CSV readr::read_csv(), read.csv(), data.table::fread() readr::write_csv(), write.csv(), fwrite()
Excel readxl::read_excel() writexl::write_xlsx(), openxlsx
R object readRDS() saveRDS()
Several objects load() save()
Parquet / Feather arrow::read_parquet() arrow::write_parquet()
JSON jsonlite::fromJSON() jsonlite::toJSON(), write_json()
SPSS / Stata / SAS haven::read_sav(), read_dta(), read_sas() haven::write_sav(), write_dta()
Databases DBI::dbGetQuery(), dbReadTable() DBI::dbWriteTable()

Code Example

library(readr)

path <- tempfile(fileext = ".csv")
write_csv(mtcars, path)                          # write a CSV
cars <- read_csv(path, show_col_types = FALSE)   # read it back as a tibble
dim(cars)                                        # 32 11

# Specify column types explicitly for reliable imports
spec <- cols(mpg = col_double(), cyl = col_integer(), .default = col_double())
cars <- read_csv(path, col_types = spec)

# Exact R object round trip (keeps factors, attributes)
rds <- tempfile(fileext = ".rds")
saveRDS(iris, rds)
identical(readRDS(rds), iris)                    # TRUE

# Parquet for large data and other tools
# arrow::write_parquet(cars, "cars.parquet"); arrow::read_parquet("cars.parquet")

# Database with DBI (SQLite in memory)
library(DBI)
con <- dbConnect(RSQLite::SQLite(), ":memory:")
dbWriteTable(con, "mtcars", mtcars)
dbGetQuery(con, "SELECT cyl, AVG(mpg) AS mpg FROM mtcars GROUP BY cyl")
dbDisconnect(con)

Choosing

  • Small to medium CSVs: readr (clear types, progress, tibbles).

  • Very large CSVs: data.table::fread() or arrow::open_dataset().

  • Exchanging with Python or Spark: Parquet.

  • Saving R work: .rds.

Common Pitfalls

  • write.csv() adds row names by default; use row.names = FALSE or readr.

  • Locale issues with decimal commas; use read_csv2() or locale(decimal_mark = ",").

  • Relying on guessed column types for IDs with leading zeros; read them as character.

Interview Tips

  • Mention readr, readxl, arrow and DBI as the modern toolkit.

  • Explain why .rds and Parquet are better than CSV for some uses.

10

What are the key differences between R and Python for machine learning?

R and Python can both do machine learning well, but they lean in different directions: R excels at statistical modelling, inference, visualization and reproducible reporting, with tidymodels and caret as unified ML frameworks, while Python dominates deep learning, large-scale and production machine learning, with scikit-learn, PyTorch, TensorFlow and a broad engineering ecosystem. Many teams use R for analysis and experimentation and Python for deep learning and deployment, and the two interoperate through reticulate, Arrow and shared file formats.

Read full answer

Applies toR 4.4+ (tidymodels 1.2), Python 3.12 (scikit-learn 1.5+)

Comparison

Aspect R Python
Origin and community Statisticians, researchers Software engineers, ML researchers
Classical ML tidymodels, caret, mlr3, ranger, xgboost scikit-learn, XGBoost, LightGBM
Statistical inference Excellent (lm, glm, mixed models, survival) Good (statsmodels)
Deep learning torch for R, keras3 (smaller ecosystem) PyTorch, TensorFlow, JAX, Hugging Face
Visualization ggplot2 Matplotlib, seaborn, plotly
Data wrangling dplyr, data.table pandas, Polars
Reporting R Markdown, Quarto, Shiny Jupyter, Quarto, Streamlit, Dash
Deployment Plumber APIs, Shiny, vetiver FastAPI, MLflow, BentoML, cloud SDKs
Big data sparklyr, arrow, duckdb PySpark, Dask, Ray, Polars
Packages CRAN, Bioconductor PyPI, conda
Indexing 1-based 0-based

Same Task in Both

library(tidymodels)
set.seed(42)
split <- initial_split(iris, prop = 0.8, strata = Species)
fit <- rand_forest(trees = 300) |>
  set_engine("ranger") |>
  set_mode("classification") |>
  fit(Species ~ ., data = training(split))
predict(fit, testing(split)) |>
  bind_cols(testing(split)) |>
  accuracy(truth = Species, estimate = .pred_class)
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
model = RandomForestClassifier(n_estimators=300, random_state=42).fit(X_tr, y_tr)
print(round(model.score(X_te, y_te), 3))

When to Choose Which

  • R: statistical analysis, clinical and academic research, rich visual reports, Shiny dashboards.

  • Python: deep learning, NLP and computer vision, integrating ML into software and production systems.

  • Both: reticulate calls Python from R; Arrow and Parquet share data.

Common Pitfalls

  • Claiming one language is universally better.

  • Ignoring the team’s skills and the existing production stack.

Interview Tips

  • Give an honest, balanced comparison and pick based on the task.

  • Mention interoperability (reticulate, Arrow).

11

How do you select a subset of a data frame in R?

You subset a data frame in R either with base indexing, df[rows, columns] using row numbers, logical conditions or column names, df$col and df[["col"]] for single columns, and the subset() function, or with dplyr verbs: filter() for rows, select() for columns, slice() for row positions and distinct() for unique rows. Base indexing needs no packages, while dplyr reads like a sentence, uses column names directly and chains with the pipe.

Read full answer

Applies toR 4.4+, dplyr 1.1+

Base R

Goal Code
Rows by condition mtcars[mtcars$mpg > 25, ]
Columns by name mtcars[, c("mpg", "hp")]
Rows and columns mtcars[mtcars$cyl == 4, c("mpg", "wt")]
By position mtcars[1:5, 1:3]
Drop columns mtcars[, !(names(mtcars) %in% c("vs", "am"))]
One column as vector mtcars$mpg, mtcars[["mpg"]]
subset() subset(mtcars, mpg > 25, select = c(mpg, hp))

dplyr

Goal Code
Rows filter(mtcars, mpg > 25, cyl == 4)
Columns select(mtcars, mpg, hp), select(mtcars, starts_with("d"))
Positions slice(mtcars, 1:5), slice_max(mtcars, mpg, n = 3)
Random rows slice_sample(mtcars, n = 5)
Unique rows distinct(mtcars, cyl, gear)

Code Example

library(dplyr)

# Base R: logical row index, named columns
mtcars[mtcars$mpg > 30, c("mpg", "hp")]
# 4 cars: Fiat 128 (32.4), Honda Civic (30.4), Toyota Corolla (33.9), Lotus Europa (30.4)

# Watch out for NA in conditions: which() drops them
airquality[which(airquality$Ozone > 100), c("Ozone", "Month")]

# dplyr: the same idea, chained
mtcars |>
  filter(cyl == 4, hp > 90) |>
  select(mpg, hp, wt) |>
  arrange(desc(mpg))

# Select helpers
iris |> select(starts_with("Petal"), Species) |> head(3)

# Top 3 by a column
mtcars |> slice_max(mpg, n = 3)

Comparison

Aspect Base R dplyr
Dependencies None dplyr
NA in conditions Returns NA rows unless which() filter() drops NA rows
Row names Kept Dropped for tibbles (use tibble::rownames_to_column())
Readability Compact, more symbols Verb-based, chainable

Common Pitfalls

  • Forgetting the comma: df[df$x > 1] selects columns, not rows.

  • df[, "col"] returns a vector for data frames (use drop = FALSE to keep a data frame).

  • subset() inside functions can behave unexpectedly because of non-standard evaluation.

Interview Tips

  • Show both base and dplyr, and mention the NA behaviour difference.

  • Use which() or filter() for conditions on columns with missing values.

12

How is the dplyr package used for data manipulation in R?

dplyr is the tidyverse package for data manipulation, built around a small set of verbs that each do one thing: filter() keeps rows, select() picks columns, mutate() adds or changes columns, arrange() sorts, summarise() reduces groups to summaries, and group_by() makes the other verbs work per group, with joins (left_join() and others) to combine tables. The verbs take a data frame first and return a data frame, so they chain with the pipe (|> or %>%) into readable pipelines, and the same code can run against databases through dbplyr.

Read full answer

Applies toR 4.4+, dplyr 1.1+

Core Verbs

Verb Purpose Example
filter() Rows by condition filter(mpg > 20)
select() Columns select(mpg, cyl, starts_with("d"))
mutate() New or changed columns mutate(kpl = mpg * 0.425)
arrange() Sort arrange(desc(mpg))
summarise() Aggregate summarise(mean_mpg = mean(mpg))
group_by() Per-group operations group_by(cyl)
count() Group counts count(cyl, gear)
distinct() Unique rows distinct(cyl)
rename() Rename columns rename(miles_per_gallon = mpg)
across() Apply to many columns mutate(across(where(is.numeric), scale))
Joins Combine tables left_join(x, y, by = "id")

Code Example

library(dplyr)

mtcars |>
  mutate(kpl = round(mpg * 0.425, 1),               # km per litre
         power_to_weight = hp / wt) |>
  filter(am == 1) |>                                # manual transmission
  group_by(cyl) |>
  summarise(n = n(),
            mean_kpl = mean(kpl),
            max_ptw = max(power_to_weight),
            .groups = "drop") |>
  arrange(desc(mean_kpl))

# Several columns at once
iris |>
  group_by(Species) |>
  summarise(across(starts_with("Sepal"), list(mean = mean, sd = sd)), .groups = "drop")

# Window functions inside groups
mtcars |>
  group_by(cyl) |>
  mutate(rank_in_cyl = min_rank(desc(mpg))) |>
  filter(rank_in_cyl == 1) |>
  select(cyl, mpg)

# Conditional columns
mtcars |> mutate(efficiency = case_when(mpg >= 25 ~ "high",
                                        mpg >= 18 ~ "medium",
                                        .default = "low")) |> count(efficiency)

Why Use It

  • Readable pipelines that mirror the analysis steps.

  • Consistent: first argument is data, columns referred to by name.

  • Fast (C++ backend), and works on databases (dbplyr), data.table (dtplyr) and Arrow/DuckDB.

Common Pitfalls

  • Forgetting to ungroup() (or .groups = "drop"), so later operations stay grouped.

  • dplyr::filter masks stats::filter; use dplyr::filter() if both are needed.

  • Using $ inside verbs (filter(df$x > 1)); refer to columns directly.

Interview Tips

  • Name the core verbs and group_by() + summarise() pattern.

  • Mention across() and case_when() for modern dplyr.

13

How do you reshape data with the tidyr package?

tidyr reshapes data between wide and long formats with pivot_longer(), which gathers several columns into name-value pairs (wide → long), and pivot_wider(), which spreads name-value pairs into columns (long → wide); it also splits and combines columns with separate_wider_delim() and unite(), and handles nested data with nest(), unnest() and missing combinations with complete(). The goal is “tidy data”: each variable a column, each observation a row and each value a cell, which is the format dplyr, ggplot2 and modelling functions expect.

Read full answer

Applies toR 4.4+, tidyr 1.3+

Key Functions

Function Purpose
pivot_longer() Wide → long (replaces gather())
pivot_wider() Long → wide (replaces spread())
separate_wider_delim() Split one column into several by a delimiter
unite() Combine columns into one
nest() / unnest() Create or expand list-columns
complete() Add missing combinations of values
fill() Fill missing values with previous/next values
drop_na() / replace_na() Handle missing values

Code Example

library(tidyr)
library(dplyr)

sales_wide <- tibble(
  store = c("A", "B"),
  q1 = c(100, 80),
  q2 = c(120, 90),
  q3 = c(130, 95)
)

# Wide -> long
sales_long <- sales_wide |>
  pivot_longer(cols = q1:q3, names_to = "quarter", values_to = "sales")
sales_long        # 6 rows: store, quarter, sales

# Long -> wide
sales_long |> pivot_wider(names_from = quarter, values_from = sales)

# Names that carry two variables: "Sepal.Length" -> part + measure
iris |>
  mutate(id = row_number()) |>
  pivot_longer(-c(id, Species), names_to = c("part", "measure"), names_sep = "\\.") |>
  head(4)

# Split and combine columns
people <- tibble(name = c("Ana Silva", "Ben Ito"))
people |> separate_wider_delim(name, delim = " ", names = c("first", "last"))

# Fill in missing combinations
tibble(store = c("A", "A", "B"), quarter = c("q1", "q2", "q1"), sales = c(1, 2, 3)) |>
  complete(store, quarter, fill = list(sales = 0))   # adds B-q2 with 0

When to Reshape

Need Format
ggplot2 with colour/facet by variable Long
Grouped summaries across measures Long
Report tables, correlation matrices Wide
Model with one column per feature Wide

Common Pitfalls

  • pivot_wider() with duplicate keys creates list-columns; aggregate first or use values_fn = sum.

  • Using the old gather()/spread(), which are superseded.

  • Forgetting names_sep/names_pattern when column names hold several variables.

Interview Tips

  • Define tidy data and when you’d pivot each way.

  • Mention values_fn for duplicate keys.

14

What does the aggregate() function do in R?

aggregate() is a base R function that splits data into groups, applies a summary function such as mean, sum or length to each group, and returns the results as a data frame; it is most often used with a formula, aggregate(y ~ group, data = df, FUN = mean), which reads “summarise y by group”. It handles several grouping variables and several value columns at once, and is the base R counterpart of dplyr’s group_by() + summarise().

Read full answer

Applies toR 4.4+

Syntax

Form Example
Formula aggregate(mpg ~ cyl, data = mtcars, FUN = mean)
Several groups aggregate(mpg ~ cyl + am, data = mtcars, FUN = mean)
Several values aggregate(cbind(mpg, hp) ~ cyl, data = mtcars, FUN = mean)
All other columns aggregate(. ~ Species, data = iris, FUN = mean)
Data frame + by list aggregate(mtcars$mpg, by = list(cyl = mtcars$cyl), FUN = mean)
Extra arguments to FUN aggregate(Ozone ~ Month, airquality, mean, na.action = na.omit)

Code Example

aggregate(mpg ~ cyl, data = mtcars, FUN = mean)
#   cyl      mpg
# 1   4 26.66364
# 2   6 19.74286
# 3   8 15.10000

aggregate(cbind(mpg, hp) ~ cyl + am, data = mtcars, FUN = mean)   # 6 groups

aggregate(. ~ Species, data = iris, FUN = mean)                  # every numeric column by species

# Several statistics: FUN returns a vector, giving a matrix column
res <- aggregate(mpg ~ cyl, data = mtcars, FUN = function(x) c(mean = mean(x), n = length(x)))
res$mpg                                                           # matrix with mean and n

# The formula method drops rows with NA by default (na.action = na.omit)
aggregate(Ozone ~ Month, data = airquality, FUN = mean)

# dplyr equivalent
library(dplyr)
mtcars |> group_by(cyl) |> summarise(mpg = mean(mpg))

aggregate() vs Alternatives

Tool Notes
aggregate() Base R, formula interface, data frame output
tapply() One value column, returns an array
dplyr::summarise() Readable, several named summaries, works on databases
data.table dt[, .(m = mean(x)), by = g] Fastest on large data

Common Pitfalls

  • Functions returning several values produce a matrix column, which surprises later code; use dplyr for multiple summaries.

  • The formula method silently drops rows with any NA in used columns.

  • Very slow on large data compared with data.table or dplyr.

Interview Tips

  • Show the formula form and cbind() for several columns.

  • Mention the NA behaviour and the dplyr equivalent.

15

How do you merge data frames in R?

Data frames are merged in R with base merge(x, y, by = "key"), where all.x = TRUE, all.y = TRUE or all = TRUE turn the default inner join into a left, right or full outer join, or with dplyr’s join functions, inner_join(), left_join(), right_join(), full_join(), plus the filtering joins semi_join() and anti_join(). Rows are matched on one or more key columns; stacking data frames with the same columns is done with rbind() or dplyr::bind_rows(), and side by side with cbind() or bind_cols().

Read full answer

Applies toR 4.4+, dplyr 1.1+

Join Types

Join Keeps dplyr Base merge()
Inner Matching rows in both inner_join(x, y, by = "id") merge(x, y, by = "id")
Left All of x left_join() all.x = TRUE
Right All of y right_join() all.y = TRUE
Full All of both full_join() all = TRUE
Semi x rows with a match (x columns only) semi_join() —
Anti x rows without a match anti_join() —

Code Example

library(dplyr)

customers <- tibble(id = c(1, 2, 3), name = c("Ana", "Ben", "Cai"))
orders <- tibble(order_id = 101:104, customer_id = c(1, 1, 3, 4), amount = c(20, 35, 15, 50))

inner_join(customers, orders, by = join_by(id == customer_id))   # 3 rows (Ana x2, Cai)
left_join(customers, orders, by = join_by(id == customer_id))    # 4 rows: Ben with NA amount
full_join(customers, orders, by = join_by(id == customer_id))    # 5 rows: includes customer 4
anti_join(customers, orders, by = join_by(id == customer_id))    # Ben: no orders
semi_join(customers, orders, by = join_by(id == customer_id))    # Ana, Cai

# Base R equivalent of the left join
merge(customers, orders, by.x = "id", by.y = "customer_id", all.x = TRUE)

# Detect unexpected duplicate keys (one-to-one expected)
left_join(customers, orders, by = join_by(id == customer_id), relationship = "one-to-many")

# Stacking rows and columns
bind_rows(customers, tibble(id = 4, name = "Dee"))

Merge Checks

  • Count rows before and after; joins on duplicated keys multiply rows.

  • Use relationship = (dplyr 1.1+) to declare and verify the expected cardinality.

  • Use anti_join() to find keys that don’t match (typos, different types).

  • Make key types match (character vs numeric IDs).

Common Pitfalls

  • Many-to-many joins that silently explode the row count (dplyr warns).

  • Joining on columns with the same name that aren’t keys, creating .x/.y suffixes.

  • Factor vs character keys with different levels.

Interview Tips

  • Draw the join types and mention semi/anti joins.

  • Stress checking row counts and key uniqueness.

16

How do you handle categorical variables in R models?

In R, categorical variables are stored as factors, and modelling functions such as lm() and glm() automatically turn a factor into dummy (indicator) variables using treatment contrasts, with the first level as the reference; you control this by setting the reference with relevel() or forcats::fct_relevel(), ordering levels, grouping rare levels with fct_lump_n(), and choosing encodings such as one-hot dummies, contrasts, or target and embedding encodings for high-cardinality variables. In tidymodels, recipes steps like step_dummy(), step_other() and step_novel() make these encodings reproducible on new data.

Read full answer

Applies toR 4.4+, forcats 1.0+, recipes 1.1+

Strategies

Strategy Tool When
Factor with treatment contrasts factor(), used by lm()/glm() Default for statistical models
Set the reference level relevel(f, ref = "b"), fct_relevel() Interpretable comparisons
One-hot / dummy variables model.matrix(), step_dummy(one_hot = TRUE) Models needing numeric input (glmnet, xgboost)
Ordered factors factor(x, ordered = TRUE) Ordinal data (low < medium < high)
Group rare levels fct_lump_n(), step_other() Many levels with few rows
Unseen levels at prediction step_novel(), step_unknown() Production data
Target / effect encoding embed::step_lencode_glm() High cardinality
Native handling ranger, rpart, lightgbm handle factors Tree models

Code Example

library(forcats)

df <- transform(mtcars, cyl = factor(cyl), am = factor(am, labels = c("auto", "manual")))
fit <- lm(mpg ~ cyl + am + wt, data = df)
coef(fit)            # cyl6 and cyl8 are compared with the reference level cyl4

df$cyl <- relevel(df$cyl, ref = "8")
coef(lm(mpg ~ cyl + am + wt, data = df))   # now compared with 8 cylinders

head(model.matrix(~ cyl + am, data = df))   # the dummy columns R creates

# Group rare levels
fct_lump_n(factor(c("a", "a", "a", "b", "b", "c", "d")), n = 2)   # c and d become "Other"

# tidymodels recipe for a model needing numeric input
library(recipes)
rec <- recipe(Species ~ ., data = iris) |>
  step_novel(all_nominal_predictors()) |>
  step_other(all_nominal_predictors(), threshold = 0.05) |>
  step_dummy(all_nominal_predictors(), one_hot = FALSE)

Interpreting Dummy Coefficients

  • Each coefficient is the difference from the reference level, holding other variables constant.

  • Test the overall factor effect with anova() or car::Anova(), not single coefficients.

Common Pitfalls

  • Numeric codes (1, 2, 3) treated as continuous; convert to factor.

  • Full one-hot encoding plus an intercept in linear models (perfect collinearity); R’s default drops one level.

  • New levels at prediction time causing errors; handle them in the recipe.

Interview Tips

  • Explain treatment contrasts and reference levels.

  • Mention recipes for consistent preprocessing.

17

How do you scale or normalize data in R?

In R you standardize numeric data (mean 0, standard deviation 1) with the base scale() function, normalize to the 0-1 range with min-max scaling (x − min) / (max − min), and apply either consistently in modelling with tidymodels recipes (step_normalize(), step_range()) or caret’s preProcess(), which learn the parameters from the training data and reuse them on new data. Scaling matters for distance- and gradient-based methods (k-NN, k-means, SVM, PCA, regularized regression, neural networks) but not for tree-based models.

Read full answer

Applies toR 4.4+, recipes 1.1+, caret 6.0-94

Methods

Method Formula R
Standardization (z-score) (x − mean) / sd scale(x), step_normalize()
Min-max (x − min) / (max − min) step_range(min = 0, max = 1)
Robust (x − median) / IQR custom mutate()
Log log(x) or log1p(x) step_log()
Yeo-Johnson / Box-Cox Power transforms step_YeoJohnson(), step_BoxCox()

Code Example

library(dplyr)

z <- scale(mtcars$mpg)                 # matrix with attributes "scaled:center" and "scaled:scale"
round(c(mean(z), sd(z)), 3)            # 0 1
attr(z, "scaled:center")               # 20.09062 (the mean of mpg)

min_max <- function(x) (x - min(x, na.rm = TRUE)) / (max(x, na.rm = TRUE) - min(x, na.rm = TRUE))
range(min_max(mtcars$hp))              # 0 1

mtcars_std <- mtcars |> mutate(across(c(mpg, hp, wt), ~ as.numeric(scale(.x))))

# Learn scaling on training data, apply to test data
library(recipes)
set.seed(42)
idx <- sample(nrow(iris), 120)
train <- iris[idx, ]
test <- iris[-idx, ]
rec <- recipe(Species ~ ., data = train) |>
  step_normalize(all_numeric_predictors()) |>
  prep(training = train)               # means and sds from training data only
train_std <- bake(rec, new_data = NULL)
test_std <- bake(rec, new_data = test) # same parameters applied to test data
round(colMeans(test_std[, 1:4]), 2)    # near 0 but not exactly

When to Scale

Model Scaling needed?
k-NN, k-means, SVM, PCA Yes
Ridge, lasso, elastic net Yes
Neural networks Yes
Linear regression (unregularized) No for predictions; helps interpretation
Decision trees, random forests, gradient boosting No

Common Pitfalls

  • Scaling the full dataset before splitting (leakage).

  • scale() returns a matrix; wrap with as.numeric() inside mutate().

  • Min-max scaling is sensitive to outliers.

Interview Tips

  • Explain fit-on-train, apply-to-test with prep()/bake().

  • Say which models need scaling and which don’t.

18

Why should you split data into training and test sets in R?

Splitting data into a training set and a test set is essential because a model must be judged on data it has never seen: performance on the training data is optimistic, since flexible models can memorize noise (overfitting), while performance on a held-out test set estimates how the model will do on new data. In R this is done with rsample::initial_split() (often stratified), caret::createDataPartition() or base sample(), with all preprocessing learned on the training set only and the test set used once at the end; cross-validation on the training set is used for tuning.

Read full answer

Applies toR 4.4+, rsample 1.2+, caret 6.0-94

Why It Matters

  • Honest evaluation: training error underestimates real error.

  • Detect overfitting: a big gap between training and test performance.

  • Model selection: compare models fairly on data none of them saw.

  • Avoid leakage: preprocessing (scaling, imputation, feature selection) must be learned on training data only.

Typical Splits

Scheme Use
70-80% train / 20-30% test Default for medium data
Train / validation / test Tuning on validation, final check on test
k-fold CV on training + test Best for small and medium data
Stratified split Classification, especially imbalanced
Time-based split Time series: train on the past
Group split Multiple rows per patient or customer

Code Example

library(rsample)
library(dplyr)

set.seed(42)
split <- initial_split(iris, prop = 0.8, strata = Species)
train <- training(split)
test <- testing(split)
nrow(train); nrow(test)                     # 120 and 30
count(train, Species)                       # about 40 of each species (stratified)

# Training vs test error shows overfitting
fit <- lm(mpg ~ poly(hp, 6), data = mtcars[1:24, ])
rmse <- function(obs, pred) sqrt(mean((obs - pred)^2))
rmse(mtcars$mpg[1:24], predict(fit))                        # small
rmse(mtcars$mpg[25:32], predict(fit, mtcars[25:32, ]))      # usually much larger

# Time-based split
ts_split <- initial_time_split(airquality, prop = 0.8)      # first 80% of rows to train

# Cross-validation folds on the training data for tuning
folds <- vfold_cv(train, v = 5, strata = Species)

Common Pitfalls

  • Using the test set repeatedly during tuning, which makes it a validation set.

  • Random splits for time series or grouped data.

  • Forgetting set.seed(), so splits can’t be reproduced.

  • Preprocessing before the split.

Interview Tips

  • Explain overfitting and data leakage in one sentence each.

  • Mention stratification and time-based splits as context-dependent choices.

19

What is the purpose of the caret package in R?

caret (Classification And REgression Training) gives R a single, consistent interface to hundreds of machine learning models: one train() function handles resampling (cross-validation, bootstrap), hyperparameter tuning over a grid, preprocessing (centering, scaling, imputation, PCA), and performance evaluation, with helpers for data splitting (createDataPartition()), feature selection and confusionMatrix(). It removes the need to learn each package’s own syntax; it is still widely used, though its author now develops tidymodels as the successor, so new projects often use tidymodels.

Read full answer

Applies toR 4.4+, caret 6.0-94

Main Functions

Function Purpose
createDataPartition() Stratified train/test split
trainControl() Resampling method, folds, repeats, summary function
train() Fit and tune any supported model (method = "rf", "glmnet", "svmRadial", "xgbTree", …)
preProcess argument "center", "scale", "knnImpute", "pca", "YeoJohnson"
predict() Predictions with the best model
confusionMatrix() Accuracy, kappa, sensitivity, specificity
varImp() Variable importance
rfe() Recursive feature elimination
resamples() Compare models’ resampling results

Code Example

library(caret)

set.seed(42)
idx <- createDataPartition(iris$Species, p = 0.8, list = FALSE)
train_data <- iris[idx, ]
test_data <- iris[-idx, ]

ctrl <- trainControl(method = "repeatedcv", number = 5, repeats = 3)

set.seed(42)
knn_fit <- train(Species ~ ., data = train_data, method = "knn",
                 preProcess = c("center", "scale"),
                 tuneGrid = data.frame(k = c(3, 5, 7, 9, 11)),
                 trControl = ctrl)
knn_fit$bestTune                         # chosen k

set.seed(42)
rf_fit <- train(Species ~ ., data = train_data, method = "rf",
                tuneLength = 3, trControl = ctrl)

summary(resamples(list(knn = knn_fit, rf = rf_fit)))   # compare CV accuracy

pred <- predict(knn_fit, test_data)
confusionMatrix(pred, test_data$Species)
varImp(rf_fit)

caret vs tidymodels

Aspect caret tidymodels
Style One big train() function Modular packages (recipes, parsnip, tune, yardstick)
Preprocessing preProcess argument recipes steps
Status Maintained, mature Active development, recommended for new work
Learning curve Low Moderate

Common Pitfalls

  • Forgetting set.seed() before train(), so folds differ between models.

  • Preprocessing outside train(), which leaks across folds.

  • Needing the underlying model package installed (e.g. randomForest for method = "rf").

Interview Tips

  • Describe trainControl() + train() as the core pattern.

  • Mention tidymodels as the modern successor.

20

How do you perform linear regression analysis in R?

Linear regression in R is done with lm(y ~ x1 + x2, data = df), which fits the model by ordinary least squares; summary() then reports the coefficients with standard errors, t-values and p-values, the residual standard error, R² and adjusted R², and the overall F-test, while confint() gives confidence intervals, predict() makes predictions, and plot(model) shows diagnostic plots to check the assumptions. The formula syntax supports factors, interactions (x1 * x2), polynomial terms (poly(x, 2) or I(x^2)) and transformations (log(y)).

Read full answer

Applies toR 4.4+ (stats), broom 1.0+

Workflow

  1. Explore relationships with plots and correlations.

  2. Fit: model <- lm(mpg ~ wt + hp, data = mtcars).

  3. Interpret: summary(model), confint(model).

  4. Check assumptions: linearity, independence, constant variance, normal residuals (plot(model)), influential points (Cook’s distance), multicollinearity (car::vif()).

  5. Compare models: anova(m1, m2), AIC, adjusted R².

  6. Predict: predict(model, newdata, interval = "prediction").

Code Example

model <- lm(mpg ~ wt + hp, data = mtcars)
summary(model)
# Coefficients (rounded): intercept 37.23, wt -3.88, hp -0.032; both predictors significant
# Multiple R-squared about 0.83

confint(model)                                  # 95% intervals for each coefficient

new_cars <- data.frame(wt = c(2.5, 3.5), hp = c(100, 150))
predict(model, new_cars, interval = "confidence")   # mean response with interval
predict(model, new_cars, interval = "prediction")   # individual car, wider interval

par(mfrow = c(2, 2))
plot(model)                                      # residuals vs fitted, Q-Q, scale-location, leverage
par(mfrow = c(1, 1))

# Factor and interaction
model2 <- lm(mpg ~ wt * factor(am) + hp, data = mtcars)
anova(model, model2)                             # does the extra term improve the fit?

# Tidy output
library(broom)
tidy(model, conf.int = TRUE)
glance(model)                                    # R-squared, AIC, BIC

Reading summary()

Output Meaning
Estimate Change in y for one unit of x, others held constant
Std. Error, t value, Pr(>|t|) Uncertainty and significance of each coefficient
Residual standard error Typical prediction error in y units
R² / adjusted R² Share of variance explained (adjusted penalizes extra terms)
F-statistic Whether the model beats an intercept-only model

Common Pitfalls

  • Interpreting coefficients causally from observational data.

  • Ignoring assumption checks and influential points.

  • Extrapolating beyond the range of the data.

  • Comparing models with different responses (y vs log(y)) by R².

Interview Tips

  • Walk through the summary() output confidently.

  • Distinguish confidence and prediction intervals.

21

How does hypothesis testing work in R?

Hypothesis testing in R follows the usual steps: state a null hypothesis (no effect or difference) and an alternative, choose a test that fits the data, run it with a built-in function such as t.test(), wilcox.test(), chisq.test(), prop.test(), cor.test() or aov(), and compare the reported p-value with a significance level (often 0.05) while also reading the effect size and confidence interval. A small p-value means the data would be unlikely if the null hypothesis were true; it doesn’t measure the size or importance of the effect, and it doesn’t prove the alternative.

Read full answer

Applies toR 4.4+ (stats)

Common Tests

Question Test R function
One mean vs a value One-sample t-test t.test(x, mu = 20)
Two group means Welch t-test (default) t.test(y ~ group, data = df)
Paired measurements Paired t-test t.test(after, before, paired = TRUE)
Non-normal two groups Wilcoxon rank-sum wilcox.test(y ~ group, data = df)
Several group means One-way ANOVA aov(y ~ group, data = df), then TukeyHSD()
Two categorical variables Chi-squared test chisq.test(table(a, b))
Small counts Fisher’s exact test fisher.test(table(a, b))
Proportions Proportion test prop.test(x = c(45, 30), n = c(100, 100))
Correlation Pearson / Spearman cor.test(x, y, method = "spearman")
Normality Shapiro-Wilk shapiro.test(x)

Code Example

# Do manual and automatic cars differ in fuel economy?
t.test(mpg ~ am, data = mtcars)
# Welch t-test: t about -3.77, p about 0.0014; mean mpg 17.1 (automatic) vs 24.4 (manual)

# Is transmission associated with the number of cylinders?
tab <- table(mtcars$am, mtcars$cyl)
chisq.test(tab)            # warns: expected counts are small
fisher.test(tab)           # exact alternative for small tables

# Do iris species differ in sepal length?
fit <- aov(Sepal.Length ~ Species, data = iris)
summary(fit)               # very small p-value
TukeyHSD(fit)              # which pairs differ, with adjusted intervals

# Correlation between weight and mpg
cor.test(mtcars$wt, mtcars$mpg)   # r about -0.87

# Adjusting for multiple tests
p.adjust(c(0.01, 0.04, 0.03), method = "holm")

Interpreting Results

  • p-value: probability of results at least this extreme if the null were true.

  • Confidence interval: plausible range for the effect; more informative than p alone.

  • Effect size: difference in means, Cohen’s d, odds ratio, correlation.

  • Assumptions: independence, approximate normality for t-tests (robust with larger samples), equal variances only for the classic Student test (var.equal = TRUE).

Common Pitfalls

  • Treating p > 0.05 as proof of no effect.

  • Running many tests without adjusting (p.adjust).

  • Ignoring assumptions or the warning from chisq.test() about small expected counts.

  • Paired data analysed as independent groups.

Interview Tips

  • Choose the test from the data type and design, then interpret the p-value and interval correctly.

  • Mention that t.test() uses Welch’s test by default.

22

What are common statistical distributions in R?

R provides common probability distributions through a consistent family of four functions per distribution: d* for the density or probability mass (dnorm), p* for the cumulative probability (pnorm), q* for quantiles (qnorm), and r* for random numbers (rnorm); the most used are the normal, binomial, Poisson, uniform, exponential, t, chi-squared, F, beta and gamma distributions. They are used to compute probabilities and critical values, simulate data, check model assumptions and build statistical tests.

Read full answer

Applies toR 4.4+ (stats)

The Four-Function Pattern

Prefix Meaning Example
d Density / probability mass dbinom(3, size = 10, prob = 0.5)
p Cumulative probability P(X ≤ x) pnorm(1.96) is about 0.975
q Quantile (inverse of p) qnorm(0.975) is about 1.96
r Random draws rnorm(100, mean = 0, sd = 1)

Common Distributions

Distribution Use Functions
Normal Measurements, errors dnorm, pnorm, qnorm, rnorm
Binomial Successes in n trials dbinom, pbinom, …
Poisson Counts per interval dpois, ppois, …
Uniform Equal likelihood dunif, runif
Exponential Waiting times dexp, rexp
t Small-sample means dt, pt, qt
Chi-squared Variances, contingency tests dchisq, pchisq
F Ratio of variances, ANOVA df, pf
Beta Probabilities, proportions dbeta, rbeta
Gamma Positive skewed data dgamma, rgamma
Negative binomial Overdispersed counts dnbinom

Code Example

pnorm(1.96)                       # 0.9750021
qnorm(0.975)                      # 1.959964
pnorm(110, mean = 100, sd = 15) - pnorm(90, mean = 100, sd = 15)   # about 0.495

dbinom(3, size = 10, prob = 0.5)  # 0.1171875 (= 120 / 1024)
pbinom(3, size = 10, prob = 0.5)  # 0.171875 (P(X <= 3) = 176 / 1024)

dpois(2, lambda = 4)              # about 0.1465
1 - ppois(6, lambda = 4)          # P(X > 6) about 0.111

qt(0.975, df = 10)                # 2.228139: critical t value for a 95% CI

set.seed(42)
x <- rnorm(1000, mean = 50, sd = 10)
c(mean = mean(x), sd = sd(x))     # close to 50 and 10

hist(x, breaks = 30, freq = FALSE)
curve(dnorm(x, 50, 10), add = TRUE, lwd = 2)   # overlay the theoretical density
qqnorm(x); qqline(x)              # check normality visually

Choosing a Distribution

  • Continuous and symmetric: normal (or t with heavy tails).

  • Counts: Poisson, or negative binomial when the variance exceeds the mean.

  • Yes/no out of n: binomial.

  • Positive and skewed: log-normal, gamma, exponential.

  • Fit candidates with MASS::fitdistr() or fitdistrplus and compare.

Common Pitfalls

  • Using dnorm() as a probability for continuous variables; densities aren’t probabilities.

  • Forgetting lower.tail = FALSE (or 1 − p) for upper-tail probabilities.

  • Poisson models for overdispersed counts.

Interview Tips

  • Explain the d/p/q/r naming convention; it applies to every distribution.

  • Show one calculation, such as a critical value with qnorm or qt.

23

How do you conduct time series analysis in R?

Time series analysis in R starts by storing the data as a time series object (base ts for regular series, or a tsibble in the tidyverts ecosystem), plotting it to see trend and seasonality, decomposing it (decompose(), stl(), or feasts::STL()), checking stationarity and autocorrelation (ACF and PACF plots, unit-root tests), fitting models such as exponential smoothing (ETS) and ARIMA (forecast::auto.arima() or fable::ARIMA()), checking the residuals, and forecasting with prediction intervals evaluated on a hold-out period. The forecast package and the newer fable framework cover most needs.

Read full answer

Applies toR 4.4+, forecast 8.23, fable 0.4 / tsibble 1.1

Steps

Step Base / forecast tidyverts
Create series ts(x, start = c(1949, 1), frequency = 12) as_tsibble()
Plot plot(), autoplot() autoplot()
Decompose decompose(), stl() model(STL())
Autocorrelation acf(), pacf() ACF(), PACF()
Stationarity tseries::adf.test(), forecast::ndiffs() features(unitroot_kpss)
Models ets(), auto.arima() ETS(), ARIMA()
Residual checks checkresiduals() gg_tsresiduals()
Forecast forecast(fit, h = 12) forecast(h = "1 year")
Accuracy accuracy(fc, test) accuracy()

Code Example

library(forecast)

air <- log(AirPassengers)                 # monthly 1949-1960; log stabilizes variance
plot(stl(air, s.window = "periodic"))      # trend, seasonal and remainder components

train <- window(air, end = c(1958, 12))
test <- window(air, start = c(1959, 1))

fit_arima <- auto.arima(train)            # chooses (p,d,q)(P,D,Q)[12] by AICc
fit_ets <- ets(train)
fc_arima <- forecast(fit_arima, h = length(test))
fc_ets <- forecast(fit_ets, h = length(test))

accuracy(fc_arima, test)                  # RMSE, MAE, MAPE on the test period
accuracy(fc_ets, test)
checkresiduals(fit_arima)                 # residuals should look like white noise

autoplot(fc_arima) + autolayer(test, series = "actual")
exp(fc_arima$mean[1:3])                   # back-transform the first forecasts to passengers

Key Concepts

  • Trend and seasonality: long-run movement and repeating patterns.

  • Stationarity: constant mean and variance; achieved by differencing (diff()), often needed for ARIMA.

  • ACF/PACF: help identify AR and MA orders.

  • Prediction intervals: widen with the horizon.

Common Pitfalls

  • Random train/test splits; always hold out the most recent period.

  • Forgetting to back-transform log forecasts.

  • Ignoring residual autocorrelation (the model missed structure).

Interview Tips

  • Walk through plot → decompose → model → check residuals → forecast and evaluate.

  • Mention auto.arima(), ETS and the tidyverts packages.

24

What is the role of the glm() function in R?

glm() fits generalized linear models, which extend linear regression to responses that are not normally distributed by combining a linear predictor with a link function and an error distribution chosen with the family argument: binomial for yes/no outcomes (logistic regression), poisson for counts, Gamma for positive skewed data, and gaussian for ordinary linear regression. It estimates coefficients by maximum likelihood (iteratively reweighted least squares) and works with the same tools as lm(): summary(), predict(), anova(), confint() and AIC().

Read full answer

Applies toR 4.4+ (stats), broom 1.0+

Families and Links

Family Response Default link Typical use
gaussian Continuous identity Linear regression
binomial 0/1 or proportions logit Classification, probabilities
poisson Counts log Event counts, rates (with offset)
quasipoisson Overdispersed counts log Variance larger than mean
Gamma Positive, skewed inverse Costs, durations
inverse.gaussian Positive, skewed 1/μ² Rarely used

For overdispersed counts, MASS::glm.nb() fits negative binomial regression.

Code Example

# Logistic regression: probability of a manual transmission
logit <- glm(am ~ wt + hp, data = mtcars, family = binomial)
summary(logit)                            # coefficients on the log-odds scale
exp(coef(logit))                          # odds ratios
predict(logit, data.frame(wt = 2.5, hp = 120), type = "response")   # a probability

# Poisson regression: counts of warp breaks by wool and tension
pois <- glm(breaks ~ wool + tension, data = warpbreaks, family = poisson)
summary(pois)
exp(coef(pois))                           # rate ratios
deviance(pois) / df.residual(pois)        # well above 1 signals overdispersion

quasi <- glm(breaks ~ wool + tension, data = warpbreaks, family = quasipoisson)
summary(quasi)$dispersion                 # estimated dispersion

# Model comparison
logit_small <- glm(am ~ wt, data = mtcars, family = binomial)
anova(logit_small, logit, test = "Chisq") # likelihood-ratio test
AIC(logit_small, logit)

Interpreting Coefficients

Family Coefficient meaning
binomial (logit) Change in log-odds; exp() gives an odds ratio
poisson (log) Change in log count; exp() gives a rate ratio
gaussian Change in the mean response

Common Pitfalls

  • Forgetting type = "response" in predict(), which returns the linear predictor (log-odds).

  • Ignoring overdispersion in Poisson models, which understates standard errors.

  • Perfect separation in logistic regression (“fitted probabilities numerically 0 or 1”); consider brglm2 or regularization.

  • Reading R² from glm(); use deviance, AIC or pseudo-R² instead.

Interview Tips

  • Explain family + link and give logistic and Poisson examples.

  • Mention odds ratios and type = "response".

25

What is multicollinearity and how do you address it in R?

Multicollinearity means two or more predictors in a regression are strongly correlated, so the model can’t separate their individual effects: coefficients become unstable, their standard errors inflate, signs can flip, and individual p-values look insignificant even when the predictors jointly matter, although predictions on similar data can still be fine. In R you detect it with correlation matrices and variance inflation factors (car::vif()), and address it by removing or combining redundant predictors, centering variables used in interactions or polynomials, using principal components, or using regularized regression such as ridge or the elastic net with glmnet.

Read full answer

Applies toR 4.4+, car 3.1+, glmnet 4.1+

Detection

Method Rule of thumb
Correlation matrix (cor()) Pairwise |r| above about 0.8 is a warning sign
VIF (car::vif()) Above 5 is concerning, above 10 serious
Condition number (kappa()) Large values (over about 30) suggest collinearity
Unstable coefficients Big changes when adding or removing variables

Remedies

Remedy When
Drop one of the correlated predictors Redundant measures (e.g. two weight variables)
Combine them (average, index) Several measures of one concept
Center before interactions/polynomials Structural collinearity from x and x^2
Principal components regression Many correlated predictors
Ridge or elastic net (glmnet) Prediction focus; shrinks correlated coefficients together
Collect more data Sometimes reduces variance

Code Example

library(car)

round(cor(mtcars[, c("disp", "hp", "wt", "cyl")]), 2)   # disp, cyl and wt strongly correlated

fit <- lm(mpg ~ disp + hp + wt + cyl, data = mtcars)
vif(fit)                     # disp has the highest VIF of the four

fit2 <- lm(mpg ~ hp + wt, data = mtcars)   # drop redundant size measures
vif(fit2)                    # much lower VIFs
summary(fit2)$coefficients

# Ridge regression keeps all predictors but shrinks them
library(glmnet)
x <- scale(as.matrix(mtcars[, c("disp", "hp", "wt", "cyl")]))
set.seed(42)
cv <- cv.glmnet(x, mtcars$mpg, alpha = 0)  # alpha = 0 is ridge
coef(cv, s = "lambda.min")

# Structural collinearity: center before squaring
mtcars$wt_c <- mtcars$wt - mean(mtcars$wt)
vif(lm(mpg ~ wt_c + I(wt_c^2), data = mtcars))   # lower than with raw wt and wt^2

When It Matters

  • Inference (which variables matter, coefficient interpretation): yes, address it.

  • Prediction on data with the same correlation structure: often not a problem.

Common Pitfalls

  • Dropping variables only by p-value when they are jointly important.

  • Using VIF thresholds mechanically; consider the research question.

  • Interpreting ridge coefficients like unbiased OLS estimates.

Interview Tips

  • Define the consequences (unstable estimates, inflated SEs) and VIF.

  • Distinguish inference from prediction goals.

26

How does k-means clustering work and how do you implement it in R?

k-means clustering partitions observations into k groups by repeatedly assigning each point to the nearest of k cluster centres (centroids) and then moving each centre to the mean of its assigned points, until assignments stop changing; it minimizes the total within-cluster sum of squared distances. In R it is run with the base kmeans() function on scaled numeric data, using several random starts (nstart) because results depend on initialization, and the number of clusters is chosen with the elbow method, silhouette widths or the gap statistic (for example via factoextra).

Read full answer

Applies toR 4.4+ (stats), cluster 2.1+, factoextra 1.0.7

Algorithm (Lloyd’s)

  1. Choose k and initial centres (random points; R’s default is the Hartigan-Wong algorithm).

  2. Assign each point to its nearest centre (Euclidean distance).

  3. Recompute each centre as the mean of its points.

  4. Repeat steps 2-3 until assignments stop changing or the iteration limit is reached.

Code Example

library(cluster)

X <- scale(iris[, 1:4])                    # k-means uses distances: scale first

set.seed(42)
km <- kmeans(X, centers = 3, nstart = 25)  # 25 random starts, keep the best
km$size                                    # cluster sizes
km$centers                                 # centres on the scaled data
km$tot.withinss                            # total within-cluster sum of squares

table(cluster = km$cluster, species = iris$Species)   # compare with known species

# Elbow method: within-cluster SS for k = 1..8
set.seed(42)
wss <- sapply(1:8, function(k) kmeans(X, centers = k, nstart = 25)$tot.withinss)
plot(1:8, wss, type = "b", xlab = "k", ylab = "Total within-cluster SS")

# Average silhouette width (higher is better)
sil <- silhouette(km$cluster, dist(X))
mean(sil[, "sil_width"])

# Optional visual helpers
# factoextra::fviz_nbclust(X, kmeans, method = "silhouette")
# factoextra::fviz_cluster(km, data = X)

Strengths and Limitations

Strengths Limitations
Simple, fast, scales to large data k must be chosen
Easy to interpret centres Assumes roughly spherical, similar-size clusters
Works well with scaled numeric features Sensitive to outliers and initialization
Numeric data only (use k-modes / PAM with Gower for mixed types)

Common Pitfalls

  • Not scaling variables, so large-range features dominate.

  • nstart = 1, giving a poor local optimum.

  • Forcing k-means on non-spherical clusters; try DBSCAN or hierarchical clustering.

Interview Tips

  • Explain assign-update iterations and the objective.

  • Mention nstart, scaling and choosing k.

27

How is the e1071 package used for machine learning in R?

e1071 is a long-standing CRAN package from TU Wien’s statistics department that provides several machine learning and statistics tools, most notably svm(), an interface to the LIBSVM library for support vector machines (classification, regression and novelty detection), together with naiveBayes(), tune() for grid-search hyperparameter tuning with cross-validation, fuzzy c-means clustering (cmeans()), and utilities such as skewness() and kurtosis(). It is often used directly or as the engine behind caret and tidymodels SVM models.

Read full answer

Applies toR 4.4+, e1071 1.7

Main Functions

Function Purpose
svm() Support vector machines: C-classification, nu-classification, eps-regression, one-classification
tune(), tune.svm() Grid search with cross-validation
naiveBayes() Naive Bayes classifier
cmeans() Fuzzy c-means clustering
skewness(), kurtosis() Distribution shape
bclust() Bagged clustering
hamming.distance() Distance for binary vectors

Code Example

library(e1071)

set.seed(42)
idx <- sample(nrow(iris), 120)
train <- iris[idx, ]
test <- iris[-idx, ]

# SVM with a radial kernel (features are scaled by default: scale = TRUE)
model <- svm(Species ~ ., data = train, kernel = "radial", cost = 1, gamma = 0.25,
             probability = TRUE)
pred <- predict(model, test)
mean(pred == test$Species)                       # test accuracy
table(predicted = pred, actual = test$Species)

# Tune cost and gamma with 10-fold cross-validation
set.seed(42)
tuned <- tune.svm(Species ~ ., data = train, cost = 2^(-1:4), gamma = 2^(-4:0))
tuned$best.parameters
best <- tuned$best.model

# Probabilities
attr(predict(model, test[1:3, ], probability = TRUE), "probabilities")

# Naive Bayes
nb <- naiveBayes(Species ~ ., data = train)
mean(predict(nb, test) == test$Species)

Common Pitfalls

  • Forgetting probability = TRUE when fitting if you need probabilities later.

  • Large data: kernel SVMs scale poorly with many rows; consider linear models or LiblineaR.

  • Untuned cost and gamma, which can underfit or overfit badly.

Interview Tips

  • Name svm() (LIBSVM) and tune() as the main uses.

  • Mention that caret’s "svmRadial" uses kernlab, while tidymodels can use kernlab or LiblineaR, as alternatives to e1071.

28

How do you implement a decision tree in R?

Decision trees in R are usually built with the rpart package, which grows a CART tree with rpart(y ~ ., data, method = "class") for classification or method = "anova" for regression, controls growth with rpart.control() (cp, minsplit, maxdepth), and prunes it back with prune() using the complexity parameter that minimizes cross-validated error from printcp(); rpart.plot draws readable diagrams. Alternatives are party::ctree() (conditional inference trees with statistical stopping rules) and tidymodels’ decision_tree() with the rpart engine.

Read full answer

Applies toR 4.4+, rpart 4.1.23, rpart.plot 3.1

Key Settings

Setting Meaning
method "class" (classification), "anova" (regression), "poisson", "exp"
cp Complexity parameter: a split must improve fit by this fraction
minsplit Minimum observations to attempt a split (default 20)
minbucket Minimum observations in a leaf
maxdepth Maximum depth
xval Number of cross-validation folds (default 10)

Code Example

library(rpart)
library(rpart.plot)

set.seed(42)
idx <- sample(nrow(iris), 105)
train <- iris[idx, ]
test <- iris[-idx, ]

tree <- rpart(Species ~ ., data = train, method = "class",
              control = rpart.control(cp = 0.001, minsplit = 10))
printcp(tree)                                   # cp table with cross-validated error (xerror)

best_cp <- tree$cptable[which.min(tree$cptable[, "xerror"]), "CP"]
pruned <- prune(tree, cp = best_cp)

rpart.plot(pruned, type = 4, extra = 104)       # class, probabilities and % of observations
pred <- predict(pruned, test, type = "class")
mean(pred == test$Species)                      # test accuracy
pruned$variable.importance

# Regression tree
reg <- rpart(mpg ~ ., data = mtcars, method = "anova", control = rpart.control(minsplit = 5))
predict(reg, mtcars[1:3, ])

Reading the Tree

  • Each node shows the predicted class, class probabilities and share of observations.

  • Splits read as rules, e.g. “Petal.Length < 2.5 → setosa”.

  • variable.importance sums the improvement of each variable across splits, including surrogate splits.

Common Pitfalls

  • Overfitting with a very small cp; prune with cross-validation.

  • Instability: small data changes can change the tree; use random forests for accuracy.

  • Using predict() without type = "class", which returns probabilities.

Interview Tips

  • Explain cp and pruning by cross-validated error.

  • Mention rpart.plot and that rpart handles missing values with surrogate splits.

29

What are the differences between random forest and gradient boosting in R?

Random forests (in R, ranger or randomForest) build many deep trees independently on bootstrap samples with random feature subsets and average them, reducing variance; gradient boosting (in R, xgboost, lightgbm or gbm) builds shallow trees one after another, each fitted to the errors of the current model with a small learning rate, reducing bias. Random forests are robust and need little tuning, while well-tuned boosting is often more accurate on tabular data but needs careful tuning and early stopping; both are available through tidymodels’ rand_forest() and boost_tree().

Read full answer

Applies toR 4.4+, ranger 0.16, xgboost 1.7 (R package), tidymodels 1.2

Comparison

Aspect Random forest Gradient boosting
R packages ranger (fast), randomForest xgboost, lightgbm, gbm
Training Parallel, independent trees Sequential
Tree depth Deep Shallow (depth 3-8)
Reduces Variance Bias
Key parameters num.trees, mtry, min.node.size nrounds, eta, max_depth, subsample, colsample_bytree
Overfitting with more trees No Yes; use early stopping
Tuning effort Low Higher
Validation OOB error built in Watchlist / CV with early stopping
Data input Formula / data frame xgboost needs a numeric matrix (xgb.DMatrix)

Code Example

library(ranger)
library(xgboost)

set.seed(42)
idx <- sample(nrow(mtcars), 24)
train <- mtcars[idx, ]
test <- mtcars[-idx, ]
rmse <- function(obs, pred) sqrt(mean((obs - pred)^2))

# Random forest
rf <- ranger(mpg ~ ., data = train, num.trees = 500, mtry = 3, importance = "permutation", seed = 42)
rf$prediction.error                               # OOB mean squared error
rmse(test$mpg, predict(rf, test)$predictions)

# Gradient boosting
dtrain <- xgb.DMatrix(as.matrix(train[, -1]), label = train$mpg)
dtest <- xgb.DMatrix(as.matrix(test[, -1]), label = test$mpg)
params <- list(objective = "reg:squarederror", eta = 0.05, max_depth = 3, subsample = 0.8)
set.seed(42)
cv <- xgb.cv(params, dtrain, nrounds = 1000, nfold = 5, early_stopping_rounds = 30, verbose = 0)
gb <- xgb.train(params, dtrain, nrounds = cv$best_iteration)
rmse(test$mpg, predict(gb, dtest))

# Same models through tidymodels
library(parsnip)
rand_forest(trees = 500, mtry = 3) |> set_engine("ranger") |> set_mode("regression")
boost_tree(trees = 500, learn_rate = 0.05, tree_depth = 3) |> set_engine("xgboost") |> set_mode("regression")

mtcars is tiny, so the comparison here only illustrates the code; differences appear on larger data.

Common Pitfalls

  • Boosting without early stopping or validation.

  • Comparing a tuned model with an untuned one.

  • Forgetting that xgboost needs numeric matrices (dummy-encode factors).

Interview Tips

  • Frame it as variance vs bias reduction and parallel vs sequential.

  • Name the R packages and their key parameters.

30

How do you train a support vector machine in R?

A support vector machine (SVM) in R is usually trained with e1071::svm() (LIBSVM) or kernlab::ksvm(), choosing a kernel (linear, radial, polynomial), scaling the features (done by default in svm()), and tuning the cost parameter C and kernel parameters such as gamma with cross-validation, for example with tune.svm(), caret’s train(method = "svmRadial") or tidymodels’ svm_rbf() with tune_grid(). The SVM finds the boundary with the widest margin between classes; the kernel lets it model non-linear boundaries.

Read full answer

Applies toR 4.4+, e1071 1.7, kernlab 0.9-33, tidymodels 1.2

Key Parameters

Parameter Effect
kernel "linear", "radial" (RBF), "polynomial", "sigmoid"
cost © Penalty for misclassification: high = narrow margin, risk of overfitting
gamma RBF width: high = very local, wiggly boundary
degree Polynomial kernel degree
class.weights Weights for imbalanced classes
probability Fit Platt scaling to produce probabilities

Code Example

library(tidymodels)

set.seed(42)
split <- initial_split(iris, prop = 0.8, strata = Species)
train <- training(split)
test <- testing(split)

rec <- recipe(Species ~ ., data = train) |> step_normalize(all_numeric_predictors())
svm_spec <- svm_rbf(cost = tune(), rbf_sigma = tune()) |>
  set_engine("kernlab") |>
  set_mode("classification")
wf <- workflow() |> add_recipe(rec) |> add_model(svm_spec)

set.seed(42)
folds <- vfold_cv(train, v = 5, strata = Species)
grid <- grid_regular(cost(), rbf_sigma(), levels = 5)
res <- tune_grid(wf, resamples = folds, grid = grid, metrics = metric_set(accuracy, roc_auc))
show_best(res, metric = "accuracy", n = 3)

final <- finalize_workflow(wf, select_best(res, metric = "accuracy")) |> last_fit(split)
collect_metrics(final)                       # test accuracy and ROC AUC

# The same with e1071 directly
library(e1071)
fit <- svm(Species ~ ., data = train, kernel = "radial", cost = 4, gamma = 0.25)
mean(predict(fit, test) == test$Species)

Choosing a Kernel

Kernel When
Linear Many features (text), roughly linear separation, large data
Radial (RBF) Default for non-linear problems
Polynomial Known polynomial interactions

Common Pitfalls

  • Not scaling features (kernlab via recipes needs explicit normalization).

  • Tuning C and gamma on the test set.

  • Kernel SVMs on very large datasets (training time grows faster than linearly with rows).

Interview Tips

  • Explain the margin, C and gamma in one sentence each.

  • Mention cross-validated tuning with tune.svm() or tidymodels.

31

How is a confusion matrix used in R?

A confusion matrix is a table that compares a classifier’s predicted classes with the actual classes, counting true positives, true negatives, false positives and false negatives (or every predicted-actual pair for multi-class problems), and it is the basis for metrics such as accuracy, precision, recall (sensitivity), specificity and F1. In R you can build one with base table(predicted, actual), with caret::confusionMatrix(), which also reports accuracy with a confidence interval, kappa, sensitivity and specificity, or with yardstick::conf_mat() in tidymodels.

Read full answer

Applies toR 4.4+, caret 6.0-94, yardstick 1.3+

Layout (Binary)

Actual positive Actual negative
Predicted positive TP FP
Predicted negative FN TN

Derived Metrics

Metric Formula Question answered
Accuracy (TP + TN) / total How often correct overall?
Precision TP / (TP + FP) When it says positive, how often right?
Recall / sensitivity TP / (TP + FN) How many positives were found?
Specificity TN / (TN + FP) How many negatives were correctly rejected?
F1 2 · precision · recall / (precision + recall) Balance of precision and recall
Kappa Agreement beyond chance Useful with imbalance

Code Example

actual <- factor(c("yes", "yes", "yes", "no", "no", "no", "no", "yes", "no", "no"),
                 levels = c("yes", "no"))
predicted <- factor(c("yes", "no", "yes", "no", "yes", "no", "no", "yes", "no", "no"),
                    levels = c("yes", "no"))

table(Predicted = predicted, Actual = actual)
#          Actual
# Predicted yes no
#       yes   3  1
#       no    1  5

library(caret)
cm <- confusionMatrix(predicted, actual, positive = "yes", mode = "everything")
cm$overall["Accuracy"]        # 0.8
cm$byClass[c("Sensitivity", "Specificity", "Precision", "F1")]
# Sensitivity 0.75, Specificity 0.8333, Precision 0.75, F1 0.75

library(yardstick)
df <- data.frame(truth = actual, estimate = predicted)
conf_mat(df, truth, estimate)
metrics <- metric_set(accuracy, precision, recall, f_meas)
metrics(df, truth = truth, estimate = estimate)   # yardstick treats the first level ("yes") as the event

Multi-Class

  • Rows and columns for every class; the diagonal holds correct predictions.

  • Per-class precision and recall, averaged as macro (equal weight per class) or weighted by class size.

Common Pitfalls

  • The positive class: caret and yardstick use the first factor level by default; set positive = or event_level =.

  • Accuracy on imbalanced data hides poor minority recall.

  • Swapping the predicted and actual arguments, which transposes the matrix.

Interview Tips

  • Compute precision and recall by hand from a 2×2 table.

  • Mention the positive-class level issue in R.

32

How do you calculate ROC and AUC in R?

An ROC curve plots the true positive rate against the false positive rate as the classification threshold varies, and the AUC (area under it) summarizes ranking quality: 1.0 is perfect, 0.5 is random guessing, and it equals the probability that a randomly chosen positive gets a higher score than a randomly chosen negative. In R you compute them from predicted probabilities with pROC::roc() and auc() (with confidence intervals via ci.auc()), or yardstick::roc_curve() and roc_auc() in tidymodels; ROCR is an older alternative.

Read full answer

Applies toR 4.4+, pROC 1.18, yardstick 1.3+

Code Example

library(pROC)

# Logistic regression for manual (1) vs automatic (0) transmission
set.seed(42)
idx <- sample(nrow(mtcars), 22)
train <- mtcars[idx, ]
test <- mtcars[-idx, ]
fit <- glm(am ~ wt + hp, data = train, family = binomial)
prob <- predict(fit, test, type = "response")       # probabilities, not classes

roc_obj <- roc(response = test$am, predictor = prob, levels = c(0, 1), direction = "<")
auc(roc_obj)                                         # area under the curve
ci.auc(roc_obj)                                      # DeLong confidence interval
plot(roc_obj, print.auc = TRUE, legacy.axes = TRUE)  # x axis as 1 - specificity
coords(roc_obj, "best", ret = c("threshold", "sensitivity", "specificity"))   # Youden's J

# Tiny hand-checkable example
scores <- c(0.9, 0.8, 0.7, 0.4, 0.3, 0.2)
labels <- c(1, 1, 0, 1, 0, 0)
auc(roc(labels, scores, levels = c(0, 1), direction = "<", quiet = TRUE))
# 8 of 9 positive-negative pairs ranked correctly: AUC = 0.889

# tidymodels / yardstick
library(yardstick)
df <- data.frame(truth = factor(labels, levels = c(1, 0)), .pred_1 = scores)
roc_auc(df, truth, .pred_1)                          # 0.889; first level "1" is the event
roc_curve(df, truth, .pred_1)                        # thresholds, sensitivity, specificity

Interpreting AUC

AUC Rough meaning
0.5 No better than random
0.7-0.8 Acceptable
0.8-0.9 Good
> 0.9 Excellent (check for leakage)

ROC vs Precision-Recall

  • ROC AUC can look high on very imbalanced data even when precision is poor.

  • Use precision-recall curves (yardstick::pr_auc(), PRROC) when positives are rare.

Common Pitfalls

  • Passing predicted classes instead of probabilities.

  • Wrong event level or direction, which flips AUC to 1 − AUC.

  • Choosing the threshold on the test set.

Interview Tips

  • Explain AUC as the probability of ranking a positive above a negative.

  • Mention levels and direction in pROC to avoid silent flips.

33

What cross-validation techniques are available in R?

R supports all the common cross-validation schemes: k-fold and repeated k-fold, stratified folds, leave-one-out (LOOCV), the bootstrap, Monte Carlo (repeated random) splits, group-based folds that keep related rows together, time-series rolling-origin splits, and nested cross-validation. They are available in tidymodels’ rsample (vfold_cv(), loo_cv(), bootstraps(), mc_cv(), group_vfold_cv(), rolling_origin(), nested_cv()) and in caret through trainControl(method = ...), and resampling results are then summarized across folds to estimate performance and tune models.

Read full answer

Applies toR 4.4+, rsample 1.2+, caret 6.0-94

Techniques

Technique rsample caret trainControl(method = ) Use
k-fold vfold_cv(v = 10) "cv" Default choice
Repeated k-fold vfold_cv(v = 10, repeats = 3) "repeatedcv" More stable estimates
Stratified vfold_cv(strata = y) Stratified by default for factors Classification, imbalance
Leave-one-out loo_cv() "LOOCV" Very small data
Bootstrap bootstraps(times = 25) "boot" Variance estimates
Monte Carlo mc_cv(prop = 0.8, times = 25) "LGOCV" Random repeated splits
Grouped group_vfold_cv(group = id) index = custom folds Repeated measures, patients
Time series rolling_origin(), sliding_period() "timeslice" Forecasting
Nested nested_cv() (manual) Unbiased tuning + evaluation

Code Example

library(tidymodels)

set.seed(42)
folds <- vfold_cv(mtcars, v = 5, repeats = 2)
lm_wf <- workflow() |> add_formula(mpg ~ wt + hp) |> add_model(linear_reg())
res <- fit_resamples(lm_wf, resamples = folds, metrics = metric_set(rmse, rsq))
collect_metrics(res)                                   # mean RMSE and R-squared across 10 fits

# Grouped CV: all rows of one group stay in the same fold
df <- transform(mtcars, brand = sub(" .*", "", rownames(mtcars)))
set.seed(42)
gfolds <- group_vfold_cv(df, group = brand, v = 5)

# Rolling origin for time series
ts_df <- data.frame(t = 1:48, y = cumsum(rnorm(48)))
roll <- rolling_origin(ts_df, initial = 24, assess = 6, skip = 5, cumulative = TRUE)
nrow(roll)                                             # number of resamples

# caret equivalent
library(caret)
ctrl <- trainControl(method = "repeatedcv", number = 5, repeats = 2)
set.seed(42)
train(mpg ~ wt + hp, data = mtcars, method = "lm", trControl = ctrl)$results

Choosing a Scheme

  • Independent rows: 5- or 10-fold, repeated for small data.

  • Classification: stratify by the outcome.

  • Several rows per entity: grouped folds.

  • Time-ordered data: rolling origin, never shuffled folds.

  • Reporting after tuning: nested CV or a separate test set.

Common Pitfalls

  • Preprocessing outside the resampling loop (leakage); put it in a recipe or caret’s preProcess.

  • Shuffled folds on time series.

  • Forgetting set.seed(), so folds differ between models.

Interview Tips

  • Match the scheme to the data structure.

  • Mention nested CV for unbiased estimates after tuning.

34

How do you tune hyperparameters in R?

Hyperparameters in R are tuned by evaluating candidate settings with resampling and picking the best: in tidymodels you mark parameters with tune() in a model specification, create cross-validation folds with rsample, and search with tune_grid() (regular or random grids) or tune_bayes() (Bayesian optimization), then finalize the workflow with select_best(); in caret you pass a tuneGrid or tuneLength to train() with a trainControl() resampling scheme. Racing methods (finetune::tune_race_anova()) drop poor candidates early to save time.

Read full answer

Applies toR 4.4+, tidymodels 1.2 (tune, dials, finetune), caret 6.0-94

Search Strategies

Strategy tidymodels caret
Regular grid grid_regular() tuneGrid = expand.grid(...)
Random / space-filling grid_random(), grid_space_filling() search = "random" in trainControl()
Bayesian optimization tune_bayes() —
Racing finetune::tune_race_anova() adaptive_cv
Simulated annealing finetune::tune_sim_anneal() —

Code Example

library(tidymodels)

set.seed(42)
split <- initial_split(iris, prop = 0.8, strata = Species)
train <- training(split)
folds <- vfold_cv(train, v = 5, strata = Species)

rf_spec <- rand_forest(mtry = tune(), min_n = tune(), trees = 500) |>
  set_engine("ranger") |>
  set_mode("classification")
wf <- workflow() |> add_formula(Species ~ .) |> add_model(rf_spec)

grid <- grid_regular(mtry(range = c(1, 4)), min_n(range = c(2, 20)), levels = 4)   # 16 candidates
set.seed(42)
res <- tune_grid(wf, resamples = folds, grid = grid, metrics = metric_set(accuracy, roc_auc))

show_best(res, metric = "roc_auc", n = 3)
autoplot(res)                                    # performance across the grid

best <- select_best(res, metric = "roc_auc")     # or select_by_one_std_err() for a simpler model
final_fit <- finalize_workflow(wf, best) |> last_fit(split)
collect_metrics(final_fit)                       # performance on the held-out test set

# Bayesian optimization continuing from the grid results
set.seed(42)
bayes <- tune_bayes(wf, resamples = folds, initial = res, iter = 10,
                    param_info = extract_parameter_set_dials(wf) |>
                      update(mtry = mtry(c(1, 4))),
                    metrics = metric_set(roc_auc))

caret Version

library(caret)
set.seed(42)
fit <- train(Species ~ ., data = iris, method = "ranger",
             tuneGrid = expand.grid(mtry = 1:4, splitrule = "gini", min.node.size = c(1, 5, 10)),
             trControl = trainControl(method = "cv", number = 5))
fit$bestTune

Common Pitfalls

  • Tuning on the test set; keep it for last_fit() only.

  • Unknown parameter ranges (e.g. mtry depends on the number of predictors); set them with update() or finalize().

  • Huge regular grids; random or Bayesian search is more efficient.

Interview Tips

  • Describe the tune() → folds → tune_grid() → select_best() → last_fit() flow.

  • Mention racing or Bayesian search for efficiency.

35

What is overfitting and how do you prevent it in R models?

Overfitting happens when a model learns the noise and quirks of its training data instead of the general pattern, so it performs very well on training data but poorly on new data; it is detected by a large gap between training and validation or test performance. In R you prevent it by evaluating with cross-validation or a test set (rsample, caret), keeping models simple (fewer features, pruned trees with rpart’s cp), adding regularization (glmnet ridge and lasso, min_n in trees, weight decay or dropout in neural networks), using ensembles such as random forests, stopping boosting early, and getting more data.

Read full answer

Applies toR 4.4+, glmnet 4.1+, rpart 4.1.23, tidymodels 1.2

Remedies

Remedy R tools
Honest evaluation initial_split(), vfold_cv(), caret trainControl()
Simpler model Fewer predictors, lower polynomial degree
Regularization glmnet (alpha = 0 ridge, 1 lasso), penalty in parsnip
Tree pruning rpart cp, prune(), min_n, tree_depth
Early stopping xgboost early_stopping_rounds, keras callbacks
Ensembles ranger, bagging
Feature selection inside CV recipes filters, caret rfe()
More / augmented data Collect more, data augmentation for images

Code Example

set.seed(42)
idx <- sample(nrow(mtcars), 22)
train <- mtcars[idx, ]
test <- mtcars[-idx, ]
rmse <- function(obs, pred) sqrt(mean((obs - pred)^2))

# Polynomial degree: training error keeps falling, test error rises again
for (d in c(1, 3, 6)) {
  fit <- lm(mpg ~ poly(hp, d), data = train)
  cat(sprintf("degree %d: train %.2f  test %.2f\n", d,
              rmse(train$mpg, predict(fit)), rmse(test$mpg, predict(fit, test))))
}

# Lasso: cross-validation chooses how strongly to shrink coefficients
library(glmnet)
x_train <- as.matrix(train[, -1])
x_test <- as.matrix(test[, -1])
set.seed(42)
cv <- cv.glmnet(x_train, train$mpg, alpha = 1, nfolds = 5)
coef(cv, s = "lambda.1se")                       # several coefficients shrunk to exactly 0
rmse(test$mpg, predict(cv, x_test, s = "lambda.1se"))

# Pruned decision tree
library(rpart)
tree <- rpart(mpg ~ ., data = train, control = rpart.control(cp = 0, minsplit = 4))
pruned <- prune(tree, cp = tree$cptable[which.min(tree$cptable[, "xerror"]), "CP"])
c(full = rmse(test$mpg, predict(tree, test)), pruned = rmse(test$mpg, predict(pruned, test)))

Diagnosing

  • Plot learning curves (performance vs training size) and validation curves (performance vs complexity).

  • Large train-test gap = overfitting; both errors high = underfitting.

Common Pitfalls

  • Reporting training accuracy.

  • Data leakage that hides overfitting (preprocessing on all data, duplicate rows).

  • Repeatedly checking the test set while tuning.

Interview Tips

  • Define overfitting with the train-test gap and the bias-variance trade-off.

  • Give one R-specific remedy per model family.

36

How do you create a plot with ggplot2?

A ggplot2 plot is built in layers following the grammar of graphics: start with ggplot(data, aes(x = ..., y = ...)) to map variables to aesthetics, add one or more geometric layers such as geom_point(), geom_line() or geom_bar() with +, and refine the result with scales, facets, labels (labs()) and themes. Each component is independent, so the same data and mappings can be shown in many ways by swapping layers, and finished plots are saved with ggsave().

Read full answer

Applies toR 4.4+, ggplot2 3.5+

Building Blocks

Component Purpose Example
Data A data frame ggplot(mtcars)
Aesthetics Map variables to x, y, colour, size, shape, fill aes(wt, mpg, colour = factor(cyl))
Geoms Visual marks geom_point(), geom_line(), geom_col(), geom_histogram(), geom_boxplot()
Stats Transformations geom_smooth(method = "lm"), stat_summary()
Scales Control mapping and axes scale_colour_brewer(), scale_y_log10()
Facets Small multiples facet_wrap(~ gear)
Coordinates Axes system coord_flip(), coord_cartesian(ylim = ...)
Labels Titles and axis names labs(title = , x = , y = )
Themes Non-data appearance theme_minimal(), theme(legend.position = "bottom")

Code Example

library(ggplot2)

p <- ggplot(mtcars, aes(x = wt, y = mpg, colour = factor(cyl))) +
  geom_point(size = 3, alpha = 0.8) +
  geom_smooth(aes(group = 1), method = "lm", se = TRUE, colour = "grey30") +   # one overall trend
  scale_colour_brewer(palette = "Dark2") +
  labs(title = "Heavier cars use more fuel",
       x = "Weight (1000 lbs)", y = "Miles per gallon", colour = "Cylinders") +
  theme_minimal(base_size = 12) +
  theme(legend.position = "bottom")
p

ggsave("mpg_vs_weight.png", p, width = 7, height = 5, dpi = 300)

# Other common plots
ggplot(iris, aes(Species, Sepal.Length, fill = Species)) + geom_boxplot(show.legend = FALSE)
ggplot(mtcars, aes(mpg)) + geom_histogram(binwidth = 2, fill = "steelblue", colour = "white")
ggplot(mtcars, aes(factor(cyl))) + geom_bar() + labs(x = "Cylinders", y = "Count")

Aesthetics: Mapping vs Setting

  • Mapped inside aes(): varies with data (aes(colour = cyl)) and gets a legend.

  • Set outside aes(): fixed value (geom_point(colour = "red")).

Common Pitfalls

  • Putting a fixed colour inside aes() (aes(colour = "red")), which creates a legend named “red”.

  • Using %>%/|> between layers instead of +.

  • Numeric variables mapped to colour when they should be discrete; wrap in factor().

  • geom_bar() counts rows; use geom_col() for pre-computed values.

Interview Tips

  • Explain the layers: data, aesthetics, geoms, then scales/facets/themes.

  • Mention mapping vs setting aesthetics.

37

What is the purpose of facets in ggplot2?

Facets in ggplot2 split one plot into a grid of small panels, one for each value (or combination of values) of one or two categorical variables, all with the same axes and geoms, so you can compare patterns across groups side by side instead of overloading one panel with colours. facet_wrap(~ var) wraps panels for one variable into rows and columns, and facet_grid(rows = vars(a), cols = vars(b)) makes a matrix for two variables; the scales argument lets axes vary by panel when ranges differ a lot.

Read full answer

Applies toR 4.4+, ggplot2 3.5+

facet_wrap vs facet_grid

Function Layout Best for
facet_wrap(~ class) Wraps panels into a 2-D layout One variable with many levels
facet_grid(rows = vars(a), cols = vars(b)) Rows × columns matrix Two variables, or aligned comparisons
facet_grid(a ~ b) Formula form of the same

Key Arguments

Argument Effect
scales = "free", "free_x", "free_y" Axes vary per panel (comparisons get harder)
ncol, nrow Layout of facet_wrap()
labeller = label_both Panel labels like “cyl: 4”
space = "free" Panel size proportional to data range (facet_grid)
margins = TRUE Add an “all” panel (facet_grid)

Code Example

library(ggplot2)

# One variable: a panel per number of cylinders
ggplot(mtcars, aes(wt, mpg)) +
  geom_point() +
  geom_smooth(method = "lm", se = FALSE) +
  facet_wrap(~ cyl, labeller = label_both)

# Two variables: transmission (rows) by gears (columns)
ggplot(mtcars, aes(hp, mpg)) +
  geom_point() +
  facet_grid(rows = vars(am), cols = vars(gear), labeller = label_both)

# Free y scales when ranges differ by group
ggplot(iris, aes(Sepal.Length)) +
  geom_histogram(bins = 15) +
  facet_wrap(~ Species, ncol = 1, scales = "free_y")

# Show the whole data in grey behind each panel for context
ggplot(mtcars, aes(wt, mpg)) +
  geom_point(data = transform(mtcars, cyl = NULL), colour = "grey80") +
  geom_point(colour = "steelblue") +
  facet_wrap(~ cyl)

When to Use Facets

  • Comparing a relationship across groups (regions, segments, experiments).

  • Too many groups for one panel’s colour legend.

  • Showing distributions per category.

Common Pitfalls

  • Too many panels to read; filter or group levels first.

  • Free scales that make panels look similar when the values differ a lot; label clearly.

  • Faceting by a continuous variable; bin it first with cut() or cut_number().

Interview Tips

  • Contrast facet_wrap() and facet_grid().

  • Mention scales = "free" and its trade-off.

38

How do you create interactive visualizations in R?

Interactive visualizations in R are created with htmlwidgets-based packages that render in the browser, RStudio viewer, R Markdown or Quarto: plotly turns ggplot2 plots into interactive charts with ggplotly() or builds them directly with plot_ly(), leaflet makes interactive maps, DT makes searchable tables, dygraphs handles time series, visNetwork draws networks, and ggiraph adds tooltips to ggplot2; for full applications with inputs and reactive updates, shiny (and bslib for layout) builds web apps. Static ggplot2 is best for reports, interactivity for exploration and dashboards.

Read full answer

Applies toR 4.4+, plotly 4.10, leaflet 2.2, DT 0.33, shiny 1.9

Packages

Package Use
plotly Interactive charts; ggplotly() converts ggplot2
ggiraph Hover and click interactivity on ggplot2
leaflet Interactive maps
DT Interactive tables (sorting, search, pagination)
dygraphs Zoomable time series
visNetwork Network graphs
echarts4r, highcharter Rich JavaScript chart libraries
shiny + bslib Full interactive apps and dashboards
crosstalk Linked brushing between widgets without a server

Code Example

library(ggplot2)
library(plotly)

p <- ggplot(mtcars, aes(wt, mpg, colour = factor(cyl),
                        text = paste("Car:", rownames(mtcars)))) +
  geom_point(size = 3) +
  labs(colour = "Cylinders")
ggplotly(p, tooltip = c("text", "x", "y"))      # hover shows car names

plot_ly(iris, x = ~Sepal.Length, y = ~Petal.Length, color = ~Species,
        type = "scatter", mode = "markers")

library(DT)
datatable(mtcars, options = list(pageLength = 5), filter = "top")

library(leaflet)
leaflet() |>
  addTiles() |>
  addMarkers(lng = -9.14, lat = 38.72, popup = "Lisbon")

# A minimal Shiny app
library(shiny)
ui <- fluidPage(
  selectInput("var", "X variable", choices = c("wt", "hp", "disp")),
  plotOutput("plot")
)
server <- function(input, output) {
  output$plot <- renderPlot(ggplot(mtcars, aes(.data[[input$var]], mpg)) + geom_point())
}
# shinyApp(ui, server)   # run interactively

Choosing

  • Exploration and HTML reports: plotly, DT, leaflet in Quarto or R Markdown.

  • Dashboards with user inputs: Shiny (or Quarto dashboards with shiny runtime).

  • Static publications: plain ggplot2.

Common Pitfalls

  • Widgets don’t render in PDF or Word output; use static plots there.

  • Very large data in widgets makes pages slow; aggregate or sample.

  • ggplotly() doesn’t support every ggplot2 feature perfectly; check the result.

Interview Tips

  • Name plotly, leaflet, DT and Shiny with one use each.

  • Explain htmlwidgets vs Shiny (no server vs reactive server).

39

How do you use and customize themes in ggplot2?

Themes in ggplot2 control everything in a plot that isn’t data, such as fonts, background, grid lines, axis text, legend position and panel borders; you apply a complete built-in theme like theme_minimal(), theme_bw() or theme_classic() and then adjust individual elements with theme(), using element functions element_text(), element_line(), element_rect() and element_blank(). A custom theme can be saved as a function and reused, or set for a whole session with theme_set(), which keeps a team’s charts consistent.

Read full answer

Applies toR 4.4+, ggplot2 3.5+

Built-in Themes

Theme Look
theme_grey() Default: grey panel, white grid
theme_bw() White panel, grey grid, border
theme_minimal() No background or border, light grid
theme_classic() Axis lines, no grid (journal style)
theme_light(), theme_dark() Light or dark panels
theme_void() Nothing but data (maps, diagrams)

Extension packages: ggthemes (styles of well-known publications), hrbrthemes, bbplot.

Common theme() Elements

Element Example
Title plot.title = element_text(face = "bold", size = 16)
Axis text axis.text.x = element_text(angle = 45, hjust = 1)
Grid panel.grid.minor = element_blank()
Background panel.background = element_rect(fill = "white")
Legend legend.position = "bottom", legend.title = element_blank()
Facet labels strip.background = element_rect(fill = "grey90")
Margins plot.margin = margin(10, 10, 10, 10)

Code Example

library(ggplot2)

base <- ggplot(mtcars, aes(wt, mpg, colour = factor(cyl))) +
  geom_point(size = 3) +
  labs(title = "Fuel economy by weight", x = "Weight (1000 lbs)", y = "MPG", colour = "Cylinders")

base + theme_minimal()

base + theme_bw(base_size = 13) +
  theme(plot.title = element_text(face = "bold", hjust = 0),
        panel.grid.minor = element_blank(),
        legend.position = "bottom",
        axis.title = element_text(colour = "grey30"))

# A reusable house theme
theme_company <- function(base_size = 12) {
  theme_minimal(base_size = base_size) %+replace%
    theme(plot.title = element_text(face = "bold", size = rel(1.3), hjust = 0,
                                    margin = margin(b = 8)),
          panel.grid.minor = element_blank(),
          panel.grid.major.x = element_blank(),
          legend.position = "top",
          legend.justification = "left")
}
base + theme_company()

theme_set(theme_company())   # default for all later plots in the session

Common Pitfalls

  • Calling a complete theme after theme(), which overwrites your tweaks; add complete themes first.

  • Using theme() to change data colours; use scale_* functions instead.

  • Tiny fonts in saved images; set base_size and check with ggsave() dimensions.

Interview Tips

  • Explain complete themes vs theme() tweaks and element functions.

  • Mention theme_set() and a reusable theme function for consistency.

40

What is the best way to visualize high-dimensional data in R?

There is no single best method, but the usual approach is to reduce the data to two or three dimensions and plot it: principal component analysis (prcomp()) for linear structure and an interpretable view of variables (biplots), and non-linear embeddings such as t-SNE (Rtsne) or UMAP (uwot) for clusters and local structure; alongside these, pairwise scatterplot matrices (GGally::ggpairs()), correlation heatmaps (corrplot), heatmaps with clustering (pheatmap), and parallel coordinate plots show the original variables directly. Scale the data first, and remember that t-SNE and UMAP distances between clusters are not meaningful.

Read full answer

Applies toR 4.4+, ggplot2 3.5+, GGally 2.2, Rtsne 0.17, uwot 0.2, corrplot 0.94

Methods

Method Shows R
PCA scatter / biplot Main linear variation, variable loadings prcomp(), factoextra::fviz_pca_biplot()
t-SNE Local neighbourhoods, clusters Rtsne::Rtsne()
UMAP Clusters with more global structure, faster uwot::umap()
Scatterplot matrix All pairwise relationships (up to ~10 variables) GGally::ggpairs()
Correlation heatmap Which variables move together corrplot::corrplot()
Clustered heatmap Rows and columns grouped pheatmap::pheatmap(), heatmap()
Parallel coordinates Profiles across many variables GGally::ggparcoord()
MDS Distances preserved in 2-D cmdscale()

Code Example

library(ggplot2)

X <- scale(iris[, 1:4])

# PCA
pca <- prcomp(X)
summary(pca)$importance[2, 1:2]          # PC1 explains about 73%, PC2 about 23% of variance
pcs <- data.frame(pca$x[, 1:2], Species = iris$Species)
ggplot(pcs, aes(PC1, PC2, colour = Species)) + geom_point() + coord_equal()

# UMAP
library(uwot)
set.seed(42)
emb <- umap(X, n_neighbors = 15, min_dist = 0.1)
ggplot(data.frame(emb, Species = iris$Species), aes(X1, X2, colour = Species)) + geom_point()

# t-SNE (duplicates must be removed first)
library(Rtsne)
keep <- !duplicated(X)
set.seed(42)
ts <- Rtsne(X[keep, ], perplexity = 30)
ggplot(data.frame(ts$Y, Species = iris$Species[keep]), aes(X1, X2, colour = Species)) + geom_point()

# Pairwise view and correlations
GGally::ggpairs(iris, columns = 1:4, aes(colour = Species))
corrplot::corrplot(cor(mtcars), method = "color", order = "hclust")

Choosing

  • Interpretability: PCA (axes are combinations of known variables).

  • Clusters in complex data (single-cell, embeddings): UMAP or t-SNE.

  • Few variables: scatterplot matrix.

  • Variable relationships: correlation heatmap.

Common Pitfalls

  • Not scaling variables before PCA, t-SNE or UMAP.

  • Reading cluster sizes or distances between clusters in t-SNE/UMAP as meaningful.

  • Drawing conclusions from one t-SNE run; try several perplexities and seeds.

Interview Tips

  • Mention PCA first for interpretability, then UMAP/t-SNE for structure.

  • Warn about the limits of non-linear embeddings.

41

How do you load a CSV file in R and summarize its statistics?

To load a CSV file in R and summarize it, read it with readr::read_csv() (or base read.csv()), check its structure with glimpse() or str(), get quick per-column statistics with summary(), and build richer summaries with dplyr, such as counts, means, medians, standard deviations and missing values per column or per group; packages like skimr produce a full overview in one call. Always confirm that column types were read correctly, because numbers stored as text or dates read as strings change every summary.

Read full answer

Applies toR 4.4+, readr 2.1+, dplyr 1.1+, tidyr 1.3+, skimr 2.1

Code Example

library(readr)
library(dplyr)
library(tidyr)

# Create a CSV to read (replace with your own path)
path <- tempfile(fileext = ".csv")
write_csv(airquality, path)

df <- read_csv(path, show_col_types = FALSE)
glimpse(df)                         # 153 rows, 6 columns, types per column
summary(df)                         # min, quartiles, mean, max and NA count per column

# Missing values per column
df |> summarise(across(everything(), ~ sum(is.na(.x))))

# Numeric summaries in a tidy long table
df |>
  select(where(is.numeric)) |>
  pivot_longer(everything(), names_to = "variable") |>
  group_by(variable) |>
  summarise(n = sum(!is.na(value)),
            mean = mean(value, na.rm = TRUE),
            sd = sd(value, na.rm = TRUE),
            median = median(value, na.rm = TRUE),
            min = min(value, na.rm = TRUE),
            max = max(value, na.rm = TRUE),
            missing = sum(is.na(value)))

# Grouped summary
df |>
  group_by(Month) |>
  summarise(mean_temp = mean(Temp), mean_ozone = mean(Ozone, na.rm = TRUE), days = n())

# One-call overview
# skimr::skim(df)

What to Check

Check Function
Dimensions and types glimpse(), str(), dim()
Basic statistics summary()
Missing values colSums(is.na(df))
Categorical counts count(df, col, sort = TRUE)
Duplicates sum(duplicated(df))
Distributions hist(), ggplot2::geom_histogram()

Reading Options

  • col_types = cols(...) to set types explicitly.

  • na = c("", "NA", "-") for custom missing markers.

  • locale = locale(decimal_mark = ",") or read_csv2() for European formats.

  • data.table::fread() for very large files.

Common Pitfalls

  • IDs with leading zeros read as numbers; read them as character.

  • summary() of character columns only shows length and class; convert categories to factors or use count().

  • Forgetting na.rm = TRUE, giving NA results.

Interview Tips

  • Show read → inspect types → summary → missing values → grouped summaries.

  • Mention skimr::skim() as a quick overview tool.

42

How do you filter and sort a data frame with dplyr on several conditions?

With dplyr you filter rows on several conditions with filter(), where comma-separated conditions are combined with AND and | expresses OR, using helpers such as %in%, between() and !is.na(); you then sort with arrange(), listing columns in priority order and wrapping a column in desc() for descending order. Chained with the pipe, the steps read top to bottom like the question you are asking of the data.

Read full answer

Applies toR 4.4+, dplyr 1.1+

Condition Building Blocks

Need Code
AND filter(df, a > 1, b == "x") or a > 1 & b == "x"
OR filter(df, a > 1 | b == "x")
In a set cyl %in% c(4, 6)
Range (inclusive) between(mpg, 20, 30)
Not missing !is.na(Ozone)
Text match grepl("Merc", name) or stringr::str_detect(name, "^Merc")
Per-group condition group_by(g) |> filter(x == max(x))
Across columns filter(if_all(c(a, b), ~ .x > 0)), if_any()

Code Example

library(dplyr)
library(tibble)

cars <- mtcars |> rownames_to_column("model")

# Efficient, powerful manual cars, sorted by mpg (desc) then hp (asc)
cars |>
  filter(am == 1,                       # manual
         mpg > 20,
         hp >= 90 | cyl == 6) |>        # either condition
  arrange(desc(mpg), hp) |>
  select(model, mpg, hp, cyl)
# Lotus Europa 30.4, Porsche 914-2 26.0, Datsun 710 22.8, Volvo 142E 21.4, Mazda RX4 21.0, Mazda RX4 Wag 21.0

# Set membership, ranges and missing values
airquality |>
  filter(Month %in% c(7, 8), between(Temp, 80, 90), !is.na(Ozone)) |>
  arrange(Month, desc(Ozone)) |>
  head()

# Best car per cylinder group, sorted
cars |>
  group_by(cyl) |>
  filter(mpg == max(mpg)) |>
  ungroup() |>
  arrange(cyl)

# Conditions on several columns at once
iris |> filter(if_all(starts_with("Sepal"), ~ .x > 3)) |> nrow()   # rows where both sepal measures > 3

Common Pitfalls

  • filter(x == NA) returns nothing; use is.na(x).

  • == with floating point values; use between() or near().

  • Forgetting ungroup() after grouped filtering.

  • Using && or || (scalar operators) instead of & and |.

Interview Tips

  • Explain that commas mean AND and that filter() drops NA rows.

  • Show desc() and multiple sort keys.

43

How do you write an R function for Euclidean distance between two points?

The Euclidean distance between two points is the square root of the sum of squared differences of their coordinates, sqrt(Σ (a_i − b_i)²), and in R it is a one-line vectorized function, sqrt(sum((a - b)^2)), that works for points in any number of dimensions; good practice adds input checks for numeric vectors of equal length. For many points at once, the built-in dist() returns all pairwise distances, so you rarely need a loop.

Read full answer

Applies toR 4.4+ (base and stats)

Code Example

euclidean_distance <- function(a, b) {
  # Check the inputs: numeric vectors of the same length without missing values
  stopifnot(is.numeric(a), is.numeric(b))
  if (length(a) != length(b)) stop("Points must have the same number of dimensions.")
  if (anyNA(a) || anyNA(b)) stop("Points must not contain missing values.")
  sqrt(sum((a - b)^2))
}

euclidean_distance(c(0, 0), c(3, 4))          # 5
euclidean_distance(c(1, 2, 3), c(4, 6, 8))    # sqrt(9 + 16 + 25) = 7.071068
euclidean_distance(5, 2)                      # 3 (one dimension)

# Distance from one point to every row of a matrix (vectorized)
distances_to <- function(point, m) sqrt(rowSums(sweep(m, 2, point)^2))
m <- matrix(c(0, 0, 3, 4, 6, 8), ncol = 2, byrow = TRUE)
distances_to(c(0, 0), m)                       # 0 5 10

# All pairwise distances with base R
dist(m)                                        # 5, 10 and 5 between the three points
as.matrix(dist(m))[1, 3]                       # 10

# Scaling matters when features use different units
scaled <- scale(mtcars[, c("mpg", "hp")])
euclidean_distance(scaled["Mazda RX4", ], scaled["Datsun 710", ])

Related Distances

Distance Formula R
Euclidean sqrt(Σ (a − b)²) dist(x, method = "euclidean")
Manhattan Σ |a − b| dist(x, method = "manhattan")
Minkowski (Σ |a − b|^p)^(1/p) dist(x, method = "minkowski", p = 3)
Cosine 1 − a·b / (‖a‖‖b‖) 1 - sum(a * b) / sqrt(sum(a^2) * sum(b^2))

Common Pitfalls

  • Unequal lengths: R recycles the shorter vector silently, giving wrong answers; check lengths.

  • Comparing features with different scales without standardizing.

  • Using loops over points instead of vectorized code or dist().

Interview Tips

  • Write the vectorized one-liner and add validation.

  • Mention dist() and scaling for real use (k-NN, clustering).

44

How do you write an R script that performs k-means clustering?

A k-means clustering script in R loads and selects numeric features, scales them, chooses the number of clusters with the elbow method or average silhouette width, runs kmeans() with a fixed seed and many random starts (nstart), attaches the cluster labels to the data, profiles each cluster with grouped summaries, and visualizes the result, typically on the first two principal components. Scaling and multiple starts are the two settings that most affect result quality.

Read full answer

Applies toR 4.4+, dplyr 1.1+, ggplot2 3.5+, cluster 2.1+

Code Example

library(dplyr)
library(ggplot2)
library(cluster)

# 1. Data: numeric features only
data <- USArrests                                  # crime rates per US state
X <- scale(data)                                   # 2. standardize

# 3. Choose k: elbow and silhouette
set.seed(42)
ks <- 2:8
wss <- sapply(ks, function(k) kmeans(X, centers = k, nstart = 25)$tot.withinss)
sil <- sapply(ks, function(k) {
  km <- kmeans(X, centers = k, nstart = 25)
  mean(silhouette(km$cluster, dist(X))[, "sil_width"])
})
print(data.frame(k = ks, wss = round(wss, 1), silhouette = round(sil, 3)))
best_k <- ks[which.max(sil)]

# 4. Final model
set.seed(42)
km <- kmeans(X, centers = best_k, nstart = 50, iter.max = 100)
data$cluster <- factor(km$cluster)

# 5. Profile clusters on the original scale
data |>
  group_by(cluster) |>
  summarise(across(Murder:Rape, mean), states = n())

# 6. Visualize on the first two principal components
pca <- prcomp(X)
plot_df <- data.frame(pca$x[, 1:2], cluster = data$cluster, state = rownames(data))
ggplot(plot_df, aes(PC1, PC2, colour = cluster, label = state)) +
  geom_text(size = 3) +
  labs(title = paste("k-means with k =", best_k), colour = "Cluster") +
  theme_minimal()

Script Checklist

Step Why
Numeric features only k-means uses Euclidean distance
scale() Prevents large-range variables dominating
set.seed() + nstart Reproducible and avoids poor local minima
Elbow / silhouette Choose k with evidence
Profile on original scale Clusters people can interpret
PCA plot 2-D view of the clusters

Common Pitfalls

  • Choosing k only by eye from the elbow; combine with silhouette and domain sense.

  • Categorical variables coded as numbers; use other methods (k-prototypes, PAM with Gower distance).

  • Interpreting cluster numbers as ordered; they are arbitrary labels.

Interview Tips

  • Walk through scale → choose k → fit → profile → visualize.

  • Mention nstart and the arbitrary cluster labels.

45

How do you build a linear regression model in R and plot its summary?

Build the model with lm(), inspect it with summary() for coefficients, p-values and R², and plot it in three useful ways: the fitted relationship (ggplot2 with geom_smooth(method = "lm") or predicted lines), a coefficient plot with confidence intervals from broom::tidy(conf.int = TRUE), and the diagnostic plots from plot(model) to check residual assumptions. Together they show what the model says, how certain it is, and whether its assumptions hold.

Read full answer

Applies toR 4.4+, ggplot2 3.5+, broom 1.0+

Code Example

library(ggplot2)
library(broom)

model <- lm(mpg ~ wt + hp + factor(am), data = mtcars)
summary(model)                       # coefficients, standard errors, p-values, R-squared

# 1. Coefficient plot with 95% confidence intervals
coefs <- tidy(model, conf.int = TRUE)
coefs <- coefs[coefs$term != "(Intercept)", ]
ggplot(coefs, aes(estimate, reorder(term, estimate))) +
  geom_vline(xintercept = 0, linetype = "dashed", colour = "grey50") +
  geom_errorbarh(aes(xmin = conf.low, xmax = conf.high), height = 0.2) +
  geom_point(size = 3) +
  labs(x = "Effect on mpg (95% CI)", y = NULL, title = "Linear model coefficients") +
  theme_minimal()

# 2. Fitted vs actual
aug <- augment(model)                # adds .fitted, .resid, .hat, .cooksd
ggplot(aug, aes(.fitted, mpg)) +
  geom_point() +
  geom_abline(slope = 1, intercept = 0, linetype = "dashed") +
  labs(x = "Fitted mpg", y = "Actual mpg")

# 3. Residual diagnostics
par(mfrow = c(2, 2))
plot(model)                          # residuals vs fitted, Q-Q, scale-location, leverage
par(mfrow = c(1, 1))

# 4. Model-level statistics
glance(model)[, c("r.squared", "adj.r.squared", "sigma", "AIC")]

# 5. Relationship for one predictor with the others held at typical values
grid <- data.frame(wt = seq(1.5, 5.5, by = 0.1), hp = median(mtcars$hp), am = 0)
grid$pred <- predict(model, grid)
ggplot(mtcars, aes(wt, mpg)) + geom_point() + geom_line(data = grid, aes(wt, pred), colour = "blue")

What the Plots Tell You

Plot Question
Coefficient plot Which predictors matter and in which direction, with uncertainty
Fitted vs actual How well the model predicts overall
Residuals vs fitted Non-linearity, unequal variance
Normal Q-Q Normality of residuals
Residuals vs leverage Influential observations (Cook’s distance)

Common Pitfalls

  • Comparing coefficients measured in different units as “importance”; standardize predictors first if needed.

  • Ignoring patterns in residual plots.

  • Interpreting the effect of one predictor without holding others constant.

Interview Tips

  • Show both coefficient and diagnostic plots.

  • Mention broom’s tidy(), glance() and augment().

46

How do you visualize a decision tree model in R?

To visualize a decision tree in R, fit it with rpart() and draw it with rpart.plot::rpart.plot(), which shows split rules, predicted classes or values, probabilities and the share of observations in each node; alternatives are partykit (plot(as.party(tree))), which adds distributions in the leaves, and base plot() with text(). Prune the tree first using the complexity parameter so the diagram stays readable, and print the rules with rpart.rules() for a text version.

Read full answer

Applies toR 4.4+, rpart 4.1.23, rpart.plot 3.1, partykit 1.2

Code Example

library(rpart)
library(rpart.plot)

set.seed(42)
tree <- rpart(Species ~ ., data = iris, method = "class",
              control = rpart.control(cp = 0.01, minsplit = 10))

# Prune to the cp with the lowest cross-validated error
best_cp <- tree$cptable[which.min(tree$cptable[, "xerror"]), "CP"]
pruned <- prune(tree, cp = best_cp)

# Main diagram: class, class probabilities and % of observations per node
rpart.plot(pruned, type = 4, extra = 104, box.palette = "auto",
           main = "Decision tree for iris species")

rpart.rules(pruned, cover = TRUE)          # the same tree as readable rules
plotcp(tree)                               # cross-validated error vs cp (pruning choice)

# partykit: leaves show class distributions as bar charts
library(partykit)
plot(as.party(pruned))

# Regression tree for mpg
reg_tree <- rpart(mpg ~ wt + hp + cyl, data = mtcars, control = rpart.control(minsplit = 6))
rpart.plot(reg_tree, type = 2, extra = 101, digits = 3)   # mean mpg and n per node

# Save to a file
png("iris_tree.png", width = 1600, height = 1000, res = 200)
rpart.plot(pruned, type = 4, extra = 104)
dev.off()

Useful rpart.plot Options

Argument Effect
type Layout of labels (0-5); 4 labels both branches
extra Node info: 104 = class probabilities + percentage (classification), 101 = n + percentage (regression)
box.palette Node colours by class or value
fallen.leaves Put leaves at the bottom (default TRUE)
tweak Scale the text size
digits Rounding of node values

Common Pitfalls

  • Plotting an unpruned tree that is too big to read.

  • extra values meant for classification used on regression trees.

  • Presenting one tree as definitive; trees change with small data changes.

Interview Tips

  • Mention rpart.plot and rpart.rules() for diagrams and text.

  • Explain pruning with plotcp() before plotting.

47

How do you write R code for grid search hyperparameter tuning?

Grid search in R evaluates every combination of a set of hyperparameter values with cross-validation and keeps the best one: in tidymodels you mark parameters with tune(), build a grid with grid_regular() or expand_grid(), run tune_grid() over vfold_cv() folds, and finalize with select_best() and last_fit(); in caret you pass tuneGrid = expand.grid(...) to train(); and you can also write the loop yourself to see exactly what happens.

Read full answer

Applies toR 4.4+, tidymodels 1.2 (tune, dials, rsample), ranger 0.16

Code Example

library(tidymodels)

set.seed(42)
split <- initial_split(mtcars, prop = 0.8)
train <- training(split)
folds <- vfold_cv(train, v = 5)

spec <- rand_forest(mtry = tune(), min_n = tune(), trees = 500) |>
  set_engine("ranger") |>
  set_mode("regression")
wf <- workflow() |> add_formula(mpg ~ .) |> add_model(spec)

grid <- expand_grid(mtry = c(2, 4, 6, 8), min_n = c(2, 5, 10))   # 12 combinations
set.seed(42)
res <- tune_grid(wf, resamples = folds, grid = grid, metrics = metric_set(rmse, rsq))

collect_metrics(res) |> filter(.metric == "rmse") |> arrange(mean) |> head(3)
best <- select_best(res, metric = "rmse")
final <- finalize_workflow(wf, best) |> last_fit(split)
collect_metrics(final)                                    # test-set RMSE and R-squared

The Same Idea by Hand

library(ranger)

set.seed(42)
k <- 5
fold_id <- sample(rep(1:k, length.out = nrow(mtcars)))
grid <- expand.grid(mtry = c(2, 4, 6), min.node.size = c(2, 5))

cv_rmse <- function(mtry, node) {
  errs <- sapply(1:k, function(f) {
    train <- mtcars[fold_id != f, ]
    valid <- mtcars[fold_id == f, ]
    fit <- ranger(mpg ~ ., data = train, mtry = mtry, min.node.size = node,
                  num.trees = 300, seed = 42)
    sqrt(mean((valid$mpg - predict(fit, valid)$predictions)^2))
  })
  mean(errs)
}

grid$rmse <- mapply(cv_rmse, grid$mtry, grid$min.node.size)
grid[order(grid$rmse), ]                                   # best combination first

Grid Design Tips

Tip Why
Use log-spaced values for rates and penalties e.g. 10^seq(-4, 0, 1)
Start coarse, then refine Saves time
Prefer random or Bayesian search for many parameters Grids grow exponentially
Same folds for every candidate Fair comparison
Consider the one-standard-error rule select_by_one_std_err() picks a simpler model

Common Pitfalls

  • Tuning on the test set.

  • Preprocessing outside the resampling loop.

  • Grids with values outside sensible ranges (e.g. mtry above the number of predictors).

Interview Tips

  • Explain the full loop: grid × folds → mean score → best → final test.

  • Mention random and Bayesian alternatives when the grid is large.

48

How do you write an R function for accuracy, precision, recall and F1?

A classification metrics function in R builds the confusion counts from the true and predicted labels for a chosen positive class, true positives, false positives, false negatives and true negatives, and computes accuracy = (TP + TN) / total, precision = TP / (TP + FP), recall = TP / (TP + FN), and F1 = 2 · precision · recall / (precision + recall), returning NA where a denominator is zero. For multi-class problems, compute the per-class values and average them (macro or weighted); yardstick and caret provide tested versions.

Read full answer

Applies toR 4.4+, yardstick 1.3+

Code Example

classification_metrics <- function(actual, predicted, positive) {
  stopifnot(length(actual) == length(predicted))
  actual <- as.character(actual)
  predicted <- as.character(predicted)

  tp <- sum(predicted == positive & actual == positive)
  fp <- sum(predicted == positive & actual != positive)
  fn <- sum(predicted != positive & actual == positive)
  tn <- sum(predicted != positive & actual != positive)

  safe_div <- function(a, b) if (b == 0) NA_real_ else a / b
  precision <- safe_div(tp, tp + fp)
  recall <- safe_div(tp, tp + fn)
  f1 <- if (is.na(precision) || is.na(recall) || precision + recall == 0) NA_real_
        else 2 * precision * recall / (precision + recall)

  c(accuracy = (tp + tn) / length(actual), precision = precision, recall = recall, f1 = f1)
}

actual    <- c("spam", "spam", "ham", "ham", "spam", "ham", "ham", "spam")
predicted <- c("spam", "ham",  "ham", "spam", "spam", "ham", "ham", "spam")
round(classification_metrics(actual, predicted, positive = "spam"), 3)
# TP 3, FP 1, FN 1, TN 3 -> accuracy 0.75, precision 0.75, recall 0.75, f1 0.75

# Macro-averaged metrics for several classes
macro_metrics <- function(actual, predicted) {
  per_class <- sapply(unique(actual), function(cl) classification_metrics(actual, predicted, cl))
  rowMeans(per_class, na.rm = TRUE)
}
set.seed(42)
truth <- iris$Species
guess <- truth
guess[sample(150, 15)] <- sample(levels(truth), 15, replace = TRUE)   # add some errors
round(macro_metrics(truth, guess), 3)

# Check against yardstick
library(yardstick)
df <- data.frame(truth = factor(actual, levels = c("spam", "ham")),
                 estimate = factor(predicted, levels = c("spam", "ham")))
metric_set(accuracy, precision, recall, f_meas)(df, truth = truth, estimate = estimate)

Common Pitfalls

  • Choosing the wrong positive class; always pass it explicitly.

  • Division by zero when the model never predicts the positive class.

  • Averaging per-class F1 (macro) vs computing F1 from pooled counts (micro) without saying which.

  • Accuracy alone on imbalanced data.

Interview Tips

  • Derive each metric from the confusion counts.

  • Mention macro vs micro averaging for multi-class problems.

49

How do you add a regression line with confidence band in ggplot2?

In ggplot2, a scatter plot with a regression line and confidence interval is ggplot(df, aes(x, y)) + geom_point() + geom_smooth(method = "lm", se = TRUE): geom_smooth() fits a linear model to the plotted data and draws the fitted line with a shaded 95% confidence band for the mean response, and level changes the confidence level. Map a grouping variable to colour to get one line per group, use formula = y ~ poly(x, 2) for curved fits, and add a prediction band yourself from predict(..., interval = "prediction") when you want the range for individual points.

Read full answer

Applies toR 4.4+, ggplot2 3.5+

Code Example

library(ggplot2)

# Basic scatter + linear fit + 95% confidence band
ggplot(mtcars, aes(wt, mpg)) +
  geom_point(size = 2.5, alpha = 0.8) +
  geom_smooth(method = "lm", formula = y ~ x, se = TRUE, level = 0.95,
              colour = "steelblue", fill = "steelblue", alpha = 0.2) +
  labs(x = "Weight (1000 lbs)", y = "Miles per gallon",
       title = "Fuel economy falls with weight",
       subtitle = "Linear fit with 95% confidence interval") +
  theme_minimal()

# One line per group
ggplot(mtcars, aes(hp, mpg, colour = factor(am, labels = c("automatic", "manual")))) +
  geom_point() +
  geom_smooth(method = "lm", formula = y ~ x) +
  labs(colour = "Transmission")

# Quadratic fit
ggplot(mtcars, aes(hp, mpg)) +
  geom_point() +
  geom_smooth(method = "lm", formula = y ~ poly(x, 2))

# Add a 95% prediction band (where individual cars are expected to fall)
fit <- lm(mpg ~ wt, data = mtcars)
grid <- data.frame(wt = seq(min(mtcars$wt), max(mtcars$wt), length.out = 100))
band <- cbind(grid, predict(fit, grid, interval = "prediction"))
ggplot(mtcars, aes(wt, mpg)) +
  geom_ribbon(data = band, aes(x = wt, ymin = lwr, ymax = upr), inherit.aes = FALSE,
              fill = "grey85") +
  geom_smooth(method = "lm", formula = y ~ x, colour = "black") +
  geom_point()

Confidence vs Prediction Band

Band Meaning Width
Confidence (geom_smooth se = TRUE) Where the mean mpg at each weight lies Narrower
Prediction Where an individual new car’s mpg lies Wider

Common Pitfalls

  • Not setting method = "lm": the default is LOESS for small data (and a GAM for larger), which isn’t a straight line.

  • Expecting the plotted line to match a multi-variable model; geom_smooth() uses only x and y.

  • Extrapolating the band beyond the data range.

Interview Tips

  • Explain what the shaded band represents.

  • Mention per-group fits and the difference between confidence and prediction intervals.

50

How do you split data and train a random forest in R?

A complete random forest script in R sets a seed, splits the data into training and test sets with stratification (rsample::initial_split() or caret::createDataPartition()), trains the forest on the training set only with ranger or randomForest (or tidymodels’ rand_forest()), checks the out-of-bag error, predicts on the test set, and reports metrics such as accuracy and a confusion matrix plus variable importance. The test set is used once, at the end, to estimate performance on new data.

Read full answer

Applies toR 4.4+, rsample 1.2+, ranger 0.16, yardstick 1.3+

Code Example

library(rsample)
library(ranger)
library(yardstick)
library(dplyr)

set.seed(42)
split <- initial_split(iris, prop = 0.75, strata = Species)   # stratified 75/25 split
train <- training(split)
test <- testing(split)

rf <- ranger(Species ~ ., data = train,
             num.trees = 500,
             mtry = 2,                       # predictors tried at each split
             min.node.size = 1,
             importance = "permutation",
             probability = FALSE,
             seed = 42)
rf$prediction.error                          # out-of-bag error rate

pred <- predict(rf, data = test)$predictions
results <- tibble(truth = test$Species, estimate = pred)
accuracy(results, truth, estimate)
conf_mat(results, truth, estimate)

sort(rf$variable.importance, decreasing = TRUE)   # permutation importance

# The same with randomForest
library(randomForest)
set.seed(42)
rf2 <- randomForest(Species ~ ., data = train, ntree = 500, importance = TRUE)
print(rf2)                                   # OOB error and confusion matrix
mean(predict(rf2, test) == test$Species)
varImpPlot(rf2)

tidymodels Version

library(tidymodels)
set.seed(42)
split <- initial_split(iris, prop = 0.75, strata = Species)
wf <- workflow() |>
  add_formula(Species ~ .) |>
  add_model(rand_forest(trees = 500) |> set_engine("ranger") |> set_mode("classification"))
final <- last_fit(wf, split)                 # fits on training, evaluates on test
collect_metrics(final)

Key Settings

ranger randomForest Meaning
num.trees ntree Number of trees
mtry mtry Predictors per split (default floor(sqrt(p)) for classification)
min.node.size nodesize Minimum leaf size
importance importance = TRUE Variable importance

Common Pitfalls

  • Evaluating on the training data.

  • Forgetting set.seed() (or ranger’s seed), so results change each run.

  • Character outcomes: convert to factors for classification.

Interview Tips

  • Show split → fit → OOB → test metrics → importance.

  • Mention ranger as the fast modern implementation.

51

How would you forecast next quarter's sales in R?

To forecast next quarter’s sales in R, aggregate the sales history to a regular frequency (for example monthly), plot it to understand trend and seasonality, hold out the most recent months for testing, fit several candidate models such as seasonal naive, ETS (exponential smoothing) and ARIMA with the fable or forecast package, compare their accuracy on the hold-out period, then refit the best model on all data and forecast the next three months with prediction intervals, summing them for the quarter. Add known drivers such as promotions, prices or holidays as regressors when they help, and present the forecast as a range rather than a single number.

Read full answer

Applies toR 4.4+, fable 0.4, tsibble 1.1, feasts 0.4, dplyr 1.1+

Steps

  1. Prepare: aggregate transactions to monthly totals; fill gaps; handle outliers and calendar effects.

  2. Explore: plot, STL decomposition, seasonal plots.

  3. Split: last 6-12 months as the test period (or time-series cross-validation).

  4. Model: benchmarks (naive, seasonal naive) plus ETS, ARIMA, and regression with drivers.

  5. Evaluate: RMSE, MAE, MAPE or MASE on the test period.

  6. Forecast: refit on all data, forecast 3 months, aggregate to the quarter with intervals.

  7. Communicate: point forecast, 80%/95% intervals, assumptions.

Code Example

library(dplyr)
library(tsibble)
library(fable)
library(feasts)

# Example monthly sales with trend and seasonality (replace with your data)
set.seed(42)
months <- yearmonth("2021 Jan") + 0:59
sales <- tsibble(
  month = months,
  sales = round(1000 + 10 * (0:59) + 150 * sin(2 * pi * (0:59) / 12) + rnorm(60, 0, 40)),
  index = month
)

sales |> model(STL(sales ~ season(window = "periodic"))) |> components() |> autoplot()

train <- sales |> filter(month < yearmonth("2025 Jan"))
fits <- train |>
  model(snaive = SNAIVE(sales),
        ets = ETS(sales),
        arima = ARIMA(sales))
fits |> forecast(h = 12) |> accuracy(sales) |> arrange(RMSE)   # compare on 2025

best <- sales |> model(ets = ETS(sales))                       # refit the chosen model on all data
fc <- best |> forecast(h = 3)                                  # next quarter (3 months)
fc |> hilo(level = 95)

# Quarter total with an interval from simulated future paths
sims <- best |> generate(h = 3, times = 1000, bootstrap = TRUE)
totals <- sims |> as_tibble() |> group_by(.rep) |> summarise(total = sum(.sim))
quantile(totals$total, c(0.025, 0.5, 0.975))

Model Options

Model Good for
Seasonal naive Benchmark: same month last year
ETS Trend and seasonality, smooth changes
ARIMA Autocorrelation patterns, with seasonal terms
Regression with ARIMA errors Drivers like price, promotions, holidays
Prophet (fable.prophet) Multiple seasonalities, holiday effects

Common Pitfalls

  • Summing the upper bounds of three monthly intervals to get a quarterly interval; simulate instead.

  • Random train/test splits.

  • Forgetting structural changes (new stores, pricing changes).

Interview Tips

  • Always compare against a naive benchmark.

  • Explain how you would report uncertainty for the quarter.

52

How would you identify customer segments for market analysis in R?

Customer segmentation in R starts by building one row per customer with meaningful features, often RFM (recency, frequency, monetary value) plus product mix, channel and demographics, then transforming and scaling them, choosing a clustering method (k-means for numeric features, PAM with Gower distance for mixed types, hierarchical clustering or Gaussian mixtures), selecting the number of segments with silhouette or BIC, profiling and naming the segments, and validating that they are stable and actionable. The output should be a small set of segments that marketing can recognise and target differently.

Read full answer

Applies toR 4.4+, dplyr 1.1+, cluster 2.1+, mclust 6.1

Steps

  1. Customer table: aggregate transactions to recency, frequency, monetary value, average basket, category shares, tenure.

  2. Transform: log skewed money and counts; scale numeric features.

  3. Cluster: k-means, PAM (Gower distance for categorical + numeric), hierarchical, or mclust.

  4. Choose k: silhouette, elbow, BIC (mclust), and business usability (4-7 segments is common).

  5. Profile: segment sizes and averages; give names like “loyal high spenders” or “lapsed”.

  6. Validate: stability across samples and seeds; differences in outcomes such as churn or response.

  7. Act: campaigns per segment, then measure results.

Code Example

library(dplyr)
library(cluster)

# Example transactions (replace with real data)
set.seed(42)
tx <- tibble(
  customer = sample(1:300, 3000, replace = TRUE),
  date = as.Date("2026-01-01") + sample(0:300, 3000, replace = TRUE),
  amount = round(rlnorm(3000, meanlog = 3.5, sdlog = 0.6), 2)
)
today <- as.Date("2026-11-01")

rfm <- tx |>
  group_by(customer) |>
  summarise(recency = as.numeric(today - max(date)),
            frequency = n(),
            monetary = sum(amount), .groups = "drop")

X <- rfm |>
  mutate(frequency = log1p(frequency), monetary = log1p(monetary)) |>
  select(recency, frequency, monetary) |>
  scale()

set.seed(42)
sil <- sapply(2:7, function(k) {
  km <- kmeans(X, centers = k, nstart = 25)
  mean(silhouette(km$cluster, dist(X))[, "sil_width"])
})
k <- (2:7)[which.max(sil)]
set.seed(42)
rfm$segment <- factor(kmeans(X, centers = k, nstart = 50)$cluster)

rfm |>
  group_by(segment) |>
  summarise(customers = n(), recency = median(recency),
            frequency = median(frequency), monetary = median(monetary)) |>
  arrange(desc(monetary))

# Mixed numeric and categorical features: Gower distance + PAM
# d <- daisy(customer_features, metric = "gower"); pam(d, k = 4, diss = TRUE)

Method Choice

Method Use when
k-means Numeric, scaled features; large data
PAM + Gower Mixed types; robust to outliers
Hierarchical Small data; want a dendrogram
Gaussian mixture (mclust) Soft membership, elliptical clusters
RFM scoring (quintiles) Simple, rule-based, easy to explain

Common Pitfalls

  • Clustering raw skewed spend, so a few big customers dominate.

  • Too many segments to act on.

  • Segments that don’t differ in outcomes the business cares about.

Interview Tips

  • Mention RFM, scaling, choosing k and naming segments.

  • Tie segments to actions and measurement.

53

How would you use R to detect potentially fraudulent transactions?

In R, fraud detection combines supervised models trained on labelled past fraud, such as logistic regression, random forests (ranger) or gradient boosting (xgboost) with class weights or resampling for the rare fraud class, and unsupervised anomaly detection for new patterns, such as isolation forests (isotree) or distance-based outlier scores, on engineered behavioural features like amount relative to the customer’s usual spend, transaction velocity, new merchants or devices, and unusual times or locations. Models are validated on later time periods with precision-recall metrics, and alert thresholds are set from investigation capacity and the cost of missed fraud.

Read full answer

Applies toR 4.4+, dplyr 1.1+, tidymodels 1.2, themis 1.0, isotree 0.6

Approach

Step R tools
Feature engineering per customer and time dplyr window functions (lag(), cumulative stats), slider
Imbalance themis::step_downsample(), step_smote(), class weights
Supervised models logistic_reg(), rand_forest(), boost_tree()
Anomaly detection isotree::isolation.forest(), Mahalanobis distance
Time-based validation rsample::initial_time_split(), sliding_period()
Metrics yardstick::pr_auc(), precision and recall at a threshold

Code Example

library(dplyr)
library(tidymodels)
library(themis)

set.seed(42)
n <- 20000
tx <- tibble(
  customer = sample(1:2000, n, replace = TRUE),
  time = sort(as.POSIXct("2026-01-01", tz = "UTC") + runif(n, 0, 300 * 86400)),
  amount = rlnorm(n, 3.5, 0.8),
  foreign = rbinom(n, 1, 0.03)
) |>
  arrange(customer, time) |>
  group_by(customer) |>
  mutate(avg_amount = lag(cummean(amount)),                       # customer's past average
         amount_ratio = amount / coalesce(avg_amount, amount),
         hours_since_last = as.numeric(difftime(time, lag(time), units = "hours"))) |>
  ungroup() |>
  mutate(hours_since_last = coalesce(hours_since_last, 1000),
         risk = -6 + 1.2 * log(amount_ratio + 0.1) + 2 * foreign - 0.002 * hours_since_last,
         fraud = factor(rbinom(n, 1, plogis(risk)), levels = c(1, 0))) |>
  arrange(time)

split <- initial_time_split(tx, prop = 0.8)                       # train on earlier transactions
rec <- recipe(fraud ~ amount_ratio + hours_since_last + foreign + amount, data = training(split)) |>
  step_log(amount) |>
  step_downsample(fraud, under_ratio = 5)                         # applied to training data only
model <- boost_tree(trees = 300, learn_rate = 0.05) |> set_engine("xgboost") |> set_mode("classification")
fit <- workflow() |> add_recipe(rec) |> add_model(model) |> fit(training(split))

preds <- augment(fit, testing(split))
pr_auc(preds, truth = fraud, .pred_1)                             # fraud is the first level
roc_auc(preds, truth = fraud, .pred_1)

# Unsupervised score for unusual transactions
library(isotree)
iso <- isolation.forest(training(split)[, c("amount_ratio", "hours_since_last")], ntrees = 200)
testing(split) |> mutate(anomaly = predict(iso, testing(split)[, c("amount_ratio", "hours_since_last")])) |>
  slice_max(anomaly, n = 5)

Common Pitfalls

  • Random splits that let the model see the future.

  • Features using information only available after the transaction (e.g. chargeback status).

  • Reporting accuracy instead of PR AUC and alert precision.

  • Resampling before splitting.

Interview Tips

  • Combine supervised models with anomaly detection.

  • Explain time-based validation and threshold choice from business costs.

54

How would you build a recommendation system in R?

A recommendation system in R can be built with the recommenderlab package, which takes a user-item rating matrix (realRatingMatrix or binaryRatingMatrix) and provides popularity, user-based and item-based collaborative filtering, SVD-style matrix factorization and association-rule recommenders with a shared evaluation framework; for larger or implicit data, matrix factorization with rsparse (weighted ALS) or recosystem, or Spark ALS through sparklyr, is used. Evaluate with held-out ratings using top-N precision and recall or RMSE, compare against a popularity baseline, and add content features for new items.

Read full answer

Applies toR 4.4+, recommenderlab 1.0.6, recosystem 0.5

Options

Approach R package Note
Popularity recommenderlab ("POPULAR") Baseline, cold start
User-based CF recommenderlab ("UBCF") Similar users
Item-based CF recommenderlab ("IBCF") Similar items, precomputable
Matrix factorization recommenderlab ("SVDF", "LIBMF"), recosystem Sparse explicit ratings
Implicit feedback ALS rsparse::WRMF, sparklyr::ml_als(implicit_prefs = TRUE) Clicks, plays
Association rules arules, recommenderlab "AR" Market baskets
Content-based text2vec/tidytext + cosine similarity New items

Code Example

library(recommenderlab)

data(MovieLense)                             # 943 users x 1664 movies, ratings 1-5
ratings <- MovieLense[rowCounts(MovieLense) >= 50, colCounts(MovieLense) >= 50]

set.seed(42)
scheme <- evaluationScheme(ratings, method = "split", train = 0.8, given = 10, goodRating = 4)

algorithms <- list(
  popular = list(name = "POPULAR"),
  ubcf = list(name = "UBCF", param = list(method = "cosine", nn = 30)),
  ibcf = list(name = "IBCF", param = list(k = 30)),
  svdf = list(name = "SVDF", param = list(k = 20))
)
results <- evaluate(scheme, algorithms, type = "topNList", n = c(5, 10))
avg(results)                                 # precision, recall, TPR, FPR per algorithm
plot(results, annotate = TRUE)               # ROC-style comparison

model <- Recommender(getData(scheme, "train"), method = "UBCF",
                     parameter = list(method = "cosine", nn = 30))
top5 <- predict(model, getData(scheme, "known")[1:3], n = 5)
as(top5, "list")                             # recommended movie titles for 3 users

# Rating prediction error
pred <- predict(model, getData(scheme, "known"), type = "ratings")
calcPredictionAccuracy(pred, getData(scheme, "unknown"))   # RMSE, MSE, MAE

Design Considerations

  • Explicit vs implicit: ratings use realRatingMatrix; clicks use binaryRatingMatrix or weighted ALS.

  • Cold start: popularity and content-based features for new users and items.

  • Scale: recommenderlab is for small to medium data; use recosystem, rsparse or Spark for millions of ratings.

  • Serving: precompute top-N lists and expose them with plumber or a database.

Common Pitfalls

  • Evaluating only with RMSE when the product shows top-N lists.

  • No popularity baseline.

  • Leaking future ratings into training; split by time where possible.

Interview Tips

  • Name recommenderlab’s methods and its evaluation scheme.

  • Mention scaling options and cold start handling.

55

How do you use R for sentiment analysis of customer feedback?

R analyses customer feedback text with the tidytext approach: tokenize the comments into words with unnest_tokens(), remove stop words, and either score sentiment with lexicons such as Bing, AFINN or NRC (joined with get_sentiments()), or train a supervised classifier on labelled feedback with tidymodels and the textrecipes package (tokenize, TF-IDF, then a lasso or random forest model). Lexicon methods are quick and need no labels but miss context and negation; supervised models or transformer models via text or reticulate are more accurate when labelled data exists.

Read full answer

Applies toR 4.4+, tidytext 0.4, dplyr 1.1+, tidymodels 1.2, textrecipes 1.0

Approaches

Approach Tools Pros / cons
Lexicon scoring tidytext::get_sentiments("bing" / "afinn") Fast, no labels; misses negation and sarcasm
Sentence-level lexicon with negation sentimentr Handles valence shifters (“not good”)
Supervised ML textrecipes + glmnet/ranger Learns domain language; needs labels
Topic modelling topicmodels, stm What customers talk about
Transformer embeddings text package, reticulate + Hugging Face Most accurate; heavier

Code Example

library(dplyr)
library(tidytext)

feedback <- tibble(
  id = 1:5,
  text = c("The delivery was fast and the product is excellent",
           "Terrible support, I waited two weeks for a refund",
           "Good quality but the price is too high",
           "Not happy with the broken packaging",
           "Love it, great value and friendly staff")
)

words <- feedback |>
  unnest_tokens(word, text) |>          # lowercase, one word per row
  anti_join(stop_words, by = "word")

bing <- get_sentiments("bing")          # positive / negative word list
scores <- words |>
  inner_join(bing, by = "word") |>
  count(id, sentiment) |>
  tidyr::pivot_wider(names_from = sentiment, values_from = n, values_fill = 0) |>
  mutate(score = positive - negative)
feedback |> left_join(scores, by = "id") |> select(id, score, text)
# Note: "Not happy" in comment 4 may be scored positive by a word-level lexicon

# Most frequent negative words
words |> inner_join(bing, by = "word") |> filter(sentiment == "negative") |> count(word, sort = TRUE)

# Supervised model when labels exist
library(tidymodels)
library(textrecipes)
labelled <- feedback |> mutate(label = factor(c("pos", "neg", "pos", "neg", "pos")))
rec <- recipe(label ~ text, data = labelled) |>
  step_tokenize(text) |>
  step_stopwords(text) |>
  step_tokenfilter(text, max_tokens = 500) |>
  step_tfidf(text)
spec <- logistic_reg(penalty = 0.01, mixture = 1) |> set_engine("glmnet")
wf <- workflow() |> add_recipe(rec) |> add_model(spec)
# fit(wf, data = real_labelled_feedback)   # needs hundreds of labelled comments

Common Pitfalls

  • Negation and sarcasm with word-level lexicons; use sentimentr or supervised models.

  • Removing stop words like “not” that change meaning.

  • Lexicons built for other domains (e.g. “cheap” can be positive or negative).

  • Too few labelled examples for supervised models.

Interview Tips

  • Start with the tidytext lexicon approach and explain its limits.

  • Mention textrecipes for supervised models and topic models for themes.

57

How does R handle big data and what are the best practices?

By default R holds data in memory, so big data is handled by choosing tools that avoid loading everything at once or that push computation elsewhere: data.table for fast in-memory work, arrow and DuckDB for querying files larger than memory (Parquet datasets with dplyr syntax), dbplyr to translate dplyr code into SQL that runs in the database, sparklyr for Spark clusters, and parallel processing with future, furrr or parallel. Best practices are to filter and aggregate before collecting into R, store data in columnar formats such as Parquet, use efficient types, and profile memory and time.

Read full answer

Applies toR 4.4+, data.table 1.15+, arrow 16+, duckdb 1.0+, dbplyr 2.5+, sparklyr 1.8+

Tools

Situation Tool
Fits in memory, needs speed data.table, collapse, dtplyr
Larger than memory, files arrow::open_dataset() + dplyr, duckdb
Data in a database DBI + dbplyr (dplyr → SQL)
Cluster computing sparklyr (Spark), h2o
Parallel R code future/furrr, parallel::mclapply()
Out-of-core models biglm (bounded memory), Spark MLlib via sparklyr

Code Example

library(dplyr)
library(arrow)

# Write a partitioned Parquet dataset (example data)
dir <- file.path(tempdir(), "flights_ds")
big <- tibble(year = rep(2024:2026, each = 100000),
              carrier = sample(c("AA", "BA", "LH"), 300000, replace = TRUE),
              delay = rnorm(300000, 10, 20))
write_dataset(big, dir, partitioning = "year", format = "parquet")

# Query lazily: only the needed columns and partitions are read
ds <- open_dataset(dir)
ds |>
  filter(year == 2026, delay > 0) |>
  group_by(carrier) |>
  summarise(mean_delay = mean(delay), n = n()) |>
  collect()                                    # computation happens before collect()

# The same with DuckDB on the Parquet files
library(duckdb)
con <- dbConnect(duckdb())
tbl(con, sql(paste0("SELECT * FROM read_parquet('", dir, "/*/*.parquet', hive_partitioning = true)"))) |>
  group_by(carrier) |>
  summarise(mean_delay = mean(delay, na.rm = TRUE)) |>
  collect()
dbDisconnect(con, shutdown = TRUE)

# Fast in-memory aggregation with data.table
library(data.table)
dt <- as.data.table(big)
dt[delay > 0, .(mean_delay = mean(delay), .N), by = .(year, carrier)]

Best Practices

  • Filter, select and aggregate in the database or Arrow before collect().

  • Store data as Parquet, partitioned by commonly filtered columns.

  • Use appropriate types (integers, factors, dates) and drop unused columns.

  • Avoid growing objects in loops; preallocate or vectorize.

  • Parallelize independent tasks (model fits, simulations) with future.

  • Profile with profvis and bench; check memory with lobstr::obj_size().

Common Pitfalls

  • read.csv() on multi-gigabyte files.

  • Calling collect() too early, pulling everything into memory.

  • Copies created by modifying large data frames repeatedly (data.table updates by reference with :=).

Interview Tips

  • Mention arrow/DuckDB for larger-than-memory data and dbplyr for databases.

  • Explain lazy evaluation and pushing computation to where the data lives.

58

What role does R play in reproducible research?

R supports reproducible research by letting analysis code, results and narrative live together in literate documents (R Markdown and Quarto), so every number, table and figure is regenerated from the data when the document is rendered, and by providing tools to pin the computing environment and pipeline: renv records exact package versions, targets builds analysis pipelines that rerun only what changed, Git tracks code history, and Docker images (for example from the Rocker project) capture the whole system. Combined with scripted data cleaning, fixed random seeds and shared data, this lets others rerun the analysis and get the same results.

Read full answer

Applies toR 4.4+, Quarto 1.5+, rmarkdown 2.28, renv 1.0, targets 1.7

Building Blocks

Need Tool
Code + results + text in one document Quarto (.qmd), R Markdown (.Rmd)
Exact package versions renv (renv.lock)
Pipelines and caching targets
Version control Git, GitHub/GitLab
Whole environment Docker (Rocker images)
Random results set.seed()
Project structure RStudio projects, here::here() for paths
Packaging analysis code R package structure, testthat tests
Sharing data pins, data repositories with DOIs

Code Example

A Quarto document (report.qmd) starts with a YAML header:

title: "Fuel economy analysis"
format: html
execute:
  echo: true

Below the header, prose is mixed with R code chunks (opened with three backticks followed by {r}), and rendering reruns every chunk:

set.seed(42)
model <- lm(mpg ~ wt + hp, data = mtcars)

Inline code keeps numbers in the text in sync: a backtick, the letter r and an expression such as round(coef(model)["wt"], 2), closed with another backtick, prints the current weight coefficient inside a sentence when the document is rendered.

A targets pipeline (_targets.R):

library(targets)
tar_option_set(packages = c("dplyr", "ggplot2"))

list(
  tar_target(raw, read.csv("data/cars.csv")),
  tar_target(clean, dplyr::filter(raw, !is.na(mpg))),
  tar_target(model, lm(mpg ~ wt + hp, data = clean)),
  tar_target(report, quarto::quarto_render("report.qmd"), format = "file")
)
# tar_make() runs only the steps whose inputs changed; renv::snapshot() records package versions

Practices

  • Never edit data by hand; script every transformation from raw data.

  • Use relative paths (here::here()), not setwd() with absolute paths.

  • Record sessionInfo() in reports.

  • Keep raw data read-only and separate from derived data.

Common Pitfalls

  • Results copied manually into Word or slides, which go stale.

  • Unpinned package versions changing results months later.

  • Random steps without seeds.

Interview Tips

  • Mention Quarto/R Markdown, renv and targets as the core trio.

  • Explain inline code that keeps numbers in the text in sync with the analysis.

59

How can R be used together with other programming languages?

R works with other languages through dedicated bridges: reticulate runs Python code and libraries inside R and converts objects between them, Rcpp lets you write fast C++ functions callable from R, .Call()/.C() interface with C and Fortran (which much of base R uses), DBI sends SQL to databases, JuliaCall and rJava reach Julia and Java, and Quarto documents can mix R, Python, SQL and other chunks. Across processes, languages exchange data through Arrow and Parquet, and services through REST APIs built with plumber or called with httr2.

Read full answer

Applies toR 4.4+, reticulate 1.38, Rcpp 1.0.13, DBI 1.2, plumber 1.2, httr2 1.0

Options

Language / need R tool Typical use
Python reticulate Use scikit-learn, PyTorch, Hugging Face from R
C++ Rcpp, cpp11 Speed up loops and algorithms
C / Fortran .Call(), .Fortran() Low-level numerical code
SQL DBI, dbplyr, Quarto sql chunks Query databases
Java rJava Java libraries
Julia JuliaCall Julia numerical code
JavaScript htmlwidgets, V8 Interactive visualizations
Any language via data arrow (Feather/Parquet) Zero-copy-style data exchange
Any language via services plumber (serve), httr2 (call) Microservices
Shell system2(), processx Command-line tools

Code Example

# Python from R
library(reticulate)
np <- import("numpy")
np$mean(c(1, 2, 3, 4))                    # 2.5 computed by NumPy
py_run_string("squares = [i ** 2 for i in range(5)]")
py$squares                                # list(0, 1, 4, 9, 16) back in R

# C++ from R
library(Rcpp)
cppFunction("
double sum_squares(NumericVector x) {
  double total = 0;
  for (int i = 0; i < x.size(); ++i) total += x[i] * x[i];
  return total;
}")
sum_squares(c(1, 2, 3))                   # 14

# SQL from R
library(DBI)
con <- dbConnect(RSQLite::SQLite(), ":memory:")
dbWriteTable(con, "cars", mtcars)
dbGetQuery(con, "SELECT cyl, COUNT(*) AS n FROM cars GROUP BY cyl")   # 4: 11, 6: 7, 8: 14
dbDisconnect(con)

# Share data with Python or Spark through Parquet
# arrow::write_parquet(mtcars, "mtcars.parquet")   # pandas.read_parquet("mtcars.parquet")

# Expose an R model as an API (plumber.R)
# #* @post /predict
# function(wt, hp) predict(model, data.frame(wt = as.numeric(wt), hp = as.numeric(hp)))

Common Pitfalls

  • reticulate using the wrong Python environment; set it with use_virtualenv() or RETICULATE_PYTHON.

  • Copying large objects between R and Python repeatedly; use Arrow.

  • Compilers missing for Rcpp on Windows (install Rtools).

Interview Tips

  • Mention reticulate and Rcpp first; they are the most used.

  • Explain data exchange with Arrow/Parquet and APIs with plumber.

60

What are R's limitations for machine learning and how can they be overcome?

R’s main limitations for machine learning are that it keeps data in memory by default and is single-threaded for most base code, its deep learning and large-scale ML ecosystems are smaller than Python’s, production deployment and MLOps tooling are less widespread, and the many overlapping packages (caret, tidymodels, mlr3) and inconsistent interfaces can confuse newcomers. They can be overcome with larger-than-memory and distributed tools (arrow, DuckDB, data.table, sparklyr), parallel processing (future), compiled code (Rcpp), unified frameworks (tidymodels or mlr3), deep learning through torch for R or reticulate, and deployment with vetiver, plumber, Docker and Posit Connect.

Read full answer

Applies toR 4.4+ (tidymodels 1.2, torch 0.13, vetiver 0.2)

Limitations and Workarounds

Limitation Workaround
In-memory data arrow, DuckDB, dbplyr, sparklyr
Speed of loops Vectorization, data.table, Rcpp
Mostly single-threaded future/furrr, parallel, multi-threaded engines (ranger, xgboost)
Smaller deep learning ecosystem torch for R, keras3, or reticulate with PyTorch
Fewer pretrained NLP/vision models reticulate + Hugging Face, text package
Production deployment vetiver, plumber APIs, Docker, Posit Connect, ONNX export for some models
Inconsistent package interfaces tidymodels or mlr3 as one interface
Memory copies data.table reference semantics, avoiding repeated copies
Smaller engineering talent pool Clear code standards, packages with tests, mixed R/Python teams

Code Example

# Parallel cross-validation with future
library(tidymodels)
library(future)
plan(multisession, workers = 4)                 # tune_grid uses the workers automatically

set.seed(42)
folds <- vfold_cv(mtcars, v = 5)
spec <- rand_forest(mtry = tune(), trees = 300) |> set_engine("ranger", num.threads = 1) |>
  set_mode("regression")
res <- tune_grid(workflow(mpg ~ ., spec), resamples = folds, grid = 5)
plan(sequential)

# Deep learning without leaving R
library(torch)
torch_manual_seed(42)
net <- nn_sequential(nn_linear(10, 16), nn_relu(), nn_linear(16, 1))
x <- torch_randn(32, 10)
net(x)$shape                                    # 32 1

# Serve the tuned model as an API
library(vetiver)
best_fit <- finalize_workflow(workflow(mpg ~ ., spec), select_best(res, metric = "rmse")) |> fit(mtcars)
v <- vetiver_model(best_fit, "mpg_rf")
# plumber::pr() |> vetiver_api(v) |> plumber::pr_run(port = 8000)

When to Use Another Language

  • Large-scale deep learning, LLMs and computer vision: Python is usually the better primary language.

  • Low-latency services embedded in a software stack: Python, Java, Go or compiled models.

  • Keep R for statistics, exploration, reporting and models where its tooling is strong.

Common Pitfalls

  • Forcing every task into R (or Python) out of habit.

  • Nested parallelism (parallel tuning plus multi-threaded engines) oversubscribing CPUs.

Interview Tips

  • Give an honest list and a workaround for each limitation.

  • Mention vetiver and torch to show current knowledge.

Sign in to keep reading

You've opened 2 full answers. Sign in to read all 60 R answers, mark what you know and pick up where you left off.

Report a mistake

What's wrong?

All Machine Learning & Data Science subjects