Language: R · Versions: R 4.4.2
The problem
Two subtle evaluation semantics in R 4.4 bite when writing higher-order functions:
1. Because function arguments in R are lazy promises (PRSEXP) that remain unevaluated until first read, building a list of closures via lapply(1:3, function(i) function() i) leaves every i promise unevaluated until after lapply finishes if the loop variable is mutated (in a for loop), or in for (i in 1:3) fns[[i]] <- (function(x) function() x)(i) where x is never forced inside the factory!
2. Inside a helper function filter_above <- function(df, threshold) subset(df, score > threshold), if df happens to contain a column named threshold, subset's non-standard evaluation (eval(substitute(subset), data, parent.frame())) resolves threshold to the column df$threshold instead of the parameter threshold!
Minimal example
# Trap 1: Unforced promise in factory function
make_adder <- function(delta) {
# `delta` is a lazy promise; if caller passes an expression with side effects or mutated env:
function(x) x + delta
}
fns <- list()
for (i in 1:3) {
fns[[i]] <- make_adder(i)
}
# All three return 13 (10 + 3) instead of 11, 12, 13!
print(sapply(fns, function(f) f(10)))
# Trap 2: Column masking in subset() NSE
df <- data.frame(score = c(10, 20, 30), threshold = c(5, 25, 35))
filter_above <- function(d, threshold) {
subset(d, score > threshold) # Evaluates `d$score > d$threshold` instead of `threshold = 15`!
}
print(filter_above(df, threshold = 15))
Expected
11, 12, 13fromsapply(fns, function(f) f(10)).- Rows where
score > 15(20and30), ignoringdf$threshold.
What happens instead
- In
for (i in 1:3) fns[[i]] <- make_adder(i), the promise fordeltacaptures the expressioniin the caller environment and is never evaluated insidemake_adderuntilf(10)runs—by which timeiis3! Callingforce(delta)insidemake_adderis required. - In
subset(d, score > threshold),evallooks indbeforeparent.frame(), sod$thresholdshadows the function argumentthreshold.
What are the cleanest base-R (4.4) ways to prevent data-masking collisions in NSE functions without depending on rlang (.data / .env)?
Trap 1: force the argument in the factory
make_adder <- function(delta) {
force(delta) # read the promise now, while i is still 1, 2, 3
function(x) x + delta
}
fns <- list()
for (i in 1:3) fns[[i]] <- make_adder(i)
sapply(fns, function(f) f(10))
#> [1] 11 12 13
The for-loop version behaves as you describe. Each make_adder(i) call creates a promise for delta whose expression is i, evaluated in the global environment. Nothing reads that promise until f(10) runs, and by then the loop has left i at 3. force() is just function(x) x. Calling it reads the promise once, and from then on the closure holds the value, not the expression.
One correction to your premise: lapply(1:3, function(i) function() i) does not have this problem in R 4.4. R 3.2.0's NEWS says the apply functions and Reduce() now force the arguments to the functions they apply, to get rid of exactly this clash between lazy evaluation and closures. So this returns 1, 2, 3. Hand-written for loops and your own factories still need force().
Trap 2: don't send a parameter through subset()'s data mask
Also real. subset.data.frame does eval(substitute(subset), x, parent.frame()), so columns come before the caller's variables. With your data, score > threshold compares against the column, row by row (10 > 5, 20 > 25, 30 > 35), so you get only row 1. Here are the base-R ways out, in the order I'd pick them.
1. Don't use NSE inside functions. ?subset itself says it is meant for interactive use and that functions should use [:
filter_above <- function(d, threshold) {
d[which(d$score > threshold), , drop = FALSE] # which() drops NA rows, as subset() does
}
which() matters here: plain d[d$score > threshold, ] gives you all-NA rows wherever score is NA. subset() drops those rows (r & !is.na(r)).
2. If you want to keep subset(), put the value into the call with bquote():
filter_above <- function(d, threshold) {
eval(bquote(subset(d, score > .(threshold))))
}
By the time subset() substitutes its argument, the expression already holds the number 15, not the symbol threshold, so no column can shadow it. This is the closest base-R equivalent to rlang's !! / .env.
3. When you write your own NSE function, make columns explicit rather than letting them shadow variables:
my_filter <- function(data, cond) {
r <- eval(substitute(cond), list(.data = data), parent.frame())
data[which(r), , drop = FALSE]
}
my_filter(df, .data$score > threshold) # threshold always comes from the caller
The only name the data adds is .data, so a column called threshold can't get in the way. (.data$col is a plain $ on a data frame here, with none of rlang's pronoun error-checking.)
Avoid env <- environment(); subset(d, score > env$threshold). It just moves the problem: a column named env breaks it.
What I checked and what I didn't: the R 3.2.0 change and the eval(e, x, parent.frame()) line are from R's NEWS and the subset.data.frame source, and the outputs above follow from them. I can't run code here. print(sapply(..., function(f) f(10))) in a fresh R 4.4.2 session would confirm both.