Language: Lua · Versions: Lua 5.4.7
The problem
I'm using Lua 5.4's to-be-closed variables (local f <close> = acquire()) inside a coroutine (coroutine.create / coroutine.wrap) to guarantee deterministic cleanup of file handles and database cursors. However, when the caller consumes only part of the coroutine and lets it be garbage-collected without resuming it to completion or calling coroutine.close(co), the __close metamethod is never called! Even worse, with coroutine.wrap, the underlying thread object isn't returned to the caller, so the caller can't call coroutine.close on it directly!
Minimal example
local function make_resource()
return setmetatable({}, {
__close = function(self, err)
print("Resource closed! err =", err)
end
})
end
local gen = coroutine.wrap(function()
local res <close> = make_resource()
coroutine.yield("first")
coroutine.yield("second")
end)
print("Got:", gen())
-- Caller stops after first item and GCs `gen`:
gen = nil
collectgarbage("collect")
print("After GC (notice __close never ran!)")
Expected
Deterministic invocation of __close when breaking out of a for x in gen do break end loop in Lua 5.4.
What happens instead
Because <close> hooks live on the coroutine's call stack and Lua's garbage collector does not unwind suspended coroutine stacks on collection (only __gc finalizers run during GC), dropping a suspended coroutine skips <close> unless coroutine.close(co) is explicitly called—and Lua 5.4's generic for loop supports a 4th control variable (a to-be-closed variable returned by the iterator factory!).
How do you combine coroutine.create, coroutine.close, and the 4th return value of a generic for iterator in Lua 5.4 so for x in stream() do break end automatically closes the suspended coroutine and runs all <close> handlers inside it?
Your premise holds. Dropping a suspended coroutine never runs its pending <close> variables. Only coroutine.close(co) does that, and coroutine.wrap doesn't give you the thread. The fix is to use coroutine.create and return the closer as the 4th value of the generic for explist.
local function stream(body)
local co = coroutine.create(body)
local function step()
local ok, v = coroutine.resume(co)
if not ok then error(v, 0) end -- body failed
if coroutine.status(co) == "dead" then return nil end
return v
end
local closer = setmetatable({}, {
__close = function()
local st = coroutine.status(co)
if st == "suspended" then
local ok, err = coroutine.close(co) -- runs the body's pending <close>s
if not ok then error(err, 0) end
end
end
})
return step, nil, nil, closer -- iterator, state, control, closing value
end
for x in stream(function()
local res <close> = make_resource()
coroutine.yield("first")
coroutine.yield("second")
end) do
print(x)
break -- closer.__close -> coroutine.close(co) -> res.__close
end
Why it works
- Since Lua 5.4.0, the generic for takes four values from its explist. The fourth is a to-be-closed variable (the manual calls it the closing value). It is closed on normal loop end, break, goto out, return, or an error.
- coroutine.close(co) works on a suspended or dead coroutine. It closes the coroutine's pending to-be-closed variables in reverse order and leaves it dead. It returns true, or false, err if a __close raised an error.
- The closing value must have a __close metamethod, or be nil/false. Otherwise the loop raises an error about a non-closable value.
Caveats
- If the body errors, resume returns false, err and the stack is not unwound. The body's <close> handlers are still pending. So I let __close call coroutine.close only when the status is suspended. For the error case, add a branch that calls coroutine.close(co) on a dead-by-error coroutine, which returns false, err. Then decide whether to rethrow, so you don't report the same error twice.
- This only helps in a for loop. A caller who calls step by hand and drops it gets no cleanup. A __gc on closer could call coroutine.close, but that runs arbitrary code at a GC point, so I'd avoid it. I haven't checked that path.
I'm going from the Lua 5.4 reference manual (generic for, coroutine.close, to-be-closed variables) and haven't run this on 5.4.7. A quick run with your make_resource should show err = nil printed once after the break.