GcWatch & ErrorWatch

Two small observability modules that make live servers legible. GcWatch derives garbage-collector health from the only signal Roblox exposes — the heap size — and turns it into allocation rates, cycle cadence, and reclaim sizes. ErrorWatch captures every script error, deduplicates identical ones, and reports each unique error once per window with an occurrence count — a crash loop becomes one line with (x4000), not four thousand lines. Both are shared modules: they attach to a server kernel or a client kernel the same way.

GcWatch

Mental model

Roblox exposes no GC pause timings — collectgarbage is restricted to "count", and there is no callback for collection events. What is readable is gcinfo(), the current Lua heap in KB. GcWatch samples it on a recurring Scheduler task (default every 0.1s, at Priority.Background) and reconstructs GC behavior from the deltas between samples:

• Heap grew — allocation happened. The growth accumulates into the current rate window. If the heap had been shrinking until now, the grow marks the end of one GC cycle: the cycle count increments and the shrink run’s total becomes LastReclaimKb.
• Heap shrank — a sweep is reclaiming. Consecutive shrinking samples accumulate into one reclaim run; the run stays open until growth resumes.
• Every window (default 5s), the accumulated allocation and cycle counts convert into AllocKbPerSec and CyclesPerMin, then the window resets. Rates describe the last window, not a forever-average — a leak that started ten minutes in shows up in the next window, not diluted across the session.

The numbers are approximate by nature: Luau’s GC is incremental, so sweeps interleave with allocation and one sample interval can net a grow and a shrink into a single delta. The shape is what matters, and the shape is reliable — the specs drive the sampler with injected heap curves and assert exact cycle bookkeeping.

Allocation rate is the lever. Every KB allocated must later be marked and swept by the incremental GC, and allocation spikes trigger GC assists inside your frames. Cut allocation and you cut GC time — there is no other knob.

The two stats together separate the two memory failure modes:

Heap curve	Cycles / reclaims	Diagnosis
Climbing	Reclaims near zero	Leak — the GC runs but has nothing to free. Look for insert-only tables, undisconnected connections, per-player state not cleared (session:bind exists for exactly this).
Stable	Busy cycles, steady reclaims	Churn — tables/strings created per frame and thrown away. Pool or hoist them; see Pool.

Usage

-- src/Server/Bootstrap.luau
local ReplicatedStorage = game:GetService("ReplicatedStorage")
local GcWatch = require(ReplicatedStorage.ChloeKernel.GcWatch)
local Logger = require(ReplicatedStorage.ChloeKernel.Logger)

return function(kernel)
  local Log = Logger.scope("Health")
  local Gc = GcWatch.attach(kernel)

  kernel.Scheduler:every(30, function()
    local Stats = Gc:stats()
    Log.info(`heap {math.round(Stats.HeapKb / 1024)}MB, alloc {math.round(Stats.AllocKbPerSec)}KB/s, {Stats.CyclesPerMin} cycles/min`)
    if Stats.AllocKbPerSec > 4096 then
      Log.warn("allocation rate over 4MB/s — GC assists will start eating frames")
    end
  end, kernel.Priority.Background)
end

Or skip the manual wiring entirely: kernel:enableDiagnostics() attaches a GcWatch for you (see Diagnostics integration).

API reference

Member	Description
GcWatch.attach(kernel, options?) → watch	Starts sampling on kernel.Scheduler:every(SampleSeconds, ..., Priority.Background). Reading gcinfo() is a counter read — effectively free.
watch:stats() → Stats	Current snapshot (table below).
watch:destroy()	Cancels the recurring sampler.

Options (all optional):

Field	Default	Description
SampleSeconds	0.1	Sampling interval.
WindowSeconds	5	Rate window — AllocKbPerSec and CyclesPerMin recompute and reset on this cadence.
Heap	gcinfo	Injectable heap reader, for specs.
Clock	os.clock	Injectable clock, for specs.
SkipLoop	false	Do not schedule the sampler; the caller drives _sample(now) manually (specs do this for determinism).

Stats fields:

Field	Meaning
HeapKb	Most recent heap sample.
PeakHeapKb	High-water mark since attach.
AllocKbPerSec	Allocation rate over the last closed window.
CyclesPerMin	GC cycle cadence over the last closed window.
Cycles	Total grow→shrink→grow cycles observed since attach.
LastReclaimKb	KB freed by the most recent complete sweep run.
AvgReclaimKb	ReclaimedKb / Cycles (0 before the first cycle).
ReclaimedKb	Total KB reclaimed since attach.

Note

Before the first window closes, stats() estimates rates from the partial window once at least 1 second has elapsed; before that, rates read 0. Attach early (boot time) so numbers are meaningful when you first look.

Diagnostics integration

kernel:enableDiagnostics(intervalSeconds?) attaches one GcWatch automatically (reusing the kernel’s existing one if present) and publishes three of its stats as ReplicatedStorage attributes every interval:

Attribute	Source
ChloeKernelDiag{Role}GcAllocKbS	AllocKbPerSec, rounded
ChloeKernelDiag{Role}GcCyclesMin	CyclesPerMin, one decimal
ChloeKernelDiag{Role}GcReclaimKb	LastReclaimKb, rounded

The debug panel reads these into graded GC rows in both the CLIENT and SERVER windows — its leak-vs-churn tooltips are this module’s diagnosis table made hoverable. kernel:shutdown() destroys the diagnostics-attached watch. See Kernel & Boot for the rest of the diagnostics surface.

Caution

The sampler runs at Priority.Background, which stops executing during sustained scheduler overload — so during exactly the incident you are debugging, GcWatch rates can go stale (they keep the last closed window’s values). The diagnostics loop itself runs at Priority.Kernel for this reason, and Kernel.Overload on the bus is the signal designed to survive overload — use GcWatch for memory diagnosis, not overload detection.

Caution

gcinfo() is per-VM, and in Studio the “server” and client run in one process — the server kernel’s heap reading in Studio includes editor-adjacent noise and is separate only in production. Trust production numbers; treat Studio numbers as relative.

ErrorWatch

Mental model

ErrorWatch merges two error streams into one deduplicated report:

1. ScriptContext.Error — every unhandled script error on this machine, whether or not the code involves the kernel.
2. Scheduler.OnError — task crashes the Scheduler caught and isolated. These never reach ScriptContext (the scheduler pcalls its tasks), so without this second tap a crash-looping scheduled task would be invisible to error telemetry. The error’s first line becomes the message, the full text the trace, and the source reads KernelScheduler.

Both streams funnel through the same dedup:

• Key = the error message plus the first line of the traceback. Two different call sites raising the same message count separately; the same line crashing 4000 times counts once.
• Window (default 30s): the first report of a key logs immediately — through Logger scope ErrorWatch at error level, formatted {message} [{source}] (x{count}) — and publishes Kernel.ScriptError on the bus. Repeats within the window increment the count silently. The first report after the window logs and publishes again, carrying the accumulated count.
• Eviction: interpolated values (names, ids, coordinates) mint a unique key each, and without eviction the Seen table would grow for the server’s whole lifetime. At most once per window, entries unseen for WindowSeconds * 10 (default 300s) are pruned.

The result: sinks and the bus see at most one line per unique error per window, while nothing is ever dropped from the count.

Usage

-- src/Server/Bootstrap.luau
local ReplicatedStorage = game:GetService("ReplicatedStorage")
local ErrorWatch = require(ReplicatedStorage.ChloeKernel.ErrorWatch)

return function(kernel)
  local Watcher = ErrorWatch.attach(kernel)

  -- Ship every deduplicated occurrence to your telemetry
  kernel.Bus:subscribe("Kernel.ScriptError", function(_, message, trace, source, count)
    warn(`[ops] {source}: {message} (x{count})`)
  end)
end

For webhook alerting on errors generally (not just script errors), add a Logger sink filtered to Logger.Level.Error — ErrorWatch’s windowing means the sink receives one line per unique error per window, so the webhook cannot be flooded by a crash loop.

API reference

Member	Description
ErrorWatch.attach(kernel?, options?) → watcher	Connects ScriptContext.Error, and kernel.Scheduler.OnError when a kernel is given. kernel is optional: without one, errors still log through the Logger, but nothing publishes on the bus.
watcher:counts() → { [key]: count }	Snapshot of occurrence counts, keyed by the dedup key (message .. "\n" .. firstTraceLine).
watcher:destroy()	Disconnects both captures and clears the seen table.

Options (all optional):

Field	Default	Description
WindowSeconds	30	Dedup window. One log line and one bus publish per unique error per window. Eviction horizon is WindowSeconds * 10.
SkipBind	false	Do not connect ScriptContext.Error (specs drive reports directly).

Bus topics

Topic	Payload	Fired
Kernel.ScriptError	(message, trace, sourceName, count)	On the first occurrence of a unique error, then at most once per window with the accumulated count. sourceName is the erroring script’s GetFullName(), "?" when unknown, or "KernelScheduler" for scheduler-caught task crashes.

ErrorWatch consumes Scheduler.OnError (a signal on the scheduler, not a bus topic) when attached with a kernel.

Design notes

Caution

Keep error messages stable. The dedup key is the message text — error(`no profile for {player.Name}`) mints a fresh key per player and defeats the dedup (each one logs on first sight; eviction merely bounds the memory). Put the variability in state you inspect later, keep the message constant: error("no profile for player") plus the traceback identifies the call site.

Note

ErrorWatch watches its own machine. ScriptContext.Error on a client only sees that client’s errors; attach one in the client Bootstrap too if you want client-side capture, and forward what you need through your own channel. Nothing crosses the network automatically — client error text is untrusted input like everything else clients send.

• attach requires the Logger from its own kernel clone (a relative require) — a cloned kernel gets its own ErrorWatch logging rather than silently sharing the original’s sinks.
• Both modules are fully spec-verified with injected clocks/heaps and SkipBind/SkipLoop doubles — the dedup windows, cycle bookkeeping, and bus payloads above are asserted behavior, not documentation optimism.