TestKit

TestKit is the framework’s spec runner. A ModuleScript whose name ends in .spec returns a function that registers describe and it blocks, and TestKit.runSpecs(container) runs every one under a container and returns a Results table. Specs run inside the place against the real modules, so the framework’s own suites are executable documentation of its behavior. Since 0.9.0 a test also owns its console output: Logger warnings raised inside a test are held and must match what the test declared with T.expectWarning, so a passing run prints nothing but its summary. The spec gate runs TestKit on every Studio Play, the regression runners run many of the same spec files offline, and Bench measures speed instead of correctness.

Mental model

TestKit.runSpecs(container) does this, in order:

1. Collects every ModuleScript descendant of container whose name ends in .spec and sorts them by name.
2. Starts one Watch for the whole run. It captures Logger output through Logger.capture and listens to ScriptContext.Error for errors whose message or traceback passes through container:GetFullName().
3. For each module after the first, settles deferred work left by the previous module (see Yielding and async work) and reports any output raised since the last test against that previous module.
4. Requires the module and calls its function with a context T holding describe, it, expect and expectWarning.
5. After the last module, settles once more, reports stray output, stops the watch and prints the summary.

Every it(name, body) adds one to Total, runs body under xpcall with debug.traceback, then asks the watch whether the test’s console output matched. A failure records the spec module name, the full path ("Outer > Inner > name", from the describe stack) and the error. describe(name, body) pushes name onto that stack and runs the body under xpcall.

Failures that do not come from an it still count, so a broken spec file fails the run instead of disappearing from it:

Test field	Cause
(require)	The module errored while loading.
(load)	The module returned something other than a function.
(top level)	The registration function threw outside any describe.
<path> (describe body)	A describe body threw while registering its tests.
(outside tests)	Logger warnings or uncaught spec errors raised between tests. They count against the module that ran last.

A passing run prints one line, [TestKit] PASS: <passed>/<total> tests across <specs> specs in <ms>ms, where <specs> counts spec modules. A failing run warns a [TestKit] FAIL: <failed>/<total> tests failed (<ms>ms) header and one FAIL [<spec>] <test> line per failure, with the error on the line below it. runSpecs returns:

export type Results = {
  Total: number,
  Passed: number,
  Failed: number,
  Failures: { Failure }, -- { Spec: string, Test: string, Error: string }
  DurationSeconds: number,
}

runSpecs never asserts on the results. The gate that stops a Studio boot is Gate, which reads Results.Failed. A script that wants to report and continue calls runSpecs and reads the table itself. If the run loop itself errors (outside the per-module protection), runSpecs stops the watch and re-raises.

Spec file anatomy

A spec is a ModuleScript named <Thing>.spec that returns a registration function. Where it lives decides where it runs:

Folder	Studio path	Runs
src/Shared/ChloeKernel/Tests	ReplicatedStorage.ChloeKernel.Tests	On the server at boot (the folder replicates, but only the server runs it).
src/Server/ChloeKernelServer/Tests	ServerScriptService.ChloeKernelServer.Tests	On the server at boot. Never reaches clients.
src/Server/Game/Tests	ServerScriptService.Game.Tests	On the server at boot, after the framework suites.
src/Client/Game/Tests	StarterPlayer.StarterPlayerScripts.Game.Tests	On each client before the client kernel boots.

python tools/scaffold.py service <Name> creates a starter spec in Game/Tests along with the service (see Scaffolding). This is TokenBucket.spec from the framework, complete:

src/Shared/ChloeKernel/Tests/TokenBucket.spec.luau

--!nonstrict
-- Specs use partial doubles, so nonstrict; the framework stays --!strict.
local TokenBucket = require(script.Parent.Parent.Net.TokenBucket)

return function(T)
  local describe, it, expect = T.describe, T.it, T.expect

  describe("TokenBucket", function()
    it("allows up to burst immediately, then blocks", function()
      local Bucket = TokenBucket.new(10, 3, 0)
      expect(Bucket:take(0)).toBeTruthy()
      expect(Bucket:take(0)).toBeTruthy()
      expect(Bucket:take(0)).toBeTruthy()
      expect(Bucket:take(0)).toBeFalsy()
    end)

    it("refills continuously at the configured rate", function()
      local Bucket = TokenBucket.new(10, 1, 0)
      expect(Bucket:take(0)).toBeTruthy()
      expect(Bucket:take(0.05)).toBeFalsy() -- 0.5 tokens accrued
      expect(Bucket:take(0.1)).toBeTruthy() -- 1.0 token accrued
    end)

    it("never accumulates past the burst cap", function()
      local Bucket = TokenBucket.new(100, 2, 0)
      expect(Bucket:take(60)).toBeTruthy() -- a minute idle still yields only 2
      expect(Bucket:take(60)).toBeTruthy()
      expect(Bucket:take(60)).toBeFalsy()
    end)

    it("defaults burst to the per-second rate", function()
      local Bucket = TokenBucket.new(5, nil, 0)
      for _ = 1, 5 do
        expect(Bucket:take(0)).toBeTruthy()
      end
      expect(Bucket:take(0)).toBeFalsy()
    end)
  end)
end

The spec never waits. TokenBucket.new(rate, burst, now) and bucket:take(now) take the current time as an argument, so the third test is a minute of idle time that costs nothing. That is the house style: make time, clocks and random seeds inputs, and the suite stays fast. Specs may open with --!nonstrict (directive CK001 allows it only in .spec files and under Tests, TestKit and Benches), because spec doubles are partial and strict mode would demand fully typed mocks.

Expected warnings and failure rules

Inside a test, every Logger line at Warn or Error level is held instead of reaching the Output window. When the test ends, TestKit compares what was held with what the test declared:

• T.expectWarning(fragment) declares one warning. fragment is a plain substring (no Lua patterns) of the full console line, scope prefix included, for example "[Wallet] ignored a deposit".
• Every held warning must contain at least one declared fragment. One fragment can cover several warnings.
• Every declared fragment must match at least one held warning. A warning that never came fails the test.
• An uncaught error reported through ScriptContext.Error whose message or traceback passes through the suite’s container fails the running test. That covers a spec’s Bus subscriber that throws on its own thread and a spec double run with task.spawn. Errors from unrelated scripts are ignored.
• Warnings and spec errors raised between tests (a late timer, a deferred callback) fail the module that ran last as (outside tests).
• Debug and Info lines are not held. They print as usual.

A mismatch fails the test with console output did not match the test: followed by one line per problem: unexpected warning: <line>, expected a warning containing "<fragment>" or uncaught error: <message>. If the body also threw, the problems are appended to its error. T.expectWarning outside an it raises expectWarning must be called inside it(), and an empty fragment is rejected.

The hold works only for Logger output. A raw warn() goes straight to the Output window and TestKit never sees it, which is why directive CK016 rejects raw warn() in framework modules and AGENTS.md asks game code to warn through Logger.scope("<Name>"). A game module that follows the rule, and a spec for it:

src/Server/Game/Services/EconomyService/Wallet.luau

--!strict
-- Wallet: coin balance math for the shop.
local ReplicatedStorage = game:GetService("ReplicatedStorage")

local Logger = require(ReplicatedStorage.ChloeKernel.Logger)

local Log = Logger.scope("Wallet")

local Wallet = {}

function Wallet.deposit(balance: number, amount: number): number
  if amount ~= amount or amount <= 0 then
    Log.warn(`ignored a deposit of {amount}`)
    return balance
  end
  return balance + amount
end

return Wallet

src/Server/Game/Tests/Wallet.spec.luau

--!nonstrict
local Wallet = require(script.Parent.Parent.Services.EconomyService.Wallet)

return function(T)
  local describe, it, expect = T.describe, T.it, T.expect

  describe("Wallet", function()
    it("adds a positive deposit", function()
      expect(Wallet.deposit(10, 5)).toBe(15)
    end)

    it("ignores a negative deposit and warns once", function()
      T.expectWarning("[Wallet] ignored a deposit of -5")
      expect(Wallet.deposit(10, -5)).toBe(10)
    end)
  end)
end

Remove the T.expectWarning line and the second test fails with unexpected warning: [Wallet] ignored a deposit of -5. Delete the warning from Wallet.deposit and it fails with expected a warning containing "[Wallet] ignored a deposit of -5".

The mechanism is Logger.capture(handler) -> release. While a capture is active, every line the console would write goes to handler(line, level) instead of the Output window. Captures nest: release() restores whichever capture was active before, and releasing twice does nothing. The capture sits on the console path, so Logger.setConsoleEnabled(false) inside a spec hides that spec’s warnings from TestKit as well.

Watch

TestKit/Watch.luau is the hold on its own, for harnesses that drive specs with their own T. runSpecs uses one per run, and every regression runner uses one with Forward set so its harness checks can still count the lines.

Member	Description
Watch.start(options?): Watch	Starts a capture. Options: ScriptContext and TracePath (both needed to track uncaught errors whose message or trace contains TracePath), Forward: (line) -> () to also pass held lines on.
watch:begin()	Starts a test. Lines from here on belong to it.
watch:expect(fragment)	Declares one expected warning for the running test. Errors outside a test.
watch:finish(): string?	Ends the test. Returns nil when its output matched, else the problem text.
watch:takeStray(): { string }	Returns and clears the lines raised while no test ran.
watch:stop()	Releases the capture and disconnects the error listener.

The matcher surface

T.expect(value) returns an expectation. This is the complete surface of TestKit/Expect.luau:

Matcher	Passes when
toBe(expected)	value == expected (raw equality, no coercion).
toEqual(expected)	Deep equality: every key compared in both directions, with a cycle guard.
toBeTruthy()	value is truthy.
toBeFalsy()	value is falsy.
toBeNil()	value == nil.
toBeType(name)	typeof(value) == name.
toBeCloseTo(expected, epsilon?)	value is a number within epsilon (default 1e-6) of expected.
toBeGreaterThan(expected)	value is a number > expected.
toBeLessThan(expected)	value is a number < expected.
toContain(expected)	A string contains the substring (plain find), or an array contains the value (table.find).
toHaveLength(expected)	#value == expected for strings and arrays. Anything else reads as length -1.
toThrow(pattern?)	value is a function that errors, and the error matches pattern when one is given.
.never	Negates any matcher: expect(x).never.toBe(y).

Failure messages print both sides. Tables preview their first 8 entries, and .never rewrites expected to expected NOT in the message. Three details matter in practice:

• toEqual treats a pair of tables already under comparison as equal instead of recursing. Kernel objects hold back-references (Process.Scheduler), and comparing them must terminate.
• toThrow(pattern) matches with string.find in Lua pattern mode. Escape magic characters (%-, %() when asserting on punctuation.
• toContain on strings uses a plain find, with no patterns.

Yielding and async work

it bodies run under xpcall, which tolerates yields: a body that calls task.wait stalls instead of erroring. The whole suite runs serially on the calling thread, and at Studio boot that thread is the server’s boot script, so every second a spec sleeps is a second added to Play. Results.DurationSeconds is wall-clock time and shows it.

Between modules TestKit yields through the deferred queue instead of waiting a frame. It queues a task.defer that resumes the runner and yields until it runs, twice, so whatever the previous module deferred, and whatever those handlers deferred in turn, runs before the next module starts. The yield exists because Roblox’s execution budget is per resumption: a long unyielded suite can exhaust it on a loaded host and kill the calling thread, which on the server takes the boot down with it. Waiting one frame per module did the same job but cost about 1.7 s over 104 modules. Where task.defer runs at once (the offline doubles), nothing is queued and nothing waits.

Property-changed signals are deferred under deferred signal behavior, so a spec that sets a property and checks the reaction needs one task.wait() in between, as DebugPanels.spec does. Keep those waits to one frame. For code driven by the Scheduler, call step() directly instead of waiting for real frames.

How suites run on Studio boot

In Studio, Main.server.luau builds a Gate and runs the shared suite, the server suite and then the game suite before the kernel boots. Any failure stops the server boot script: the kernel never starts, and the play session keeps running so the Output window stays readable. Main.client.luau runs the client game specs before the client kernel boots and stops that boot on a failure. Both framework suites execute in the server VM, and production servers never run specs because the whole block sits behind RunService:IsStudio(). The Spec gate page has the full sequence, the harness report and the audit and fuzz steps that follow the specs.

Writing specs for game services

Game services touch the kernel: hooks, the Bus, sessions, channels. The framework’s server specs use a partial kernel double: a real HookRegistry and a real Bus (pure Luau, cheap, and their real semantics are the point), with everything else faked. This is how NetDriver.spec.luau builds its world:

local ReplicatedStorage = game:GetService("ReplicatedStorage")

local HookRegistry = require(ReplicatedStorage.ChloeKernel.Hooks.HookRegistry)
local Bus = require(ReplicatedStorage.ChloeKernel.IPC.Bus)

local function makeFakeKernel()
  local FakePlayer = { Name = "TestPlayer" } :: any
  local Kernel = {
    Hooks = HookRegistry.new({ WarnOnError = false }),
    Bus = Bus.new(),
    Sessions = {},
  }
  function Kernel.getSession(self: any, player)
    return self.Sessions[player]
  end
  Kernel.Sessions[FakePlayer] = { Player = FakePlayer, Data = {} }
  return Kernel, FakePlayer
end

With that double a spec drives the real validation pipeline end to end. A test from the same file sends an intent through a rejecting hook chain:

it("rejected intents never reach the handler", function()
  local Driver, Kernel, Player, Created = makeDriver()
  local HandlerRan = false
  Driver:defineIntent("Buy", {}, {
    Handler = function()
      HandlerRan = true
    end,
  })
  Kernel.Hooks:on("Intent.Buy", function()
    return false
  end)
  local Rejected = nil
  Kernel.Bus:subscribe("Net.IntentRejected", function(_, _player, channel)
    Rejected = channel
  end)

  Created.CKI_Buy.SimulateServerEvent(Player)
  expect(HandlerRan).toBeFalsy()
  expect(Rejected).toBe("Buy")
  expect(Driver.Stats.IntentsRejected).toBe(1)
end)

makeDriver builds the driver with NetDriver.new(Kernel, { PacketFactory = Factory, SkipFloodWiring = true }). The PacketFactory seam swaps real remotes for fakes with a SimulateServerEvent method: no wire, no yielding, and hostile payloads on demand. WarnOnError = false keeps the registry quiet when specs feed validators input that throws on purpose.

Apply the same recipe to your own services: build the service against the double, fire its hook points and Bus topics directly, and assert on what can be observed (handler calls, Bus publishes, session data). The framework ships 107 spec modules. Registry.spec.luau, Session.spec.luau and Kits.spec.luau are good next reads.

API reference

Member	Description
TestKit.runSpecs(container: Instance): Results	Runs every *.spec ModuleScript under container, sorted by name, holding Logger output per test. Prints a summary and returns Results.
T.describe(name: string, body: () -> ())	Groups tests. Nests through a path stack. A body that throws counts as one failure.
T.it(name: string, body: () -> ())	One test, recorded under the full describe path.
T.expect(value: any)	Builds an expectation. See the matcher surface.
T.expectWarning(fragment: string)	Declares one Logger warning the running test must raise. Plain substring match.
Logger.capture(handler: ((line: string, level: number) -> ())?): () -> ()	Sends console lines to handler until the returned release runs. Nests. nil restores the Output window.
Results	{ Total, Passed, Failed, Failures = { { Spec, Test, Error } }, DurationSeconds }

Gotchas

Caution

Discovery goes by name suffix alone: any ModuleScript ending in .spec under the container runs. A WIP.spec parked in a subfolder runs on every boot until you rename it.

Caution

Specs share the VM and run in name order, so module-level state one spec mutates is visible to the next. NetTap.spec.luau diffs counters against a table.clone of the stats taken at the start of the test instead of assuming zeros. Do the same in specs against singletons, and restore shared flags in a pcall wrapper so a failed expectation cannot leak them.

Caution

Game specs under src/*/Game/Tests are game code to the directive linter, so CK005 flags any task.wait in them. Inject time instead, as TokenBucket does. The framework’s own specs are exempt.

Note

Specs never load made-up asset ids. Studio logs an error for every asset it cannot fetch, and a passing run should print nothing but its summaries. Register animations with AnimationClipProvider:RegisterAnimationClip the way AnimKit’s spec does. For the same reason, specs make no EditableMesh or EditableImage call unless ReplicatedStorage has the attribute ChloeKernelMeshApis = true: with “Allow Mesh & Image APIs” off, Studio logs an error for every EditableMesh call, even inside pcall. Set the attribute after turning the setting on, and MaterialKit’s live-engine specs run too.