TestKit & Bench

TestKit is the framework’s built-in spec runner: *.spec ModuleScripts register describe/it blocks, every Studio play-test boot runs them, and runSpecs() returns a Results table so a CI-style gate can assert on the run instead of eyeballing output. Bench is its sibling — a microbenchmark harness over *.bench modules that reports throughput plus p50/p99 per-operation latency, with a warmup phase that keeps cold-cache effects out of the numbers. The defining decision: both live inside the place and run against the real modules, so the specs in Tests/ double as executable documentation of every kernel behavior claim.

Mental model

TestKit.runSpecs(container) walks the container’s descendants and collects every ModuleScript whose name ends in .spec, sorted alphabetically. Each spec module must return a function(T); the runner calls it with a context exposing T.describe, T.it, and T.expect.

Everything is bookkeeping around xpcall:

• it(name, body) increments Total, runs body under xpcall with debug.traceback, and records a pass or a failure. A failure stores the spec module name, the full path ("Outer > Inner > test name" from the describe stack), and the error with traceback.
• describe(name, body) pushes name onto a path stack and runs the body — nesting is just the stack, joined with " > ". A describe body that throws is itself counted as one failure ("... (describe body)"), so a crash while registering tests can never silently skip them.
• A spec that fails to require, or returns something other than a function, is recorded as a failure too ("(require)" / "(load)"). Broken spec files fail the run; they do not vanish from it.

The runner prints one summary line — [TestKit] PASS — 259/259 tests across 23 specs in 41.3ms — or warns per failure, and returns:

export type Results = {
  Total: number,
  Passed: number,
  Failed: number,
  Failures: { Failure }, -- { Spec, Test, Error }
  DurationSeconds: number,
}

Nothing asserts for you. If you want a hard gate, assert on Results yourself (see the boot gate).

Spec file anatomy

A spec is a ModuleScript named <Thing>.spec that returns a registration function. Shared specs live in ChloeKernel/Tests (they replicate); server-only specs live in ChloeKernelServer/Tests and never reach clients:

-- TokenBucket.spec — a real spec from ChloeKernel/Tests (excerpted)
local TokenBucket = require(script.Parent.Parent.Net.TokenBucket)

return function(T)
  local describe, it, expect = T.describe, T.it, T.expect

  describe("TokenBucket", function()
    it("allows up to burst immediately, then blocks", function()
      local Bucket = TokenBucket.new(10, 3, 0)
      expect(Bucket:take(0)).toBeTruthy()
      expect(Bucket:take(0)).toBeTruthy()
      expect(Bucket:take(0)).toBeTruthy()
      expect(Bucket:take(0)).toBeFalsy()
    end)

    it("refills continuously at the configured rate", function()
      local Bucket = TokenBucket.new(10, 1, 0)
      expect(Bucket:take(0)).toBeTruthy()
      expect(Bucket:take(0.05)).toBeFalsy() -- 0.5 tokens accrued
      expect(Bucket:take(0.1)).toBeTruthy() -- 1.0 token accrued
    end)

    it("never accumulates past the burst cap", function()
      local Bucket = TokenBucket.new(100, 2, 0)
      expect(Bucket:take(60)).toBeTruthy() -- a minute idle still yields only 2
      expect(Bucket:take(60)).toBeTruthy()
      expect(Bucket:take(60)).toBeFalsy()
    end)
  end)
end

Note what this spec does not do: it never calls task.wait. TokenBucket:take(dt) accepts elapsed time as a parameter, so the spec injects a minute of idle time in zero real milliseconds. That is the house style — design time as an input, and the suite stays fast.

The framework’s own specs open with --!nonstrict and a one-line reason: spec doubles are partial, and strict mode would demand fully typed mocks. The framework itself stays --!strict.

The matcher surface

T.expect(value) returns an expectation object. This is the complete surface of TestKit/Expect.luau:

Matcher	Passes when
toBe(expected)	value == expected (raw equality, no coercion)
toEqual(expected)	deep equality — key-wise both directions, with a cycle guard
toBeTruthy()	value is truthy
toBeFalsy()	value is falsy
toBeNil()	value == nil
toBeType(name)	typeof(value) == name
toBeCloseTo(expected, epsilon?)	number within epsilon (default 1e-6) of expected
toBeGreaterThan(expected)	number > expected
toBeLessThan(expected)	number < expected
toContain(expected)	string contains the substring (plain find), or array contains the value (table.find)
toHaveLength(expected)	#value == expected for strings and arrays (anything else reads as -1)
toThrow(pattern?)	value must be a function; it errors, and the error matches pattern if given
.never	negates any matcher: expect(x).never.toBe(y)

Failure messages stringify both sides (tables preview up to 8 entries); .never rewrites expected to expected NOT in the message. Details worth knowing:

• toEqual guards cycles: a pair of tables already being compared counts as equal instead of recursing forever. Kernel objects hold back-references (Process.Scheduler), and deep-comparing them must terminate.
• toThrow(pattern) matches with string.find in Lua pattern mode — escape magic characters (%-, %( …) when asserting on punctuation.
• toContain on strings uses a plain find — no patterns there.

Yielding and async work

it bodies run under xpcall, which in Luau tolerates yields — a body that calls task.wait does not error, it stalls. The whole suite runs serially on the calling thread, and on Studio boot that thread is the server’s boot path, so every second a spec sleeps is a second added to your play-test startup. Results.DurationSeconds is wall clock and will tell on you.

The framework’s answer is the TokenBucket pattern above: pass elapsed time, timestamps, and random seeds as parameters so behavior is testable without waiting. For code driven by the Scheduler, call step() directly in the spec rather than waiting for real frames.

How suites run on Studio boot

Main.server.luau runs both containers before the kernel boots, in Studio only:

if RunService:IsStudio() then
  local TestKit = require(ReplicatedStorage.ChloeKernel.TestKit)
  TestKit.runSpecs(ReplicatedStorage.ChloeKernel.Tests)
  TestKit.runSpecs(script.Parent.ChloeKernelServer.Tests)
end

Both containers execute in the server VM — shared specs live in ReplicatedStorage but run at server boot alongside the server-only ones. Production servers never run specs; the gate is RunService:IsStudio().

A failing suite warns loudly but does not halt boot. To make it a hard gate, assert on the return value:

local Results = TestKit.runSpecs(ReplicatedStorage.ChloeKernel.Tests)
assert(Results.Failed == 0, `{Results.Failed} spec failure(s) — fix before shipping`)

The same boot window also runs the security audit and channel fuzzer in Studio — specs check your logic, the audit checks your configuration, the fuzzer checks your input handling.

Writing specs for game services

Game services touch the kernel — hooks, bus, sessions, net channels. The pattern the framework’s own server specs use is a partial kernel double: real HookRegistry and Bus (they are pure Luau, cheap, and you want their real semantics), fake everything else. This is how NetDriver.spec.luau builds its world:

local ReplicatedStorage = game:GetService("ReplicatedStorage")

local HookRegistry = require(ReplicatedStorage.ChloeKernel.Hooks.HookRegistry)
local Bus = require(ReplicatedStorage.ChloeKernel.IPC.Bus)

local function makeFakeKernel()
  local FakePlayer = { Name = "TestPlayer" } :: any
  local Kernel = {
    Hooks = HookRegistry.new({ WarnOnError = false }),
    Bus = Bus.new(),
    Sessions = {},
  }
  function Kernel.getSession(self: any, player)
    return self.Sessions[player]
  end
  Kernel.Sessions[FakePlayer] = { Player = FakePlayer, Data = {} }
  return Kernel, FakePlayer
end

With that double, a spec drives the real validation pipeline end to end — here is a real test from the same file, exercising an intent through its fail-closed hook chain:

it("rejected intents never reach the handler", function()
  local Driver, Kernel, Player, Created = makeDriver()
  local HandlerRan = false
  Driver:defineIntent("Buy", {}, {
    Handler = function()
      HandlerRan = true
    end,
  })
  Kernel.Hooks:on("Intent.Buy", function()
    return false
  end)
  local Rejected = nil
  Kernel.Bus:subscribe("Net.IntentRejected", function(_, _player, channel)
    Rejected = channel
  end)

  Created.CKI_Buy.SimulateServerEvent(Player)
  expect(HandlerRan).toBeFalsy()
  expect(Rejected).toBe("Buy")
  expect(Driver.Stats.IntentsRejected).toBe(1)
end)

WarnOnError = false on the HookRegistry matters: specs deliberately feed validators inputs that throw, and the registry would otherwise warn on every one. The PacketFactory seam (NetDriver.new(Kernel, { PacketFactory = Factory, SkipFloodWiring = true })) swaps real remotes for fakes with a SimulateServerEvent method — no wire, no yielding, hostile payloads on demand.

Apply the same recipe to your own services: construct the service against the double, fire its hook points and bus topics directly, and assert on the observable outcomes — handler calls, bus publishes, session data mutations. The Tests/ directories hold thirty-plus worked examples; Registry.spec.luau, Session.spec.luau, and Kits.spec.luau are good next reads.

Bench

Bench answers one question per line: how many operations per second, and what does one operation cost at the median and the tail.

Timing methodology

Bench.run(name, fn, options?):

1. Runs Setup if provided, then a warmup batch of 1,000 calls — cold-cache and cold-path effects stay out of the numbers.
2. Runs timed batches of 1,000 calls until the deadline (DurationSeconds, default 0.25). Each batch records (batch elapsed) / 1000 as its per-op latency.
3. Sorts the batch timings: p50 and p99 are percentiles over batch-averaged per-op latency, and OpsPerSecond is total ops over total elapsed.

fn receives the index within the batch (1–1000) — use it to vary inputs (Items:add(index % 50, 1)) so you are not benchmarking a warm branch on one constant.

Note

Because p99 is the 99th percentile of batch means, it captures sustained slow batches (GC pressure, contention), not one-in-a-million single-op spikes. It is built for comparing implementations, not for latency SLOs.

Write your own bench

A bench module is a ModuleScript named <Thing>.bench that returns function(bench). The framework’s own suite lives in ChloeKernel/Benches/Kernel.bench.luau; a minimal one of yours:

-- Inventory.bench — ModuleScript under ReplicatedStorage.ChloeKernel.Benches
local Inventory = require(game:GetService("ReplicatedStorage").Game.Inventory)

return function(bench)
  local Items = Inventory.new()

  bench("Inventory:add", function(index)
    Items:add(index % 50, 1)
  end)

  bench("Inventory:serialize", function()
    Items:serialize()
  end, { DurationSeconds = 0.5 })
end

Benches do not run at boot. Run them on demand — the command bar during a play test works:

local Bench = require(game.ReplicatedStorage.ChloeKernel.TestKit.Bench)
Bench.runAll(game.ReplicatedStorage.ChloeKernel.Benches)

runAll collects every *.bench descendant, sorted by name, and yields one frame between benches so consecutive benchmarks do not contend with each other’s deferred work.

Reading the output

[Bench] running 1 benchmark modules...
[Bench] Signal:fire (1 handler)                  12.41M ops/s  p50/op      78ns  p99/op     121ns
[Bench] Serde schema encode (3 fields)            1.02M ops/s  p50/op     942ns  p99/op    1.31µs

• ops/s — throughput over the whole timed window; the headline number for “can I afford this in a loop”.
• p50/op — the typical cost of one call. Multiply by your per-frame call count to get a frame cost.
• p99/op — the sustained-tail cost. A p99 far above p50 means the operation allocates (GC assists land in some batches) or hits a slow path intermittently — worth knowing before it runs 500 times a frame.

The framework’s published numbers on Benchmarks were measured with exactly this harness, so your results are directly comparable on your hardware.

API reference

TestKit

Member	Description
TestKit.runSpecs(container: Instance): Results	Runs every *.spec ModuleScript descendant (sorted by name); prints a summary; returns Results.
T.describe(name: string, body: () -> ())	Groups tests; nests via an internal path stack. A throwing body counts as one failure.
T.it(name: string, body: () -> ())	One test. Pass/fail recorded under the full describe path.
T.expect(value: any)	Builds an expectation — see the matcher surface.
Results	{ Total, Passed, Failed, Failures = { { Spec, Test, Error } }, DurationSeconds }

Bench

Member	Description
Bench.run(name: string, fn: (index: number) -> (), options?: { DurationSeconds: number?, Setup: (() -> ())? }): Result	One benchmark: warmup batch, then timed 1,000-call batches until the deadline (default 0.25s).
Bench.runAll(container: Instance): { Result }	Runs every *.bench module descendant (sorted by name), printing one aligned row per bench, one frame apart.
Result	{ Name, OpsPerSecond, P50Nanos, P99Nanos, TotalOps }

Gotchas

Caution

Spec discovery is by name suffix, not location: any ModuleScript ending in .spec under the container runs. A WIP.spec you parked in a subfolder runs on every boot until you rename it.

Caution

Specs share the VM and run in alphabetical order. Module-level state mutated by one spec is visible to the next — NetTap.spec.luau diffs counters against a table.clone of the stats taken at test start rather than assuming zeros, and your specs against singletons should do the same.

Note

The boot run does not halt on failure — the summary warns and boot continues, which keeps a broken spec from bricking your play test. Gate on Results.Failed explicitly where you want enforcement.