Fuzz and security audit

Two tools check the security surface of a game’s network layer, and they split the work cleanly. kernel:securityAudit() reads configuration: every channel and every gate hook, looking for channels that reject everything, gates that pass on a crash, escape hatches and default rate limits. Fuzz.run exercises handling: it builds hostile payloads for every intent and request and drives them through the channel’s real validator chain. Both are one call, both return results a script can assert on, and both gate every Studio play test of the stock Main.server.luau. The fail-closed rules they check live in NetDriver and Hooks; channels are declared through the Registry.

Security audit

What one call sweeps

kernel:securityAudit(options?) audits every intent and request channel registered on the NetDriver and every hook point on the gate namespaces (Intent. and Request.). It reports four kinds of finding:

Kind	Severity	Fires when	Why it matters
UnguardedChannel	High	A channel has no validator on its Intent.<Name> or Request.<Name> topic and is not Open	The channel rejects every payload at runtime. Either a validator was forgotten or the channel should not exist.
FailOpenGate	High	A hook point named Intent.* or Request.* is defined FailOpen	On a fail-open point a crashing validator passes the payload. Gate namespaces must stay fail-closed; this catches a copied definePoint that flipped the default.
OpenChannel	Medium	A channel is Open = true and has a handler	Open is the deliberate escape hatch: any client payload reaches the handler unvalidated. Sometimes right (a no-stakes ReadyUp), but each one must be confirmed safe against hostile input.
DefaultRateLimit	Info	A channel has a handler and no explicit RateLimit	The channel rides the default 30 per second rate limit. Set it to what the gameplay needs; a fire intent needs about 8.

UnguardedChannel covers every channel without a validator, with or without a handler. A game can consume an intent only through its Intent.<Name> Bus topic, with no handler at all, and before 0.9.0 such a channel was accepted and its payload reached Bus subscribers unvalidated. The fail-closed rule now applies to it too: NetDriver rejects the payload, warns once for the channel, and the audit flags it. A Registry channel declared with Validate = {} counts as unguarded too: Registry.server installs no validator for an empty table (0.9.1; before, it installed one that passed everything), and directive CK012 flags the declaration.

The same rule runs at boot without the audit. ServerKernel:start() calls auditValidators() and warns once for each unguarded channel, naming its kind and name.

Finding shape and ordering

Every finding is { Severity, Kind, Name, Detail }. Severity is "High", "Medium" or "Info". Name identifies the subject: intent Grant, request Buy, hook Intent.Buy. Detail is one line that restates the problem and the fix. Findings are sorted by severity, High first, then alphabetically by Name, so a ship gate only has to look at the first one: Findings[1] == nil or Findings[1].Severity ~= "High".

Fail-open points outside the gate namespaces are never flagged. Observation points such as Net.RateLimited or AntiExploit.MovementViolation are fail-open on purpose, so that a crashing listener cannot silence the others.

Running it

local Findings = kernel:securityAudit({ Silent = true })
for _, Finding in Findings do
  assert(Finding.Severity ~= "High", `{Finding.Kind} {Finding.Name}: {Finding.Detail}`)
end

Without Silent it also prints [ChloeKernelServer] security audit: clean, or one warning that lists every finding as [Severity] Kind (Name): Detail:

[ChloeKernelServer] security audit: 2 finding(s)
  [High] UnguardedChannel (intent Grant): has no validator and is not Open: every payload rejects; add a validator or audit Open
  [Info] DefaultRateLimit (intent Chat): rides the default rate limit (30/s): set RateLimit to what the gameplay actually needs

Run it after start(). Services register their channels between boot and start, and the audit reads configuration, so an audit before registration finishes audits nothing. A game that never built a NetDriver (kernel:net() was never called) still gets the hook sweep: the FailOpenGate check runs without networking. net:auditValidators() is the narrow version, returning only the unguarded channels as { Name, Kind } pairs.

Fuzzing

The audit checks configuration; the fuzzer checks behavior. Fuzz.run builds hostile argument sets for every registered intent and request and fires each one through the channel’s own validator chain, with the same context live traffic gets.

Hostile payload generation

Each schema entry maps to a pool by type-name prefix, so every sized wire variant (NumberU8, StringLong, Vector3F24, Boolean8) draws from the right pool:

Schema prefix	Pool contents
Number*	0, 1, -1, 255, 256, 65535, 65536, 2^31, -2^31, 2^53, 0.5, -0.5, NaN, math.huge, -math.huge
String*	"", "a", a 10,000-character string, "-1", "0", "%s%d%n" (format specifiers), "\0\1\255" (control bytes), "nil", "true", 100 repeated UTF-8 byte-order marks
Vector3*	Vector3.zero, Vector3.one, (1e6, -1e6, 1e6), a NaN component, math.huge and -math.huge components, (-0.001, 0.001, 0)
Boolean*	true, false
Anything else, including Any	nil, "", a 10 KB string, NaN, math.huge, -1, {}, a deeply nested table, a mixed array and hash table, true, a vector with a NaN component

Many of these values cannot arrive from a real client. The wire types bound live traffic, and a NumberU8 slot never carries NaN off the wire. The fuzzer sends past those bounds on purpose, as defense in depth: validators should hold even if a transport guarantee changes, and Any slots never had one.

Each case draws one pool value per schema slot from a Random seeded with Seed, CasesPerChannel times per channel, so a run is deterministic per seed and a failing case reproduces exactly.

Validation only by default

Fuzz.run runs validators only. Handlers run only with ExecuteHandlers = true, which belongs in a test with its own kernel and session: a payload that passes validation makes the handler do its real work, such as selling the item or writing the profile. Validators are game code too, and validation-only mode is not a sandbox. A validator that writes state writes it during a fuzz run as well.

The pipeline, and what counts as a failure

For each case the fuzzer:

1. Builds the context live traffic gets: { Name, Player, Session, Args, Seq }. For a predicted intent, the first argument is the sequence number: it moves to Seq and out of Args, as NetDriver does.
2. Counts the case Rejected without firing when the channel has no validator and is not Open, or when the session’s profile is inside a trade (TransactionOwner set or TransactionBlocked). NetDriver refuses both the same way.
3. Fires the channel’s Intent.<Name> or Request.<Name> chain. When a validator throws, HookRegistry.fire returns the error’s traceback as a third value and the case lands in ValidatorErrors as { Channel, Kind, Args, Error }. The chain still counts as a rejection, because the gate is fail-closed.
4. Counts Rejected when the chain rejected, when a request channel has no handler, or when the profile entered a trade while the validators ran.
5. Counts Passed otherwise. An intent with no handler (consumed through its Bus topic) passes when its validators do.
6. With ExecuteHandlers = true and a handler, runs the handler under pcall with the validated arguments. Validators that rewrite context.Args are honored, and nil holes survive by position. A crash lands in HandlerErrors and every run adds to HandlersExecuted.

Cases == Passed + Rejected always. ValidatorErrors must be empty: a validator has to reject hostile input, and one that throws is a bug even though fail-closed caught it. With handlers on, HandlerErrors must be empty too. A payload that passed validation and crashed the handler means the validator promises a domain it does not enforce, or the handler assumes more than the validator promises. Tighten the validator first, so handlers only ever see the domain it guarantees.

The Studio gate

The stock Main.server.luau runs both tools through TestKit’s Gate on every Studio play test (see Spec gate). After the kernel boots, the Audit step asserts that no High finding exists. On the first session of a Play test the Fuzz step runs validators only, because handlers would write to the tester’s own profile. Run mode has no players, so the step is not expected there.

src/Server/Main.server.luau

if not RunService:IsRunMode() then
  Gate:expect("Fuzz")
  local Fuzz = require(script.Parent.ChloeKernelServer.Fuzz)
  local Fuzzed = false
  Booted:onSession(function(session: any)
    if Fuzzed then
      return
    end
    Fuzzed = true
    task.defer(function()
      Gate:run("Fuzz", function()
        local Report = Fuzz.run(Booted, { Session = session, Silent = true })
        if #Report.ValidatorErrors > 0 then
          for _, Failure in Report.ValidatorErrors do
            warn(`[Fuzz] {Failure.Kind} "{Failure.Channel}" validator error: {Failure.Error}`)
          end
          error(`[Fuzz] {#Report.ValidatorErrors} validator error(s): see warnings above`)
        end
        print(`[Fuzz] {Report.Channels} channels, {Report.Cases} cases, validators only, 0 validator errors`)
      end)
    end)
  end)
end

A validator that crashes on hostile input fails the gate, and a plugin-started harness session ends with the failure in its report. Register a channel with a validator that indexes an unchecked argument and the Play test fails before you reach the test dummy.

Until 0.9.0 this gate could not fail. It ran validators only (handler execution became opt-in in 0.8.0) and reported handler errors, a list that stays empty without handler execution. Validator crashes counted as rejections and showed up only as warnings. The third return value of HookRegistry.fire and ValidatorErrors exist so the gate has something real to fail on.

Note

The Simulation debug intent SimNoise used to have no validator, so the Studio audit rated it High on every Play. It now validates with NetTap.debugAllowed, and the audit no longer flags it.

Running it yourself

Fuzz.run(kernel, options) needs a session, because validators and handlers read session.Player, profiles and character state. For a validators-only sweep in Studio, pass a real session. To run handlers, build an isolated kernel and a fake registry through the Driver option, the way Fuzz.spec.luau does:

src/Server/Game/Tests/BuyFuzz.spec.luau

--!nonstrict
local ReplicatedStorage = game:GetService("ReplicatedStorage")
local ServerScriptService = game:GetService("ServerScriptService")

local Bus = require(ReplicatedStorage.ChloeKernel.IPC.Bus)
local Fuzz = require(ServerScriptService.ChloeKernelServer.Fuzz)
local HookRegistry = require(ReplicatedStorage.ChloeKernel.Hooks.HookRegistry)

-- In a game, require the validator the service registers on Intent.Buy
local function validateBuy(context)
  local ItemId = context.Args[1]
  return type(ItemId) == "number" and ItemId == ItemId and ItemId % 1 == 0 and ItemId >= 1 and ItemId <= 500
end

return function(T)
  local describe, it, expect = T.describe, T.it, T.expect

  describe("Buy intent", function()
    it("rejects hostile payloads and never crashes the handler", function()
      -- WarnOnError = false: a crashing validator lands in ValidatorErrors, not in the Logger
      local Kernel = { Hooks = HookRegistry.new({ WarnOnError = false }), Bus = Bus.new() }
      Kernel.Hooks:on("Intent.Buy", validateBuy)
      local Bought = 0
      local Driver = {
        Intents = {
          Buy = {
            Schema = { "NumberU16" },
            IntentTopic = "Intent.Buy",
            Handler = function(_session, itemId)
              assert(itemId >= 1 and itemId <= 500, "handler precondition")
              Bought += 1
            end,
          },
        },
        Requests = {},
      }
      local Report = Fuzz.run(Kernel, {
        Session = { Player = {}, Data = {} },
        Driver = Driver,
        CasesPerChannel = 64,
        Seed = 3,
        Silent = true,
        ExecuteHandlers = true, -- safe: this kernel and session exist only in the test
      })
      expect(#Report.ValidatorErrors).toBe(0)
      expect(#Report.HandlerErrors).toBe(0)
      expect(Report.HandlersExecuted).toBe(Bought)
    end)
  end)
end

Unless Silent = true, each run prints one summary line (channels, cases, rejected, passed, validator errors, handlers executed, handler errors) and logs a warning per validator or handler error.

Interpreting a failure

A ValidatorErrors or HandlerErrors entry gives the channel, the kind ("Intent" or "Request"), the exact argument list and the error (a traceback for validators). Reproduce it with the same Seed and Channels = { "TheChannel" }.

For a validator error, the validator used an argument before checking its type: string.len on an Any slot that held a table, arithmetic on nil, indexing a number. Check the type first and return false.

For a handler error, decide which side is wrong:

• The handler asserts value >= 0 and the validator never checks the sign: add the bound to the validator.
• The validator checks the type and the handler indexes a nested field: fix the handler, or stop using Any for that slot.
• The handler reads session.Player.Character and the session is fabricated: that is a test artifact, not a finding. Use a real session or a fake with the fields the handler reads.

API

Security audit

Member	Description
kernel:securityAudit(options?)	One sweep over channels and gate hook points. options is { Silent: boolean? }. Prints a summary unless Silent. Returns { AuditFinding } sorted High, Medium, Info, then by name.
net:auditValidators()	Unguarded channels only, as { { Name, Kind } } with Kind "Intent" or "Request". ServerKernel:start() warns for each one.

AuditFinding is { Severity: "High" | "Medium" | "Info", Kind: string, Name: string, Detail: string }.

Fuzz

Member	Description
Fuzz.run(kernel, options)	Fuzzes every intent and request (or the allowlisted ones) and returns a Report. Errors without options.Session.

Option	Default	Description
Session	required	The session validators and handlers see. Use a real one, or a disposable fake with ExecuteHandlers.
Channels	every channel	Allowlist of channel names.
CasesPerChannel	32	Argument sets per channel.
Seed	1	Random seed. Runs are deterministic per seed.
IncludeRequests	true	false fuzzes intents only.
Driver	kernel:net()	A registry { Intents, Requests } to fuzz instead of the kernel’s NetDriver.
Silent	false	Suppresses the summary line and the per-error warnings.
ExecuteHandlers	false	Runs handlers for cases that pass. Use an isolated test kernel and session.

Report field	Description
Channels	Channels fuzzed.
Cases	Total cases. Always Passed + Rejected.
Passed	Cases the validator chain let through.
Rejected	Cases refused: no validator and not Open, a validator returned false or threw, a request without a handler, or a profile inside a trade.
HandlersExecuted	Handler runs. 0 unless ExecuteHandlers.
ValidationOnly	true unless ExecuteHandlers was set.
ValidatorErrors	{ { Channel, Kind, Args, Error } } for validators that threw. Must be empty.
HandlerErrors	{ { Channel, Kind, Args, Error } } for handlers that threw. Only filled with ExecuteHandlers.

Design notes

Caution

ExecuteHandlers = true calls real handlers with real side effects. A Buy handler behind a validator that passes a hostile but valid payload sells the item to the fuzz session. Run it only against a kernel and session built for the test, never against live players. A clean fuzz of an economy channel shows the validator holds against hostile input; deciding what a valid purchase is stays the game’s job.

Note

The fuzzer starts at the hook chain and skips the transport: wire deserialization, the session gate and the per-channel rate limits are not exercised (a 30 per second limit would stop 32 cases per channel anyway). Those layers bound what payloads arrive. The fuzzer tests whether validators hold for payloads beyond those bounds, and the transport has its own specs.

Note

UnguardedChannel findings and the fuzzer’s Rejected count describe the same rule from two sides. A channel with no validator rejects everything at runtime, the audit flags it High, and the fuzzer counts its cases as rejections without firing them. See Architecture and security.

Status

Both tools run fully in Studio, with no live-server caveat. Fuzz.spec.luau covers tight validators rejecting the pools and leaky ones leaking to HandlerErrors, handlers staying off by default, the missing-validator gate holding with handlers on, predicted sequence numbers kept out of the arguments, crashing validators reported in ValidatorErrors without running handlers, and determinism per seed with channel filtering. NetDriver.spec.luau covers every audit finding kind and auditValidators. Both run in the Studio spec gate and in the network regression runner, and the core runner checks that the harness gate reports an unfinished Fuzz step as skipped when it times out.