Lock handoff

Lock handoff is the part of DataDriver that lets a player rejoin seconds after leaving. A session lock keeps a profile from loading on two servers at once, but a lock that only expires by age makes the next server wait out the old one. Before 0.9.0 a quick rejoin was kicked: on the same server while the leave save was failing, on another server while the old one still had a stale connection for the player, and for 3 to 4 minutes after a server crash. Handoff removes those waits without weakening the lock. Its defining rule: a lock changes hands when its holder releases it or has provably stopped, and never by age while the holder is still alive. It runs over MessageDriver for requests and MemoryDriver for presence.

How it works

Every server with handoff on does two things. It subscribes to its own MessagingService topic, CKHandoff_<JobId>, and it beats a presence key (its JobId) in the MemoryStore hash map CKPresence every 3 s with a 10 s expiry. Locks written by such a server carry Presence = true.

When a player joins and the load finds another server’s lock, the joining server asks that server to release it, and reads its presence to tell a live holder from a crashed one:

Case	What happens	Wait
Same server, leave save still running	The load waits for that save, then loads the saved record	About one save
Same server, leave save failed	The player carries on with the retained profile, which is the newest copy and still under this server’s lock	None
Another server, player still connected there	The holder ends its stale session, whose leave save releases the lock, and replies	About one save
Another server, leave save failed there	The holder retries the release at once and replies when it lands	About one save
Another server crashed	Its presence expires and the lock is taken after two empty reads a beat apart	About 10 to 15 s
Another server alive, saves stalling	It keeps its lock, even once the lock is stale by age; the join fails closed after LockWaitSeconds	Kick
Holder on an older build, or with Handoff = false	No request is answered and the lock has no Presence flag, so it changes hands only by release or by age	Until the holder releases; after a crash, until the lock is LockTtlSeconds + LockSkewSeconds old

Same-server rejoins

load() waits while its key is still loading on this server or a leaving profile (one with a release requested) is still saving. The wait is event-driven: the load wakes when the reservation clears, the leaving profile deactivates or its save finishes. Player sessions wait up to LocalHandoffSeconds, or LockWaitSeconds (default 15) when that is unset. A manual load() waits options.HandoffSeconds, then LocalHandoffSeconds, then 0.

When the wait ends, a retained profile is adopted if its leave save failed, no save is running and no trade holds it. Adoption clears the release request and saves at once to re-assert the lock:

• The save lands: the player continues on the same Profile object, with everything earned before the leave.
• The save finds the lock gone (another server took it): the profile deactivates and the driver loads the record normally.
• The save fails for another reason (the backend is still down): the player keeps playing on the retained profile and the driver warns. Autosaves keep retrying.

A leave save that committed but reported failure (a lost response) leaves the record unlocked with Meta.ReleasedToken set to this profile’s token. The adoption save recognizes its own release, sees that nobody has written since, and retakes the lock instead of treating the missing lock as lost.

Cross-server requests

1. The joining server’s load fails with profile is locked by another server. The driver records the holder’s JobId and lock Token.
2. It publishes a request to CKHandoff_<holder JobId> and repeats it every 4 s while it waits.
3. The holder routes the request to the DataDriver with the same Name. If that driver no longer holds the key, or holds it under a different token (the player came back to it and holds a newer lock), it replies at once. Otherwise it records the requester and acts: a profile whose leave save failed runs its release now, a live session is ended with the kick message “You joined another server.” (which runs the leave cleanups and the release save), and a load still in flight is answered when it settles.
4. When the profile deactivates, the holder sends a reply to every recorded requester.
5. The reply wakes the joining server, which loads again at once and finds the lock gone.

Between attempts the joining server parks on the reply for 0.5 s, doubling to at most 2 s, and loads again after each wake. The whole wait is bounded by LockWaitSeconds. After that the load stays fail-closed: the player is kicked and the record is untouched.

The token in the request is what makes a late message harmless. A request delivered after the player returned to the holder names a lock that no longer exists there, so the holder replies without ending the player’s newer session.

Presence and crashed servers

Each beat is Presence:set(JobId, 1, 10) on a Background-priority Scheduler job every 3 s, plus one beat when handoff starts. A server that misses three beats reads as stopped. A presence read returns one of three answers: alive (a beat within 10 s), stopped (a successful read found no key) or unknown (the read failed).

While a join waits, it reads the holder’s presence after each wake. Two reads at least one beat (3 s) apart that both find no key mark the holder as stopped. The next load passes the holder’s JobId as StealFrom, and the transform takes a lock from that holder only if the lock carries Presence = true. The driver warns took the lock of <key> from server <JobId>, which stopped beating presence. The record that loads is the crashed server’s last save; progress after it was lost with the crash. Presence expires at most 10 s after the last beat, the silence must last 3 s, and the retry spacing adds up to 2 s, so the takeover lands about 10 to 15 s after the crash.

A crash can leave a trade between its prepare and settle writes. The load that takes the lock settles it from the journal before the profile loads (transaction recovery), and recovery applies the same ownership rule as the load, so the participant arrives in the same 10 to 15 s. Before 0.9.1 recovery went by lock age alone, and that player was kicked until the crashed server’s lock was 3 minutes old.

Presence also protects live servers. A lock older than LockTtlSeconds + LockSkewSeconds is normally stale, but when it carries Presence = true and this server has handoff on, the load first reads the holder’s presence. A holder that still beats keeps its lock. Before 0.9.0 a live server whose saves stalled past the TTL could lose its lock to a joining server, which loaded the older record and rolled the player back. An unknown answer (the presence read failed) does not protect the lock: it is taken by age as before.

Mixed-version deploys

Only a server with handoff on writes Presence = true. A lock without the flag, written by an older build or a driver with Handoff = false, is never taken through presence and never protected by it; it changes hands by release or by age, as before 0.9.0. Older servers do not subscribe to CKHandoff_ topics, so a request to one goes unanswered and the joining server waits out LockWaitSeconds. New and old servers can run side by side during a deploy without either one taking a live lock early.

Try it

Four ways a player can arrive on server B while server A still holds their lock. The crash case plays four times faster; its timings follow the real constants (presence beats every 3 s with a 10 s TTL, two missing reads at least 3 s apart before a takeover).

When it is on

attach turns handoff on when all of these hold:

• Config.Handoff is not false.
• Config.JobId is not set (tests set it to simulate servers).
• game.JobId is not empty.
• The backend is not the Memory backend.
• The server is not Studio.

Every DataDriver on a server shares one handoff (Handoff.shared(game.JobId)): one subscription and one presence beat per server, whatever the number of drivers. Passing a handoff instance as Config.Handoff uses that instance instead; the regression runner injects doubles this way.

With handoff off, a joining player still waits up to LockWaitSeconds, retrying the load with the same 0.5 s to 2 s spacing, and locks change hands only by release or by age.

Cost per server: one MessagingService subscription and one MemoryStore write every 3 s. Messages and presence reads happen only while a join is waiting on another server’s lock. MessageDriver applies MessagingService’s own limits to the requests.

Usage

There is nothing to call. The defaults suit most games; tune the waits in the DataDriver config:

local PlayerData = DataDriver.new({
  Name = "PlayerData",
  Defaults = { Coins = 0 },
  -- A joining player waits this long for another server's lock before the load fails
  LockWaitSeconds = 20,
  -- and this long for their own leave save on this server
  LocalHandoffSeconds = 10,
})
PlayerData:attach(Kernel)

Handoff = false turns it off, for example on a server type that never holds player profiles.

Configuration

Field	Default	Description
Handoff	on for live servers	false turns handoff off. A handoff instance is used as given. Anything else keeps the default rule in When it is on.
LockWaitSeconds	15	How long a joining player waits for another server’s lock. Also the same-server wait when LocalHandoffSeconds is unset.
LocalHandoffSeconds	nil	How long a load waits for a load or release of the same key on this server.
LockTtlSeconds	180	Lock age after which a lock without presence protection can be taken.
LockSkewSeconds	60	Extra age allowed for clock differences between servers.

Load option HandoffSeconds overrides the same-server wait for one load() or restoreAt() call.

Wire format

Requests and replies are small tables published through MessageDriver:

Message	Fields	Meaning
Request	{ T = "Q", D = driverName, K = key, R = requesterJobId, L = lockToken? }	Release K of driver D now; L names the lock the requester saw.
Reply	{ T = "R", D = driverName, K = key }	This server no longer holds K.

Messages with a missing driver or key, or for a driver name this server does not have, are ignored.

API reference

The module is internal to DataDriver; these members exist for tests and custom wiring.

Member	Description
Handoff.new({ JobId, Messages?, Presence? }) -> Handoff	Messages defaults to a MessageDriver with one 0.5 s retry, Presence to a MemoryDriver over CKPresence with a 10 s TTL and one 0.25 s retry.
Handoff.shared(jobId) -> Handoff	The one handoff per server that attached drivers use.
Handoff.BeatSeconds, Handoff.PresenceTtlSeconds	3 and 10.
handoff:register(driver)	Routes messages for driver.Config.Name to that driver and subscribes on first use. A failed subscribe warns and handoff falls back to polling.
handoff:start(scheduler, priority?)	Beats presence every 3 s on scheduler. Idempotent.
handoff:beat()	Writes one presence beat. The first failure after a success warns.
handoff:isAlive(jobId) -> boolean?	true for a beat within 10 s (always for this server), false when a read found none, nil when the read failed.
handoff:request(driverName, key, holder, token?)	Asks holder to release key.
handoff:reply(driverName, key, requester)	Tells requester the key is free.
handoff:destroy()	Stops the beat, disconnects the subscription and forgets the drivers.

Gotchas

Do not add rejoin cooldowns

The driver already hands players over. A game-side cooldown or lock wait on top only delays loads that would have succeeded.

• A server whose presence writes fail while it stays up can read as stopped. Its locks are only taken when one of its players joins another server and no reply comes back, and its next save then finds the lock gone (Data.LockLost) and kicks the player there. The beat warns on its first failure.
• Handoff routes by driver Name. Two drivers with the same Name on one server collide.
• LockWaitSeconds bounds every wait. Raising it keeps players on a loading screen longer when a holder is unreachable; lowering it kicks sooner.

Status

The datasafety runner’s _data_handoff.luau runs 9 multi-server cases on a virtual clock whose timers fire, with several servers sharing one backend, one MessagingService double and one MemoryStore double: same-server rejoins during a failing and a lost leave save, a stale connection handed over in under 1.5 s with its latest data, a late request that must not end a newer session, a crash taken over in under 16 s, a crash mid-trade taken over in under 16 s with the trade aborted, a stalling live holder that keeps its lock, a holder without presence that is never taken early, and one stuck autosave that does not block the others. Real MessagingService delivery, MemoryStore expiry timing and real server crashes need a published place with several servers.