Skip to content

Sync Item Handlers ​

Per-type handlers encapsulate how each sync item is encoded, decoded, and merged into the local data DB.

Pattern ​

A registry maps each type to a handler implementing a shared interface:

ts
interface SyncItemHandler<T> {
  encode(record: T): Uint8Array
  decode(bytes: Uint8Array): T
  applyUpsert(decoded: T, ctx: SyncContext): void
  applyDelete(id: string, ctx: SyncContext): void
}

const handler = getHandler(item.type)
handler.applyUpsert(decoded, ctx)

Files ​

Handlers are split across two trees while the @memry/sync-client extraction is in progress. Platform-free handlers — the ones that touch only the data DB through the driver-agnostic DrizzleDb — live in the shared package; handlers that still reach into desktop-only code (vault files, crypto, converters) remain in the desktop tree until their seams land:

packages/sync-client/src/item-handlers/   # platform-free (shared with mobile)
├─ types.ts              # SyncItemHandler, ApplyContext, resolveClockConflict
├─ base-handler.ts
├─ bookmark-handler.ts
├─ calendar-binding-handler.ts
├─ calendar-external-event-handler.ts
├─ calendar-source-handler.ts
├─ filter-handler.ts
├─ home-page-handler.ts
├─ note-pin-helpers.ts
├─ reminder-handler.ts
├─ tag-category-handler.ts
└─ task-activity-handler.ts

apps/desktop/src/main/sync/item-handlers/ # desktop-bound (for now)
├─ note-handler.ts
├─ journal-handler.ts
├─ task-handler.ts
├─ project-handler.ts
├─ inbox-handler.ts
├─ template-handler.ts
├─ custom-icon-handler.ts
├─ agent-conversation-handler.ts
├─ agent-message-handler.ts
└─ index.ts              # registry: getHandler(type), getAllHandlers()

The registry stays in the desktop tree until every handler has moved. The platform-free record sync services (bookmark-sync.ts, task-sync.ts, …), the outbox queue.ts and the offline clock helpers moved with the handlers into packages/sync-client/src/.

Desktop's implementations of the ten platform seams live under apps/desktop/src/main/sync/adapters/ — the one place in the extraction surface where electron/node imports are sanctioned. wiring.ts holds the production factories, and the shared conformance suite (@memry/sync-client/adapters/conformance) runs against the real adapters under node in adapters/conformance.test.ts.

Why Strategy Pattern (Phase 3) ​

Phase 3 replaced a switch-based ItemApplier with this registry. The reason: every sync type has subtly different conflict resolution and side effects (e.g. tasks need field-level merge; notes and journals need CRDT integration; inbox items have a triage state machine). Switch statements grew unwieldy.

Conflict Resolution ​

Every handler uses the shared resolveClockConflict() helper for vector-clock compare and merge.

For tasks, projects, and agent conversations, handlers additionally invoke mergeFields() from field-merge.ts to merge field-level vector clocks. See Sync Protocol.

Settings run the same rule per dotted path: mergeSettingsPayloads() (settings-merge.ts) calls mergeFields() over the union of both payloads' clocked paths, so the higher tick sum wins and the remote wins a tie. A path whose winner has no value keeps the local value: an older build strips a setting it does not model but still echoes its clock, and removing on that echo would delete the setting everywhere. Before #2383 settings picked the larger single tick and kept local on a tie; a peer still on that rule converges through the re-queue a concurrent merge triggers. The shared settings-merge.json vectors run against both desktop and the Rust core, including the re-queue flag; since #2399 the Rust core also keeps the local value on an absent winner and re-queues after a concurrent merge.

Agent message sync is append-only. If a message id already exists locally, the handler treats the remote item as idempotent instead of overwriting a terminal message.

buildPushPayload is optional ​

buildPushPayload rebuilds the outgoing payload from local state at push time, so the newest local state goes out even when the queued row was frozen earlier. It is optional on the interface, and BaseItemHandler supplies no default. Every handler backed by a sync table implements it; settings does not, because settings live in config.json and the preferences cache rather than in a table there is anything to rebuild from.

A type without it pushes the frozen queue payload verbatim, so queue bookkeeping alone has to be correct for it — see Push acknowledgements and in-flight mutations. When adding a handler, implement it unless the type genuinely has no local row to read back.

Keys this build does not understand survive the round trip ​

A handler schema is a plain z.object, so Zod strips every payload key it has no field for, and buildPushPayload rebuilds the payload from local columns. On their own those two facts delete a field written by a newer client: the newer app writes snoozedUntil, this build parses it away, projects what is left into columns, and the next local edit pushes a payload with no snoozedUntil in it. The server copy loses the field for every device.

apply-item.ts therefore compares the raw parsed JSON against the schema result and stores whatever was stripped in sync_unknown_fields, keyed by (type, item_id). resolvePushPayload merges that remainder back underneath the freshly built payload, so locally owned keys always win and only keys this build never mentions ride along. Nothing else reads the table — it never affects local behavior.

A handler needs no code for this; it happens around every handler at the apply and push seams. When a later build learns a field for real, its schema keeps the key, the stripped remainder comes back empty, and the row clears itself.

Only top-level keys are preserved. An unknown key nested inside a known object is still stripped by that object's schema.

id and syncedAt are never kept. Many push payloads are row dumps that carry both, but id is the envelope id and syncedAt is device-local, so no schema models them. Keeping them wrote a row for nearly every applied item.

Canvas: the payload comes from a file ​

canvas-handler.ts is the one handler whose content does not live in the data DB. A canvas scene is a .excalidraw file in the vault (see Canvas Files), so buildPushPayload reads the document off disk and applyUpsert writes it there — the row only carries file_path, title and clock. A row whose document is unreadable pushes nothing rather than an empty scene, and the conflict copy gets its own file next to the winner. None of this touches key material; transport encryption is unchanged.

Home boards: the widget layout is an opaque string ​

home-page-handler.ts carries a board's widgets as an opaque JSON string, declared widgets: z.string().optional() in HomePageSyncPayloadSchema — the same call canvas.scene makes, and for a sharper reason. A typed z.array(WidgetInstanceSchema) would zod-strip widget keys written by a newer build and reject the legacy {size:'S'|'M'|'L'} blobs still on disk. A schema failure in apply-item.ts returns 'schema_invalid': the cursor still advances, and the item waits in the schema-invalid ledger until an app update re-fetches it (see Sync Protocol, "Per-item bookkeeping and retry semantics"). A board that every build refuses would still land on zero peers.

Shape is therefore validated at the apply site: applyUpsert refuses a widgets value that is present but does not JSON.parse to an array, and returns 'skipped' before touching the clock so a later readable version of the same board still wins. An absent widgets key is the different case — the sender predates the field — and keeps the local value.

Two more decisions worth carrying to the next handler like this:

  • Ghost guard. pull-coordinator enqueues payload: '{}' on conflict and every payload field is optional, so {} parses. Without a if (!data.name) return 'skipped' guard that materialises a permanent ghost row whose empty clock makes every later real version compare as stale. Copied from template-handler.ts.
  • Hard delete. Record-sync tombstones live on the server item (deleted_at), never in a local column. A soft-delete column would also break downgrade inertness: an older build has no deletedAt in its model and would list tombstoned boards. applyDelete refuses a remote delete only when the local clock happens strictly after the tombstone (see "Delete wins" below), so a concurrent local edit does not keep the row alive.

Board selection is deliberately not synced: which board is open stays in localStorage['memry-home-active-board'] on each device.

Delete Wins Over a Concurrent Write ​

applyDelete skips a pulled tombstone only when the local clock happens strictly after it — BaseItemHandler.resolveDeleteClock (packages/sync-client/src/item-handlers/base-handler.ts). A local clock merely concurrent with the tombstone loses: the row is deleted and the local edit is dropped.

That mirrors the server. shouldRejectResurrection (apps/sync-server/src/services/sync.ts) refuses any non-delete push against a tombstoned id unless the incoming clock happens strictly after the stored one, answering SYNC_DELETE_WINS; push-coordinator drains that rejection without retrying, because a retry is refused identically every time. The rule does not expire: past the version-history window the server keeps the tombstone row as a payload-less marker, and a purged tombstone whose delete attestation verifies reaches applyDelete with the same clock a signed one would carry. An unattested one never reaches a handler.

Handlers used to keep the row on a concurrent clock, and the two rules together stranded exactly one device: the device that edited an item before it saw the delete had its push refused forever and its pull decline the tombstone, so it kept a ghost copy of an item deleted on every other device. A handler that adds its own delete guard has to use resolveDeleteClock, not a hand-rolled resolveClock comparison.

Notes and Journals Share One Table ​

Notes and journals both live in note_metadata, keyed by id alone, while the server keeps one row per (type, id). A legacy-id journal tombstone and a live note with the same id therefore both come back on every pull. Before the guard, the journal tombstone purged the note's CRDT doc and dropped its row, and the note upsert then wrote a fresh Untitled N.md, orphaning the old file on every pull. noteHandler and journalHandler now check the row's type first (journalDate set means journal) through belongsToOtherType (item-handlers/note-row-type.ts). On a mismatch they return 'skipped' with no purge and no file change, and log one warning per id and type.

A remote note delete also removes the note's note_date reminders, directly and with no sync hooks. The device that deleted the note owns those tombstones, so the receiver enqueues nothing.

Atomicity ​

All applyUpsert and applyDelete paths run inside db.transaction():

ts
ctx.db.transaction(() => {
  // upsert into primary table
  // update field_clocks
  // refresh derived index rows
})

A partial write is impossible — either every change in a handler invocation applies, or none.

Atomicity stops at the data DB, though. Index rows are derived state, published as projection events and drained on their own lane, so a handler returns before the index reflects what it just applied. Anything asking "does this item exist here?" must therefore ask the data DB, or accept that a freshly applied item can look absent. The debounced CRDT write-back is the case that matters for notes: from the index alone, a note the handler had already applied looked new, so the write-back re-derived its title from the Yjs meta map — the placeholder a note is created with — and the canonical upsert overwrote the correct title and path. It now falls back to the canonical row before treating a note as new.

The Yjs meta title is not a source of truth in the other direction either. It is set once at note creation and updated only while the note's doc is open in memory, so a note renamed from the sidebar never records the new title there. Read it only when nothing else knows the note.

Module-Level Handler State Is Per-Vault ​

A handler that keeps module-level state — the note handler's guard against re-requesting the same embedded attachment download is the one that exists — must treat that state as belonging to the open vault. It is reset from resetSyncServiceSingletons(), which runs on sync-runtime stop and therefore on vault close, vault switch and sign-out, alongside the per-type service singletons. Session-scoped collections also need a ceiling: the attachment guard is FIFO-capped, because keys are never retired individually and re-requesting one is cheap (the downloader skips files already on disk).

The same rule holds inside the engine. CrdtSyncCoordinator caches an applied-sequence cursor per note, which after a full sync means the entire vault; engine.stop() clears it. Dropping the cursors is safe because the next pass re-derives since from the server snapshot baseline and re-applying a CRDT update is a no-op.

Renderer Events From the Sync Path ​

Handlers notify the renderer through ctx.emit, typed (channel: string, data: unknown). That signature is deliberate — a handler emits on many channels — but it means the renderer-side payload contract is invisible to tsc here, and the write-back path in crdt-writeback.ts has the same hole.

Subscribers do not defend themselves. useNoteLinks reads changes.content for every note that is not the open one, so a notes:updated emitted without changes threw once per note in a pull — inside the preload listener loop, where each callback is caught individually. Nothing crashed; the subscriber simply never saw the event, and link caches stopped refreshing after a pull.

Emit through a typed helper rather than ctx.emit directly when a channel has a declared payload type. note-events.ts is the pattern:

ts
export function emitNoteUpdated(
  emit: (channel: string, data: unknown) => void,
  event: NoteUpdatedEvent
): void {
  emit(NotesChannels.events.UPDATED, event)
}

Two rules for the changes field itself:

  • Only name fields the branch actually wrote. The handler's markdown-update path rewrites frontmatter, title and path but never the note body, so it must leave content out. Including it would remount an open editor over text that did not change.
  • content implies the body moved. The CRDT write-back is the one emitter that sets it. Because scheduleWriteback also fires for local typing — a 500ms debounce, ahead of the editor's 1000ms save — the note page skips source: 'sync' entirely rather than remounting mid-keystroke over bytes the IPC CRDT provider has already applied.

The renderer normalizes a missing changes to {} in onNoteUpdated, so a newer renderer stays tolerant of an older main process. That is a compatibility floor, not a licence to omit the field.

Every applied write must emit. A handler that mutates the data DB and returns applied without notifying the renderer leaves an open window showing stale data until the app restarts — the "my other device changed it but this one still shows the old version" report. registry.test.ts walks SYNC_ITEM_TYPES, applies a minimal payload through each registered handler, and fails any type that returns applied without calling ctx.emit. The agent conversation and message handlers were written without one and are the reason the test exists. Only applied obligates a broadcast: a skipped or parse_error changed nothing locally, so there is nothing to re-read.

Two types are exempt from that probe and say so in the test. settings notifies by walking BrowserWindow.getAllWindows() itself rather than going through ctx.emit; note and journal write the index DB too, so they cannot run against a bare data-db handle and are covered by their own handler tests instead.

Nullable Fields: Omitted vs Explicit Null ​

A nullable column has two distinct remote signals and ?? cannot tell them apart:

  • Key absent — the sender predates the column. Keep the local value.
  • Key present, value null — an explicit clear. Apply it.

Collapsing them either way loses data. data.x ?? null lets an older peer wipe a column it has never heard of; data.x ?? existing.x strands a real clear forever. The inbox handler shipped the first form for ten columns, so a payload from a build predating any one of them silently nulled the local value — and an explicit clear is not hypothetical there: unsnoozeItem, unarchive and unfile all push null (main/inbox/snooze.ts, main/inbox/crud.ts). Reading those as "omitted" would leave an item snoozed on the other device forever.

The house pattern is a local hasKey helper over the raw payload:

ts
const hasKey = (k: string): boolean => Object.prototype.hasOwnProperty.call(data, k)

tx.update(inboxItems).set({
  snoozedUntil: hasKey('snoozedUntil') ? (data.snoozedUntil ?? null) : existing.snoozedUntil
})

calendar-external-event-handler.ts, inbox-handler.ts and calendar-event-handler.ts all use it. Because buildPushPayload serializes the whole row, a same-version peer always sends the key — so hasKey is false exactly when the sender is older, which is precisely when the local value must win.

Insert branches need the same audit for a different reason: they simply omitted six inbox columns, so an item archived on one device arrived un-archived on a device seeing it for the first time. If a column round-trips through buildPushPayload, it has to be written on both branches.

Local-Only Rows Must Not Be Tombstoned ​

shouldSkip on RecordSyncController is the "this row never leaves my device" switch, and it has to hold on delete as well as on create/update. A tombstone carries no body, but the item's id and its deletion time still get encrypted, uploaded and fanned out to every other device in the vault.

enqueueDelete applies it (packages/sync-core/src/record-sync.ts), so a service that passes a shouldSkip gets the guard on every path without a second copy inside buildDeletePayload.

The guard has one hard limit: it reads the row load returns, so it can only fire while that row still exists.

ts
const local = this.deps.load(itemId)
if (local !== undefined && this.deps.shouldSkip?.(local)) return

When load returns undefined the row is already gone and its local-only-ness is unknowable, so the delete is let through deliberately. Refusing there would swallow legitimate tombstones for ordinary rows — data loss in the opposite direction, and the item would be stranded on every other device.

That splits the record services in two by delete ordering:

  • Enqueue, then delete — notes and journals. deleteNoteCommand (main/notes/domain.ts) enqueues first precisely so the clock is still readable, so load still sees the row and the controller guard covers them. Flipping that order would kill the guard silently.
  • Delete, then enqueue — inbox. handleDeletePermanent (main/inbox/crud.ts) snapshots the row, deletes it, and only then enqueues, so load returns undefined on that path every time. The controller guard can never fire there, so inbox-sync.ts guards on the snapshot it is handed — the last thing that still knows the flag.

A service in the second shape has to carry its own guard. An unparseable snapshot falls through to the normal tombstone rather than dropping the delete, so payloads written by older builds keep working.

Missing Parents ​

An FK-bound child whose parent has not arrived yet must throw MissingSyncParentError from inside the transaction, naming the parent type and id:

ts
if (!parent) throw new MissingSyncParentError('task', taskId, 'project', projectId)

pull-coordinator routes only that typed error into orphanedItems, where repairOrphans re-fetches the parent by id and either replays the child or tombstones it. Let SQLite raise its anonymous FOREIGN KEY constraint failed instead and the coordinator logs "deferred retry failed — item skipped until next remote update" and drops the item. For an unchanged upstream record — a Google calendar event nobody has edited — there is no next remote update, so it never appears on that device at all while the UI keeps reporting the source as connected.

sortByApplyOrder ranks parents ahead of children, but only within a page, so cross-page ordering is unprotected and the guard is what makes it safe. task-handler.ts, calendar-external-event-handler.ts and agent-message-handler.ts implement it. Never substitute a placeholder id to get past the constraint: an invented 'unknown-source' can only ever FK-fail, and it makes the failure unclassifiable as well as fatal.

Adding a New Sync Type ​

  1. Define a Zod schema in packages/contracts/<domain>-api.ts.
  2. Add tables / columns in packages/db-schema and write a hand-written migration.
  3. Add the type to every list in packages/contracts/src/sync-api.ts: SYNC_ITEM_TYPES, RECORD_SYNC_ITEM_TYPES, RECORD_CLOCK_REQUIRED_ITEM_TYPES, and ENCRYPTABLE_ITEM_TYPES. Never add to LEGACY_RECORD_SYNC_ITEM_TYPES — it is frozen at the pre-negotiation client's vocabulary.
  4. Implement a handler (the pull side) in packages/sync-client/src/item-handlers/<domain>-handler.ts when it only needs the data DB, or apps/desktop/src/main/sync/item-handlers/<domain>-handler.ts when it still needs desktop-only code.
  5. Register it in the desktop registry index.ts.
  6. Implement a push service (the local side) in packages/sync-client/src/<domain>-sync.ts (platform-free is the default for record types), then register it in bothlocal-mutations.ts and the adapter registry in runtime.ts.
  7. Add a server-side validator in apps/sync-server if the new type has unusual constraints.
  8. Add tests under the handler file (every existing handler has one).
  9. Add a minimal payload for the type to FIXTURE_OVERRIDES in item-handlers/registry.test.ts if {} does not parse against its schema, and seed any FK parent the fixture needs. The registry test fails on an unregistered type and on an applied write that does not emit, so it is the one place a half-wired type shows up as a failure rather than as silence.
  10. Give the type an entry in DIRTY_RECOVERY (main/sync/dirty-recovery.ts): a sweep, or an exemption with a one-line reason. The table is keyed by RecordSyncItemType, so the build fails until you do, and dirty-recovery.test.ts checks it against RECORD_SYNC_ITEM_TYPES.

Steps 5 and 6 are three separate registrations and each fails silently on its own: enqueueLocalSync* typechecks and no-ops when no push service is registered, so the entity never leaves the device. Cover the seam with a test that runs a local mutation through to a peer's apply, not just per-side unit tests.

Initial seeding ​

Rows written without a vector clock — anything a seed script or an older build inserted — are picked up by each handler's seedUnclocked, driven by runInitialSeed on every full sync. That seed reads the complete handler registry (getAllRemoteSyncAdapters()), deliberately not the runtime adapter registry: every other consumer falls back with adapters?.getRemote(type) ?? getRemoteSyncAdapter(type), but getAllRemote() has no fallback, so a type missing from the runtime list would silently never seed and strand its clock-less rows on that device forever.

agent_conversation and agent_message are the intentional exception — their seedUnclocked returns 0 because agent data syncs through the entitlement-gated backfill in main/agent/sync/.

The seed is a safety net, not a licence to write clock-less rows. A type in RECORD_CLOCK_REQUIRED_ITEM_TYPES is rejected by RecordPushItemSchema when its push item carries no clock, and RecordPushRequestSchema validates the whole items array — so one clock-less row fails the entire batch with a request-level VALIDATION_ERROR, not a per-item rejection. Every push then fails and nothing drains until the row is repaired. Any write path that inserts such a row and enqueues it must stamp increment({}, deviceId) itself; the Google Calendar import (calendar/google/sync-service.ts) does this for events it has never seen before, and calendarExternalEventHandler.buildPushPayload stamps and persists a first clock for rows already queued by older builds, so a stuck queue drains on the next push instead of waiting for the next full sync. Stamping without persisting is not enough: the next local edit would tick from {} to the same clock the server already acked, and the update would be dropped as a replay.

A handler's buildPushPayload repair only reaches a create or update whose local row still exists. Three cases skip it and fall back to the frozen queue payload: a delete, a row the handler can no longer read, and a handler that throws. A frozen payload written by a build that predates the stamping rule carries no clock, and one such row fails every batch forever. So PushCoordinator makes a last-resort check on the way out: for a type in RECORD_CLOCK_REQUIRED_ITEM_TYPES, a payload that parses to an object with no clock object leaves with { [deviceId]: 1 } stamped on it. A payload that already has a clock — handler-built or frozen — is untouched, and an unparseable payload is sent as-is. This stamp is not persisted, because the cases that reach it have no local row to persist to; it matches the { id, clock: increment({}, deviceId) } fallback buildDeletePayload already uses. It is a queue-unblocking backstop, not a substitute for stamping at write time.

Sync intents: row and push obligation in one transaction ​

Tasks and projects written through the tasks domain do not enqueue from the publisher. Each command runs its synchronous write phase inside a unit of work (main/tasks/domain.ts), which commits the rows and one sync_intents row per owed mutation in a single SQLite transaction (commitLocalChange, main/sync/sync-intents.ts). The event-to-intent mapping lives only in main/tasks/sync-intents.ts. Right after COMMIT, drainSyncIntents hands each intent to the ordinary local sync adapter (clock bump, sync_queue row, offline fallback) and deletes the intent in that same transaction. The publisher (IPC, activity log, projections, source-note edits) runs after both; a failing publisher call is logged and skipped, it neither drops the remaining events nor rejects the command, because the write has already committed.

  • A throw in the write phase rolls back the rows and the intents. A throw in the sync step never undoes the edit: the intent stays pending (attempts, last_error) and later intents for the same item wait behind it.
  • A delete's sync_pending_deletes tombstone is written in the write transaction, next to its intent, so a failed sync step cannot roll back the guard that keeps a pull from re-creating the item.
  • A cascade commits its tombstones with the delete. deleteProject writes the project delete intent and one task delete intent per cascaded task, each built from getTask so it carries the task's real clock.
  • Pending intents are drained at runtime start (recoverDirtyItems, before its sweep) and at the start of every pull. The dirty sweep skips every item that had an intent: a failed one still carries its pre-edit clock, and a sweep push at that clock is refused by the server as a replay and stamped synced.
  • A remote upsert for an item with a pending intent drains that item first (ItemApplier). If the intent still cannot drain, the apply throws PendingSyncIntentError; the pull defers the item to its end-of-run retry and then to the schema-invalid ledger as pending_intent. That entry is re-fetched by id at every pull start, right after the intents drain, and merges field by field once the local edit is clocked. It keeps the manifest from counting the item server-only and is not listed as quarantined. The remote row never overwrites the un-clocked local edit.
  • A delete intent whose row exists locally again is stale (a downgrade round trip): it is dropped with its tombstone and the row is kept.
  • Only the runtime-start replay spends an intent's attempt budget; pull-start, per-item and per-edit drains retry without counting. Past five start-up attempts nothing is given up: the intent stays pending, so it keeps deferring remote rows for its item and keeps the sweep off it, and each start reports it as over cap. The retry always uses the intent's own fields, never an all-field bump.
  • An intent this build cannot read (unknown type or op, bad args) belongs to a newer build. It is left pending and untouched so a re-upgrade replays it, and this build ignores it: it does not own, guard or block the item. It is logged once per session.
  • A tag merge retag owes ['tags'] per task and a status change owes the project ['statuses'], the same field names updateTask and updateProject report.
  • onItemEnqueued (the push wake-up) is deferred one microtask and coalesced, so it fires after the caller's outermost COMMIT.

The task and project writers outside the tasks domain use the same path: inbox task conversion (inbox/filing.ts), the note-project-links projector and the tag merge retag (commitTaskRetag, tags/runtime-effects.ts). The activity log's task_activity rows and every other type still enqueue after their own commit.

Project links derived while applying a synced note are the exception: reconcileNoteLinks(..., 'remote') writes the rows and normally commits no intent. The device that edited the note's frontmatter already pushed the project, and that payload is where iOS reads markdown-note membership. Re-pushing from every receiver only bumped the project clock on each device and could push a new row's position: 0 and pinned: 0 over a pin set elsewhere. When projectHandler inserts a project from sync, linkNotesNamingProject links the notes whose frontmatter already names it, so a note applied before its project does not wait for a re-pull.

A derived link still pushes the project when the project's last synced links payload is known to lack that note. That happens when the note's writer could not resolve the project name (the project was created elsewhere while it was offline) or had no frontmatter projector. The projector keeps the note ids of the last payload this device applied or had acknowledged, per project, in memory. A project not synced since start-up counts as carrying the note, so nothing is pushed on a guess. Once a peer's push carries the note, receivers stop pushing.

The tier-0 stat ingest (an external rename, a re-add) and the large-file tier never read frontmatter. They publish note.upserted with properties: null, and the links projector skips them, so those paths keep the note's link rows, their ids and their pinned/position.

Dirty recovery ​

Outside the sync-intent path, a local edit writes the row, the clock and the outbox row in three transactions. A crash between the last two, or an increment*ClockOffline fallback while the runtime is down, leaves a clocked row with no queue row. recoverDirtyItems runs at every sync runtime start and re-enqueues those rows, driven by DIRTY_RECOVERY: one entry per record sync item type, either a sweep (select the rows with syncedAt IS NULL or a modification time past syncedAt, then hand each to the type's local sync service) or an exemption naming why the type has no usable dirty marker. Clock-less rows are left to seedUnclocked. A never-synced row goes out as a create; a modified one as a recovered update at its stored clock. Both rebind _offline ticks first through recoverPendingChange, so the placeholder device id never reaches the wire. Exempt types (settings, tag definitions and categories, folder configs, property definitions, the calendar types, canvases) are not on the sync-intent path yet and wait for its per-type rollout (#2301); agent chat has no local push path.

Three sync_run_completed events carry the P4.2 gate signal, with numeric metrics only:

  • action: 'sync_intents_replayed', from the runtime-start drain whenever it finds intents, and from a pull-start drain only when one applied or was dropped as stale: itemCount attempted, resultCount applied, retryCount failed and kept, value stale deletes dropped.
  • action: 'sync_intents_over_cap' (result: 'failed'), once per runtime start: itemCount intents still failing after five start-up replays.
  • action: 'dirty_recovery_residual' per type, with objectType set to the sync type: itemCount dirty rows that had neither a queue row nor a pending intent, resultCount rows re-enqueued.

Offline edits count as residual too, so compare migrated types against the rest. The residual count only sees rows whose write moves the modification time; task_tags and project_links writes do not, which is why the three writers above now go through intents.

Handlers that persist locally encrypted fields must receive the vault key from the sync engine during pull apply and push payload encoding. Agent conversation and message handlers use that key to decrypt their SQLite envelopes and re-encode sync payloads without exposing plaintext to the server.

Field-Level Merge Quick Reference ​

For tasks and projects:

ts
const result = mergeFields({
  local: existing,
  remote: incoming,
  fields: TASK_SYNCABLE_FIELDS,
  localFieldClocks: existing.fieldClocks,
  remoteFieldClocks: incoming.fieldClocks
})

// result.merged: T
// result.mergedFieldClocks: FieldClocks
// result.hadConflicts: boolean
// result.conflictedFields: string[]

When a record predates Phase 8 and lacks fieldClocks, initAllFieldClocks(docClock, fields) initializes them on first merge.

Released under the GNU GPL v3.0.