fix: correctness & data-integrity fixes across user profiles and turso sharding - #252
Open
phil-lipp wants to merge 5 commits into
Open
fix: correctness & data-integrity fixes across user profiles and turso sharding#252phil-lipp wants to merge 5 commits into
phil-lipp wants to merge 5 commits into
Conversation
…ing_paths
The cold-start buffer was a single instance field on the profile-manager
singleton, so items observed for one user during embedding warm-up drained
into whichever user's mergeItems ran next — merging User A's preferences into
User B's profile. Key the buffer by profileId (Map) and drain only the current
profile's bucket; legacy unattributed buffer files are dropped on load.
Separately, createProfile/updateProfile rebuilt cleanedData as only
{preferences, patterns, workflows}, silently stripping learning_paths on every
write and rendering the Learning Paths injection feature dead. Carry the field
through.
rebuildProfileUsing dereferenced originalItem unconditionally in the merged-group branch, so a keeper id the model hallucinated (present only in its mapping, not in originalById) threw and aborted the entire cleanup run. Skip such groups, and normalize the AI-controlled mapping (kept/merged/removed) to well-formed arrays so a malformed response degrades to a no-op cleanup instead of throwing. getUserProfileContext parsed the stored profileData with no guard, so one corrupt row broke context injection for every request. Wrap in try/catch and return null.
…ialization Three fire-and-forget/unawaited hazards in the learning cycle: - The retry-exhausted branch did not await markMultipleAsUserLearningCaptured, so finally cleared isLearningRunning while the write was in flight; the next cycle re-fetched and re-analyzed the same prompts (token burn), and the rejection was unhandled. Await it. - evolveAndUpdate mutates item.description/centroid in place but was called fire-and-forget, racing the JSON.stringify in updateProfile — the evolved description was included or lost nondeterministically. Make applyValidations async and await the evolve so mutation completes pre-serialization. - performUserProfileLearning had try/finally but no catch; JSON.parse of a corrupt profileData row rejected the promise (unhandled at the fire-and-forget site). Add a catch that logs and returns.
…ck cleanup - createShard committed the registry INSERT before initShardDb ran; if init threw (disk full, permissions) the row persisted pointing at an uninitialized file, so the next getWriteShard failed isShardValid and threw 'incompatible or corrupt', blocking all writes to that scope. Initialize the shard DB first (it is idempotent), then insert. - getShardByPath matched db_path with LIKE '%' || filename, so the underscores in shard names (user_<hash>_shard_N.db) acted as single-char wildcards and could match the wrong row. Anchor on the '/' separator and escape LIKE metacharacters. - readLiveLock called unlinkSync outside its try/catch; a race (already removed) or a Windows open handle threw ENOENT/EPERM out of assertNoTursoMigrationInProgress and falsely blocked writes. Wrap it.
Asserts that items buffered for one profile during embedding cold-start never drain into another profile's merge, and that each bucket drains only for its own profile.
phil-lipp
force-pushed
the
fix/correctness-data-integrity
branch
from
August 13, 2026 16:39
999d51c to
e79af24
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ten correctness / data-integrity bugs found during a code-quality pass, grouped into five reviewable commits (each builds and typechecks independently). No behavior changes beyond fixing the defects; no public API changes.
bun run typecheckclean,bun testgreen (adds one regression test).Fixes
user-profile-manager.tscoldBufferwas a single instance field on the manager singleton. Items buffered during embedding warm-up for one user drained into whichever user'smergeItemsran next, merging User A's preferences into User B's profile. Now keyed perprofileId(Map); each merge drains only its own bucket. Legacy unattributed buffer files are dropped on load. (covered by the new test)learning_pathsstripped on every write.createProfile/updateProfilerebuiltcleanedDataas only{preferences, patterns, workflows}, silently discardinglearning_pathsset bybuildLearningPaths— the Learning Paths injection feature was effectively dead. Carried through.user-memory-learning.tsmarkMultipleAsUserLearningCaptured, sofinallyclearedisLearningRunningmid-write; the next cycle re-fetched and re-analyzed the same prompts (token burn), and the rejection was unhandled.evolveAndUpdateraces serialization (lost update). It mutatesitem.description/centroidin place but was called.catch(()=>{}), racing theJSON.stringifyinupdateProfile— the evolved description was included or lost nondeterministically.applyValidationsis nowasyncand awaits the evolve so mutation completes pre-serialization.catcharound stored-JSON parses.performUserProfileLearninghadtry/finallybut nocatch;JSON.parseof a corruptprofileDatarow rejected the promise (unhandled at the fire-and-forget call site). Added.user-profile/ai-cleanup.tsoriginalById) threw onoriginalItem.frequency, aborting the entire run. Skip such groups. Also addednormalizeAIMappingso a malformedmapping(kept/merged/removed) degrades to a no-op cleanup instead of throwing on.map/.filter/.includes.user-profile/profile-context.tsJSON.parsein the injection hot path. One corrupt profile row broke context injection for every request. Wrapped in try/catch; returnsnullon parse failure (mirrorsloadColdBuffer).turso/shard-manager.tscreateShardcommitted the registryINSERTbeforeinitShardDbran; if init threw (disk full, permissions) the row persisted pointing at an uninitialized file, so the nextgetWriteShardfailedisShardValidand threw "incompatible or corrupt", blocking all writes to that scope. Now initializes the shard DB first (initShardDbis idempotent), then inserts.getShardByPathLIKE treats_as a wildcard.WHERE db_path LIKE '%' || ?with a filename full of_(user_<hash>_shard_N.db) matched any char and could return the wrong shard row. Anchored on the/separator with escaped LIKE metacharacters (ESCAPE '\').turso/operation-lock.tsunlinkSyncin lock cleanup.readLiveLockcalledunlinkSyncoutside its try/catch; a race (already removed) or a Windows open handle threw ENOENT/EPERM out ofassertNoTursoMigrationInProgress, falsely blocking writes. Wrapped it.Tests
Adds
tests/user-profile-cold-buffer-isolation.test.ts, asserting that items buffered for one profile during embedding cold-start never drain into another profile's merge, and that each bucket drains only for its own profile.Notes for reviewers
normalizeAIMappingintentionally coerces to safe empty arrays rather than throwing, so a bad model response results in a no-op cleanup (originals preserved) rather than an aborted run — the more data-preserving choice.