feat(db)!: port DataAccess to the Notes integer schema

Notes.RootBlogName/NoteBlogName/Type became RootBlogId/NoteBlogId/TypeId
on 2026-08-07, resolved through the new BlogNames and NoteTypes tables.
There is no compatibility view, so every affected statement is a hard cut.

All 14 call sites in DataAccess.cs are ported:

- Notes->Blogs joins go through Blogs.BlogId in one integer hop; the
  Notes->Posts join in GetRepliesWithFilledText is the only one that must
  route through BlogNames, since Posts carries no BlogId
- AddNote registers both blog names and the note type with INSERT OR
  IGNORE before inserting, in one transaction committed before the console
  sleep. Registering the type matters: an unseen type would resolve to
  NULL and fail NOT NULL on TypeId, silently losing the note
- The LEFT JOIN Notes in GetPosts is dropped rather than translated. It
  selected nothing, could not remove a row, and its duplicates were
  collapsed by the query's own GROUP BY
- Duplicate-key detection moves to IsNotesDuplicateKey, matching the
  constraint and table instead of an exact column list. The old literal
  string is what broke on this rename
- EnsureReplyTextColumnExists drops DEFAULT '.', matching the migrated
  schema: new rows get NULL, not a placeholder nobody wrote

verify-db-schema.sql gains BlogNames, NoteTypes, Blogs.BlogId and the new
Notes columns, plus query 1d naming a pre-migration file and pointing at
normalize-notes.sql. Blogs.BlogId is deliberately not auto-fixable -- an
added-but-empty column makes engagement joins return zero rows silently.

Verified against the live 148 MB file: query plans hit the intended
indexes, and the BlogId join matches an independent name-resolved
formulation exactly on all 4,267 GetBlogs and 2,637 GetBlogsForLikes rows.

RolodexRepository.cs (16 sites) lives in the Rolodex repo and is not
covered here.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
jim
2026-08-07 21:23:13 -05:00
co-authored by Claude Opus 5
parent c3cf89c3f1
commit b31d5842cc
4 changed files with 277 additions and 79 deletions
+30
View File
@@ -47,6 +47,36 @@ say nothing about the item being fetched, so they must not be recorded as per-it
- Long-running commands return exit 3 when a pass ends incomplete (rate-limit pause, breaker trip, or
skipped items), so a caller can distinguish that from a clean run
### `Notes` Stores Integer IDs, Not Names
As of 2026-08-07 `Notes.RootBlogName`, `NoteBlogName` and `Type` are gone, replaced by
`RootBlogId`, `NoteBlogId` and `TypeId` resolving through the `BlogNames` and `NoteTypes`
lookup tables. There is no compatibility view — naming an old column is a hard SQLite
error, so unlike `IsActive` this is a hard cut with no runtime probe. Full detail in
`URLNotesGrabberCORE/TL.db.md`.
- **Joining `Notes` to `Blogs` goes through `Blogs.BlogId`**, not `BlogNames`:
`FROM Blogs B INNER JOIN Notes N ON N.NoteBlogId = B.BlogId`. Routing it through
`BlogNames` adds a hop and ends in the text comparison the migration removed
- **Joining `Notes` to `Posts` is the opposite** — `Posts` has only `BlogName`, so it must
go through `BlogNames` (`GetRepliesWithFilledText`). This is the only such join
- **Resolve a name by filtering the lookup, never by scanning `Notes`**:
`WHERE NoteBlogId = (SELECT BlogId FROM BlogNames WHERE BlogName = @name)`. The subquery
is a unique-index probe on 20k rows and does not show against the 1.18M-row table
- **`AddNote` registers both blog names *and* the note type** with `INSERT OR IGNORE`
before inserting, all in one transaction. `NoteTypes` is a table rather than a `CHECK`
constraint precisely so an unseen type is an `INSERT`; without that registration it
would resolve to `NULL` and fail the `NOT NULL` on `TypeId`, losing the note
- **`Blogs.BlogId` is NULL on 168,202 of 188,620 rows** — every blog that has never
appeared in a note. An inner join on it silently drops them. Correct for engagement
queries, wrong for anything listing the registry
- **IDs are stable and must never be renumbered.** They are stored in 1.18M `Notes` rows.
A blog renamed upstream gets a new `BlogNames` row, not an edited one
- Prefer `TypeId = (SELECT TypeId FROM NoteTypes WHERE Type = 'reply')` over a hardcoded
ID. A negated `TypeId NOT IN (SELECT …)` is only correct because `TypeId` is `NOT NULL`
- Duplicate-key detection uses `IsNotesDuplicateKey`, which matches the constraint and the
table rather than an exact column list. The old literal string comparison broke silently
on this rename — do not reintroduce one
### `IsActive` Is Not Ours To Write
`Blogs.IsActive`, `Posts.IsActive` and `Notes.IsActive` are removal flags set by other tools
(Rolodex). `0` means removed; anything else, including `NULL`, means live. Full detail in