Both columns carry the meaning Blogs.IsActive has: 0 = removed by another
tool, anything else (including NULL) = live. Neither exists in the live
TL.db yet, and both are added from outside this crawler, so the code has
to work on databases either side of the change - naming a missing column
is a hard SQLite error.
HasIsActiveColumn asks PRAGMA table_info once per table per database path
and caches it; AndIsActive/WhereIsActive return "COALESCE(IsActive, 1) = 1"
or an empty string. Every read that selects posts or notes now carries the
filter: GetPosts (both branches, including the per-blog count subquery),
GetReplies, GetRepliesWithMissingText, GetRepliesWithFilledText,
GetAllPostTextColumns, GetAllPostsForBlog, GetPost, GetPostByIdAnyBlog,
and the engagement queries that count or join Notes - GetBlogs,
GetBlogsAll and both note-joining variants of GetBlogsForLikes.
The LEFT JOIN Notes in GetPosts is left alone on purpose: nothing is
selected from it and it can neither add nor remove a row.
LegacyPostsDbImporter is left alone too - it reads a foreign legacy
schema.
Writes were already safe and are documented rather than changed: no
INSERT column list names IsActive, no UPDATE sets it, MapPrefixToColumn
cannot map to it, and there is no INSERT OR REPLACE on Posts or Notes for
a column default to be reset by. Re-crawling a removed row refreshes its
content and leaves the flag at 0. As with Blogs, exclusion belongs at
selection, so the update paths stay keyed on rows the caller already
chose.
Verified against three synthetic databases - no IsActive columns, columns
present with a removed post and its notes, and columns present but NULL -
by running every affected reader: the queries are valid in all three, the
removed rows drop out only where the columns exist, NULL reads as live,
and AddPost/AddNote/UpsertPostFromTextFile/UpdatePostContentFields leave
an IsActive = 0 row at 0.
Co-Authored-By: Claude Opus 5 <[email protected]>
Replaces the Blogs.IsDeleted section. Rolodex adds no column of its own;
it reuses the crawler's existing IsActive flag, so removing a blog in the
UI also stops it being collected.
Co-Authored-By: Claude Opus 5 <[email protected]>
TL.db.md documents the live schema: the three content tables and their
row counts, the '.' placeholder convention the crawler writes instead of
NULL, the two incompatible date formats in Blogs.DateAdded, and the
access paths that matter on the 1.19M-row Notes table.
It also covers Blogs.IsDeleted, which Rolodex adds by ALTER TABLE and
this crawler must not write.
The file was sitting untracked next to the database it describes.
Co-Authored-By: Claude Opus 5 <[email protected]>
jimandClaude Opus 5