Merge branch 'claude/session-fa480e' into master

fix(collect): exclude posts from IsActive=0 blogs in --collect 1

Co-Authored-By: Claude Sonnet 5 <[email protected]>

.github/copilot-instructions.md

@@ -31,7 +31,7 @@ dotnet run -- --test [blogname] [postID]  # Test API for specific post
 - `--test [blogname] [postID]`: Test API note collection
 - `--posts`: Export post blogs to file
 - `--blogs`: Export blog list to file
-- `--collect`: Collect notes for all posts in DB
+- `--collect [0|1] [datetime] [blogname]`: Collect notes for posts in DB. Optional `blogname` restricts the run to one blog (exact match), e.g. `--collect 1 zomb-eh`. Add `--force` to ignore the periodic re-collect cooldown so already-collected posts are re-queued immediately (mode 1 only). Add `--fromDate <datetime>` / `--toDate <datetime>` to only re-queue already-collected posts whose original PostDate is on/after / on/before that date (mode 1 only; either or both may be given; applies with or without `--force`)
 - `--blogsR`: Export reply blogs to file
 - `--blogsO [start] [stop]`: Export blogs within range
 

AGENTS.md

@@ -47,6 +47,36 @@ say nothing about the item being fetched, so they must not be recorded as per-it
 - Long-running commands return exit 3 when a pass ends incomplete (rate-limit pause, breaker trip, or
   skipped items), so a caller can distinguish that from a clean run
 
+### `Notes` Stores Integer IDs, Not Names
+As of 2026-08-07 `Notes.RootBlogName`, `NoteBlogName` and `Type` are gone, replaced by
+`RootBlogId`, `NoteBlogId` and `TypeId` resolving through the `BlogNames` and `NoteTypes`
+lookup tables. There is no compatibility view — naming an old column is a hard SQLite
+error, so unlike `IsActive` this is a hard cut with no runtime probe. Full detail in
+`URLNotesGrabberCORE/TL.db.md`.
+
+- **Joining `Notes` to `Blogs` goes through `Blogs.BlogId`**, not `BlogNames`:
+  `FROM Blogs B INNER JOIN Notes N ON N.NoteBlogId = B.BlogId`. Routing it through
+  `BlogNames` adds a hop and ends in the text comparison the migration removed
+- **Joining `Notes` to `Posts` is the opposite** — `Posts` has only `BlogName`, so it must
+  go through `BlogNames` (`GetRepliesWithFilledText`). This is the only such join
+- **Resolve a name by filtering the lookup, never by scanning `Notes`**:
+  `WHERE NoteBlogId = (SELECT BlogId FROM BlogNames WHERE BlogName = @name)`. The subquery
+  is a unique-index probe on 20k rows and does not show against the 1.18M-row table
+- **`AddNote` registers both blog names *and* the note type** with `INSERT OR IGNORE`
+  before inserting, all in one transaction. `NoteTypes` is a table rather than a `CHECK`
+  constraint precisely so an unseen type is an `INSERT`; without that registration it
+  would resolve to `NULL` and fail the `NOT NULL` on `TypeId`, losing the note
+- **`Blogs.BlogId` is NULL on 168,202 of 188,620 rows** — every blog that has never
+  appeared in a note. An inner join on it silently drops them. Correct for engagement
+  queries, wrong for anything listing the registry
+- **IDs are stable and must never be renumbered.** They are stored in 1.18M `Notes` rows.
+  A blog renamed upstream gets a new `BlogNames` row, not an edited one
+- Prefer `TypeId = (SELECT TypeId FROM NoteTypes WHERE Type = 'reply')` over a hardcoded
+  ID. A negated `TypeId NOT IN (SELECT …)` is only correct because `TypeId` is `NOT NULL`
+- Duplicate-key detection uses `IsNotesDuplicateKey`, which matches the constraint and the
+  table rather than an exact column list. The old literal string comparison broke silently
+  on this rename — do not reintroduce one
+
 ### `IsActive` Is Not Ours To Write
 `Blogs.IsActive`, `Posts.IsActive` and `Notes.IsActive` are removal flags set by other tools
 (Rolodex). `0` means removed; anything else, including `NULL`, means live. Full detail in

RERUN.sqbpro

@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="UTF-8"?><sqlb_project><db path="C:/Users/jim/Nextcloud/C#/URLNotesGrabberCORE/URLNotesGrabberCORE/TL.db" readonly="0" foreign_keys="1" case_sensitive_like="0" temp_store="0" wal_autocheckpoint="1000" synchronous="2"/><attached/><window><main_tabs open="structure browser pragmas query" current="3"/></window><tab_structure><column_width id="0" width="300"/><column_width id="1" width="0"/><column_width id="2" width="100"/><column_width id="3" width="4305"/><column_width id="4" width="0"/><expanded_item id="0" parent="1"/><expanded_item id="1" parent="1"/><expanded_item id="2" parent="1"/><expanded_item id="3" parent="1"/></tab_structure><tab_browse><table title="Posts" custom_title="0" dock_id="4" table="4,5:mainPosts"/><dock_state state="000000ff00000000fd00000001000000020000077200000379fc0100000006fb000000160064006f0063006b00420072006f00770073006500310100000000000004a10000000000000000fb000000160064006f0063006b00420072006f00770073006500320100000000000004a10000000000000000fb000000160064006f0063006b00420072006f00770073006500330100000000000004a10000000000000000fb000000160064006f0063006b00420072006f00770073006500350100000000000005f40000000000000000fb000000160064006f0063006b00420072006f00770073006500340100000000000007720000011700fffffffb000000160064006f0063006b00420072006f00770073006500340100000000000005f40000000000000000000007720000000000000004000000040000000800000008fc00000000"/><default_encoding codec=""/><browse_table_settings><table schema="main" name="ApiKeyPoolMeta" show_row_id="0" encoding="" plot_x_axis="" unlock_view_pk="_rowid_" freeze_columns="0"><sort/><column_widths><column index="1" value="29"/><column index="2" value="64"/></column_widths><filter_values/><conditional_formats/><row_id_formats/><display_formats/><hidden_columns/><plot_y_axes/><global_filter/></table><table schema="main" name="Blogs" show_row_id="0" encoding="" plot_x_axis="" unlock_view_pk="_rowid_" freeze_columns="0"><sort/><column_widths><column index="1" value="257"/><column index="2" value="95"/><column index="3" value="54"/><column index="4" value="156"/><column index="5" value="51"/><column index="6" value="71"/><column index="7" value="85"/><column index="8" value="156"/><column index="9" value="156"/></column_widths><filter_values/><conditional_formats/><row_id_formats/><display_formats/><hidden_columns/><plot_y_axes/><global_filter/></table><table schema="main" name="Posts" show_row_id="0" encoding="" plot_x_axis="" unlock_view_pk="_rowid_" freeze_columns="0"><sort><column index="28" mode="1"/></sort><column_widths><column index="1" value="241"/><column index="2" value="148"/><column index="3" value="126"/><column index="4" value="300"/><column index="5" value="75"/><column index="6" value="187"/><column index="7" value="159"/><column index="8" value="75"/><column index="9" value="300"/><column index="10" value="300"/><column index="11" value="78"/><column index="12" value="249"/><column index="13" value="300"/><column index="14" value="53"/><column index="15" value="300"/><column index="16" value="300"/><column index="17" value="41"/><column index="18" value="75"/><column index="19" value="96"/><column index="20" value="300"/><column index="21" value="96"/><column index="22" value="300"/><column index="23" value="300"/><column index="24" value="42"/><column index="25" value="60"/><column index="26" value="218"/><column index="27" value="920"/><column index="28" value="156"/><column index="29" value="156"/></column_widths><filter_values><column index="24" value="=1"/><column index="28" value="&gt;2026-05-06 20:00:01"/></filter_values><conditional_formats/><row_id_formats/><display_formats/><hidden_columns/><plot_y_axes/><global_filter/></table></browse_table_settings></tab_browse><tab_sql><sql name="SQL 1">UPDATE Posts
+<?xml version="1.0" encoding="UTF-8"?><sqlb_project><db path="D:/NextCloud/C#/URLNotesGrabberCORE/URLNotesGrabberCORE/TL.db" readonly="0" foreign_keys="1" case_sensitive_like="0" temp_store="0" wal_autocheckpoint="1000" synchronous="2"/><attached/><window><main_tabs open="structure browser pragmas query" current="3"/></window><tab_structure><column_width id="0" width="300"/><column_width id="1" width="0"/><column_width id="2" width="100"/><column_width id="3" width="4486"/><column_width id="4" width="0"/><expanded_item id="0" parent="1"/><expanded_item id="1" parent="1"/><expanded_item id="2" parent="1"/><expanded_item id="3" parent="1"/></tab_structure><tab_browse><table title="Posts" custom_title="0" dock_id="4" table="4,5:mainPosts"/><dock_state state="000000ff00000000fd0000000100000002000005470000029afc0100000006fb000000160064006f0063006b00420072006f00770073006500310100000000000004a10000000000000000fb000000160064006f0063006b00420072006f00770073006500320100000000000004a10000000000000000fb000000160064006f0063006b00420072006f00770073006500330100000000000004a10000000000000000fb000000160064006f0063006b00420072006f00770073006500350100000000000005f40000000000000000fb000000160064006f0063006b00420072006f00770073006500340100000000000005470000011100fffffffb000000160064006f0063006b00420072006f00770073006500340100000000000005f40000000000000000000005470000000000000004000000040000000800000008fc00000000"/><default_encoding codec=""/><browse_table_settings><table schema="main" name="ApiKeyPoolMeta" show_row_id="0" encoding="" plot_x_axis="" unlock_view_pk="_rowid_" freeze_columns="0"><sort/><column_widths><column index="1" value="29"/><column index="2" value="64"/></column_widths><filter_values/><conditional_formats/><row_id_formats/><display_formats/><hidden_columns/><plot_y_axes/><global_filter/></table><table schema="main" name="Blogs" show_row_id="0" encoding="" plot_x_axis="" unlock_view_pk="_rowid_" freeze_columns="0"><sort><column index="7" mode="1"/></sort><column_widths><column index="1" value="257"/><column index="2" value="108"/><column index="3" value="63"/><column index="4" value="156"/><column index="5" value="60"/><column index="6" value="81"/><column index="7" value="85"/><column index="8" value="156"/><column index="9" value="156"/><column index="10" value="151"/><column index="11" value="125"/><column index="12" value="129"/></column_widths><filter_values><column index="4" value="1"/><column index="7" value="&gt;2026-05-27 17:22:36"/></filter_values><conditional_formats/><row_id_formats/><display_formats/><hidden_columns/><plot_y_axes/><global_filter/></table><table schema="main" name="Notes" show_row_id="0" encoding="" plot_x_axis="" unlock_view_pk="_rowid_" freeze_columns="0"><sort/><column_widths><column index="1" value="198"/><column index="2" value="144"/><column index="3" value="251"/><column index="4" value="84"/><column index="5" value="53"/><column index="6" value="300"/><column index="7" value="116"/><column index="8" value="152"/><column index="9" value="89"/><column index="10" value="63"/></column_widths><filter_values/><conditional_formats/><row_id_formats/><display_formats/><hidden_columns/><plot_y_axes/><global_filter/></table><table schema="main" name="Posts" show_row_id="0" encoding="" plot_x_axis="" unlock_view_pk="_rowid_" freeze_columns="0"><sort><column index="14" mode="1"/></sort><column_widths><column index="1" value="236"/><column index="2" value="144"/><column index="3" value="126"/><column index="4" value="32"/><column index="5" value="32"/><column index="6" value="32"/><column index="7" value="32"/><column index="8" value="32"/><column index="9" value="0"/><column index="10" value="0"/><column index="11" value="0"/><column index="12" value="243"/><column index="13" value="300"/><column index="14" value="53"/><column index="15" value="37351"/><column index="16" value="300"/><column index="17" value="41"/><column index="18" value="75"/><column index="19" value="96"/><column index="20" value="300"/><column index="21" value="96"/><column index="22" value="300"/><column index="23" value="300"/><column index="24" value="548"/><column index="25" value="60"/><column index="26" value="213"/><column index="27" value="532"/><column index="28" value="152"/><column index="29" value="152"/><column index="30" value="69"/><column index="31" value="63"/></column_widths><filter_values><column index="2" value="0"/><column index="1" value="734568371821084672"/></filter_values><conditional_formats/><row_id_formats/><display_formats/><hidden_columns><column index="9" value="1"/><column index="10" value="1"/><column index="11" value="1"/></hidden_columns><plot_y_axes/><global_filter/></table></browse_table_settings></tab_browse><tab_sql><sql name="SQL 1">UPDATE Posts
 SET HasNotesGathered = 0
 WHERE (BlogName, PostID) IN (
     SELECT p.BlogName, p.PostID
@@ -9,8 +9,8 @@ WHERE (BlogName, PostID) IN (
         SELECT 1 
         FROM Notes n 
         WHERE n.PostID = p.PostID 
-          AND n.RootBlogName = p.BlogName
+          AND n.RootBlogId = (SELECT BlogId FROM BlogNames WHERE BlogName = p.BlogName)
-          --AND n.Type NOT IN ('reblog', 'reply')
+          --AND n.TypeId NOT IN (SELECT TypeId FROM NoteTypes WHERE Type IN ('reblog', 'reply'))
     )
     ORDER BY P.PostDate ASC
     --LIMIT 500
@@ -29,39 +29,81 @@ blogname in
 'nudenymph',
 'caylachief'
 
-)</sql><sql name="New Notes">select RootBlogName, PostID, NoteBlogName || '.tumblr.com' as NoteBlogName, DatetimeCrawled, TimeStamp, type, RootBlogName || '.tumblr.com/post/' || postid, datetime(timestamp, 'unixepoch')
+)</sql><sql name="New Notes">select P.slug, N.replyText, rbn.BlogName as RootBlogName, n.PostID, nbn.BlogName || '.tumblr.com' as NoteBlogName, DatetimeCrawled, TimeStamp, nt.Type, rbn.BlogName || '.tumblr.com/post/' || n.postid, datetime(timestamp, 'unixepoch')
-from Notes
+from Notes N
+inner join Posts P on p.PostID = n.PostID
+inner join BlogNames rbn on rbn.BlogId = n.RootBlogId
+inner join BlogNames nbn on nbn.BlogId = n.NoteBlogId
+inner join NoteTypes nt on nt.TypeId = n.TypeId
 where
-DatetimeCrawled &gt; '2026-05-14 02:50:05' --and type like 'r%'
+DatetimeCrawled &gt; '2026-08-07 11:47:22' and nt.Type like 'r%'
-order by DatetimeCrawled desc</sql><sql name="Pull Blogs*">SELECT distinct␍
+and P.IsActive = 1
+order by n.DatetimeCrawled</sql><sql name="Pull Blogs">SELECT distinct
 	'''' || blogname || ''',',
 	blogs.*
 	, blogname || '.tumblr.com'
 FROM
 	Blogs
 		inner JOIN
-	Notes on notes.noteBlogName = blogs.BlogName
+	Notes on notes.noteBlogId = blogs.BlogId
+		inner JOIN
+	NoteTypes on NoteTypes.TypeId = Notes.TypeId
 WHERE
-	HasBeenOutput = 0 and type = 'reblog'
+	HasBeenOutput = 0 and NoteTypes.Type = 'reblog'
 order by
-	Notes.Type desc,
+	NoteTypes.Type desc,
 	DateAdded desc
 LIMIT 100;</sql><sql name="SQL 7">WITH ReplyCounts AS (
     SELECT
-        NoteBlogName, 
+        NoteBlogId,
         COUNT(DISTINCT replyText) AS DistinctReplyCount
     FROM Notes
 	where replyText &lt;&gt; '.'
-    GROUP BY NoteBlogName
+    GROUP BY NoteBlogId
 )
 SELECT
-    n.RootBlogName || '.tumblr.com/post/' || n.PostID AS PostURL, postid,
+    rbn.BlogName || '.tumblr.com/post/' || n.PostID AS PostURL, postid,
-    n.NoteBlogName,
+    nbn.BlogName AS NoteBlogName,
     n.replyText,
     c.DistinctReplyCount
 FROM Notes n
-JOIN ReplyCounts c ON n.NoteBlogName = c.NoteBlogName
+JOIN ReplyCounts c ON n.NoteBlogId = c.NoteBlogId
-where replyText &lt;&gt; '.' and type &lt;&gt; 'reply'
+JOIN BlogNames rbn ON rbn.BlogId = n.RootBlogId
---AND N.NoteBlogName NOT IN ( 'roadblocker21', 'thesaddemon666', 'edwardabbeyhoffman', 'tattedsoldier20', 'zomb-eh', 'animalistic13', 'indken', 'maccloud1592',
+JOIN BlogNames nbn ON nbn.BlogId = n.NoteBlogId
+JOIN NoteTypes t ON t.TypeId = n.TypeId
+where replyText &lt;&gt; '.' and t.Type &lt;&gt; 'reply'
+--AND nbn.BlogName NOT IN ( 'roadblocker21', 'thesaddemon666', 'edwardabbeyhoffman', 'tattedsoldier20', 'zomb-eh', 'animalistic13', 'indken', 'maccloud1592',
 --'moss-wizard', 'supertrucker12682', 'exploringthrupics', 'padeyepete' )
-order by c.DistinctReplyCount desc, n.NoteBlogName, n.DateModified desc, replyText, RootBlogName, PostID</sql><current_tab id="3"/></tab_sql></sqlb_project>
+order by c.DistinctReplyCount desc, nbn.BlogName, n.DateModified desc, replyText, rbn.BlogName, PostID</sql><sql name="Collect">WITH PostsWithCount AS ( SELECT P.BlogName, P.PostID, 1925013599 AS LatestNoteTimestamp, P.NotesGatheredDateTime, COUNT(P.PostID) OVER(PARTITION BY P.BlogName) AS CNT, P.HasNotesGathered, P.NotFound, P.PostDate FROM Posts P WHERE COALESCE(P.IsActive, 1) = 1 ), Unioned AS ( SELECT BlogName, PostID, LatestNoteTimestamp, NotesGatheredDateTime, CNT, PostDate FROM PostsWithCount WHERE NotFound = 0 AND HasNotesGathered = 0 UNION SELECT BlogName, PostID, LatestNoteTimestamp, NotesGatheredDateTime, CNT, PostDate FROM PostsWithCount WHERE BlogName = 'zomb-eh' AND NotFound = 0 AND NotesGatheredDateTime &lt; unixepoch('now', 'localtime', '-3 days') ) SELECT U.BlogName, U.PostID, U.LatestNoteTimestamp, U.NotesGatheredDateTime, U.CNT FROM Unioned U WHERE (U.NotesGatheredDateTime &lt; 1786134037 OR U.NotesGatheredDateTime IS NULL) ORDER BY U.NotesGatheredDateTime, U.PostDate DESC, U.BlogName, U.PostID;</sql><sql name="Del Posts">delete from posts where postid in 
+(
+'741662499571728384',
+178892849664,
+178264721139,
+177012868749,
+169950081964,
+755440787056099328
+)</sql><sql name="notes NO post*">select *
+-- delete
+from notes
+where postid not in (select distinct postid from posts where IsActive = 1)</sql><sql name="SQL 9">SELECT	
+	*
+FROM	
+	POSTS P
+WHERE	
+	P.ByLikes = 1
+	AND
+	P.DateCreated &gt; '2026-05-26 17:47:32'
+ORDER BY
+	P.DateCreated desc</sql><sql name="SQL 13">update posts set IsActive = 0 where blogname IN ( 'shoebiedoo', 'redheaded-girlygirl', 'xlittle-ghost' )</sql><sql name="SQL 14*">update Posts
+set IsActive = 0
+where postid in
+(
+
+
+'731937314675310592'
+
+
+
+)
+
+</sql><current_tab id="7"/></tab_sql></sqlb_project>

TL.sqbpro

@@ -9,16 +9,22 @@ WHERE
 ORDER BY
 	postdate desc</sql><sql name="SQL 2*">SELECT
 	datetime(TimeStamp, 'unixepoch'),
-	RootBlogName || '.tumblr.com/post/' || N.postid,
+	rbn.BlogName || '.tumblr.com/post/' || N.postid,
 	*,
-	NoteBlogName || '.tumblr.com'
+	nbn.BlogName || '.tumblr.com'
 FROM
 	Notes N
 	inner JOIN
-	Posts P on P.PostID = N.PostID and P.BlogName = N.RootBlogName
+	BlogNames rbn on rbn.BlogId = N.RootBlogId
-WHERE	RootBlogName NOT IN ('xlittle-ghost', 'glimmerin-darlin', 'vvenus-child')␍
+	inner JOIN
-		and type like 'r%'␍
+	BlogNames nbn on nbn.BlogId = N.NoteBlogId
-		and RootBlogName = 'zomb-eh'␍
+	inner JOIN
+	NoteTypes t on t.TypeId = N.TypeId
+	inner JOIN
+	Posts P on P.PostID = N.PostID and P.BlogName = rbn.BlogName
+WHERE	rbn.BlogName NOT IN ('xlittle-ghost', 'glimmerin-darlin', 'vvenus-child')
+		and t.Type like 'r%'
+		and rbn.BlogName = 'zomb-eh'
 		and P.HasImage = 1
 ORDER BY
 	TimeStamp desc</sql><current_tab id="1"/></tab_sql></sqlb_project>

URLNotesGrabberCORE/DataAccess.cs

@@ -213,6 +213,59 @@ namespace URLNotesGrabberCORE
 
         #endregion IsActive
 
+        #region Notes integer schema
+
+        // Notes stopped storing names on 2026-08-07: RootBlogName/NoteBlogName/Type became
+        // RootBlogId/NoteBlogId/TypeId, resolved through BlogNames and NoteTypes. There is no
+        // compatibility view -- a query naming an old column fails outright, so this is a hard
+        // cut rather than an optional column like IsActive. See TL.db.md.
+        //
+        // Two shapes recur below and are spelled out inline rather than hidden behind a helper,
+        // so that every statement reads as the SQL it actually runs:
+        //   (SELECT BlogId FROM BlogNames WHERE BlogName = @name)   -- unique-index probe, 20k rows
+        //   (SELECT TypeId FROM NoteTypes WHERE Type = 'reply')     -- 5 rows, effectively free
+        // Joining Notes to Blogs is the one case that must NOT route through BlogNames: Blogs
+        // carries its own BlogId, so N.NoteBlogId = B.BlogId is a single integer hop. Joining
+        // Notes to Posts is the opposite case -- Posts has only BlogName, so it has to go
+        // through BlogNames.
+
+        /// <summary>
+        /// True when the exception is a duplicate-key collision on Notes. The message embeds the
+        /// primary key's column names, which the integer migration renamed, so this matches on the
+        /// constraint and the table instead of on an exact column list -- a literal comparison
+        /// silently inverts into "log every error" the next time a column is renamed.
+        /// </summary>
+        private static bool IsNotesDuplicateKey(Exception ex)
+        {
+            return ex.Message.Contains("UNIQUE constraint failed", StringComparison.OrdinalIgnoreCase)
+                && ex.Message.Contains("Notes.", StringComparison.OrdinalIgnoreCase);
+        }
+
+        /// <summary>
+        /// Gives a blog name an ID if it does not have one. No read-back and no round trip -- a
+        /// name that is already registered keeps the ID that 1.18M Notes rows point at.
+        /// </summary>
+        private static void RegisterBlogName(SQLiteConnection connection, SQLiteTransaction? transaction, string blogName)
+        {
+            using SQLiteCommand command = new SQLiteCommand("INSERT OR IGNORE INTO BlogNames (BlogName) VALUES (@BlogName)", connection, transaction);
+            command.Parameters.AddWithValue("@BlogName", blogName);
+            command.ExecuteNonQuery();
+        }
+
+        /// <summary>
+        /// Same, for a note type. NoteTypes is a table rather than a CHECK constraint precisely so
+        /// that a type this crawler has not seen before is an INSERT and not a schema migration --
+        /// without this the type would resolve to NULL and fail the NOT NULL on Notes.TypeId.
+        /// </summary>
+        private static void RegisterNoteType(SQLiteConnection connection, SQLiteTransaction? transaction, string type)
+        {
+            using SQLiteCommand command = new SQLiteCommand("INSERT OR IGNORE INTO NoteTypes (Type) VALUES (@Type)", connection, transaction);
+            command.Parameters.AddWithValue("@Type", type);
+            command.ExecuteNonQuery();
+        }
+
+        #endregion Notes integer schema
+
         public static string Q(string input)
         {
             return "'" + input.Replace("'", "''") + "'";
@@ -371,7 +424,10 @@ namespace URLNotesGrabberCORE
                             connection.Close();
                             connection.Open();
 
-                            string addColumnSql = "ALTER TABLE Notes ADD COLUMN replyText TEXT DEFAULT '.';";
+                            // No column default: the migrated schema dropped the DEFAULT '.' that
+                            // is how 1.1M rows acquired a placeholder nobody wrote. New rows get
+                            // NULL, which every reader here already treats as "no reply text".
+                            string addColumnSql = "ALTER TABLE Notes ADD COLUMN replyText TEXT;";
                             using (SQLiteCommand addCommand = new SQLiteCommand(addColumnSql, connection))
                             {
                                 addCommand.ExecuteNonQuery();
@@ -534,12 +590,16 @@ namespace URLNotesGrabberCORE
             }
         }
 
-        public static void AddPost(string blogName, long postID, string reblogURL, string postDate, string postURL, string slug, string reblogKey, string reblogName, string summary, string quote, string body, string tags, string link, string photoURL, string photoCaption, string downloadedFiles, string audioCaption, string question, string answer, string title, bool hasImage, bool byLikes = false, string? DBPath = null, string? rootBlogName = null, string? rootURL = null)
+        // postType: a canonical PostTypes name, or null when the caller has no trustworthy type.
+        // Null is stored as NULL rather than guessed at -- OutputMode skips untyped rows, so a
+        // null costs one export line, whereas a wrong value would create a wrongly named file.
+        public static void AddPost(string blogName, long postID, string reblogURL, string postDate, string postURL, string slug, string reblogKey, string reblogName, string summary, string quote, string body, string tags, string link, string photoURL, string photoCaption, string downloadedFiles, string audioCaption, string question, string answer, string title, bool hasImage, bool byLikes = false, string? DBPath = null, string? rootBlogName = null, string? rootURL = null, string? postType = null)
         {
             DBPath ??= GetDefaultDbPath();
+            postType = PostTypes.Normalize(postType);
             try { AddBlog(blogName, byLikes, DBPath); } catch { }
             try { UpdatePostSetDate(blogName, postID, postDate, DBPath); } catch { }
-            try { UpdatePost(blogName, postID, reblogURL, postDate, postURL, slug, reblogKey, reblogName, summary, quote, body, tags, link, photoURL, photoCaption, downloadedFiles, audioCaption, question, answer, title, hasImage, byLikes, DBPath, rootBlogName, rootURL); } catch { }
+            try { UpdatePost(blogName, postID, reblogURL, postDate, postURL, slug, reblogKey, reblogName, summary, quote, body, tags, link, photoURL, photoCaption, downloadedFiles, audioCaption, question, answer, title, hasImage, byLikes, DBPath, rootBlogName, rootURL, postType); } catch { }
 
             SQLiteConnection connection;
             bool ownsConnection;
@@ -581,9 +641,10 @@ namespace URLNotesGrabberCORE
                                 RootBlogName,
                                 RootURL,
                                 HasImage,
-                                ByLikes
+                                ByLikes,
+                                PostType
                                 ) VALUES (" +
-                    Q(blogName) + ", " + postID + ", " + Q(reblogURL) + ", " + Q(postDate) + ", " + Q(postURL) + ", " + Q(slug) + ", " + Q(reblogKey) + ", " + Q(reblogName) + ", " + Q(summary) + ", " + Q(quote) + ", " + Q(body) + ", " + Q(tags) + ", " + Q(link) + ", " + Q(photoURL) + ", " + Q(photoCaption) + ", " + Q(downloadedFiles) + ", " + Q(audioCaption) + ", " + Q(question) + ", " + Q(answer) + ", " + Q(title) + ", " + Q(DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss")) + ", " + Q(DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss")) + ", " + Q(rootBlogName ?? ".") + ", " + Q(rootURL ?? ".") + ", " + (hasImage ? 1 : 0) + ", " + (byLikes ? 1 : 0) + ")";
+                    Q(blogName) + ", " + postID + ", " + Q(reblogURL) + ", " + Q(postDate) + ", " + Q(postURL) + ", " + Q(slug) + ", " + Q(reblogKey) + ", " + Q(reblogName) + ", " + Q(summary) + ", " + Q(quote) + ", " + Q(body) + ", " + Q(tags) + ", " + Q(link) + ", " + Q(photoURL) + ", " + Q(photoCaption) + ", " + Q(downloadedFiles) + ", " + Q(audioCaption) + ", " + Q(question) + ", " + Q(answer) + ", " + Q(title) + ", " + Q(DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss")) + ", " + Q(DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss")) + ", " + Q(rootBlogName ?? ".") + ", " + Q(rootURL ?? ".") + ", " + (hasImage ? 1 : 0) + ", " + (byLikes ? 1 : 0) + ", " + (postType == null ? "NULL" : Q(postType)) + ")";
                 SQLiteCommand command = new SQLiteCommand(sql, connection);
 
                 int rowsInserted = 0;
@@ -703,15 +764,34 @@ namespace URLNotesGrabberCORE
             try { AddBlog(noteBlogName, false, DBPath); } catch { }
 
             using SQLiteConnection connection2 = new SQLiteConnection("Data Source=" + DBPath);
+            int rowsInserted = 0;
 
             try
             {
                 connection2.Open();
 
+                // Notes stores integer IDs, so both participants and the type have to exist in
+                // their lookup table before the note can point at them.
+                //
+                // All four statements run in one transaction so a crash cannot leave a name or a
+                // type registered with no note. The transaction is committed before the console
+                // output below, which sleeps -- a write lock must not be held across that.
+                using (SQLiteTransaction transaction = connection2.BeginTransaction())
+                {
+                    RegisterBlogName(connection2, transaction, rootBlogName);
+                    RegisterBlogName(connection2, transaction, noteBlogName);
+                    RegisterNoteType(connection2, transaction, type ?? string.Empty);
+
                     // INSERT OR IGNORE, and no IsActive in the column list: re-crawling a note
                     // that was removed elsewhere leaves the existing row -- and its flag -- alone.
-                string sql = "INSERT OR IGNORE INTO Notes (rootBlogName, noteBlogName, PostID, TimeStamp, Type, DatetimeCrawled, DateModified, DateCreated) values(@rootBlogName, @noteBlogName, @PostID, @TimeStamp, @Type, @DatetimeCrawled, @DateModified, @DateCreated)";
+                    string sql = "INSERT OR IGNORE INTO Notes (RootBlogId, NoteBlogId, PostID, TimeStamp, TypeId, DatetimeCrawled, DateModified, DateCreated) " +
-                using (SQLiteCommand command = new SQLiteCommand(sql, connection2))
+                                 "SELECT (SELECT BlogId FROM BlogNames WHERE BlogName = @rootBlogName), " +
+                                 "       (SELECT BlogId FROM BlogNames WHERE BlogName = @noteBlogName), " +
+                                 "       @PostID, @TimeStamp, " +
+                                 "       (SELECT TypeId FROM NoteTypes WHERE Type = @Type), " +
+                                 "       @DatetimeCrawled, @DateModified, @DateCreated";
+
+                    using (SQLiteCommand command = new SQLiteCommand(sql, connection2, transaction))
                     {
                         command.Parameters.AddWithValue("@rootBlogName", rootBlogName);
                         command.Parameters.AddWithValue("@noteBlogName", noteBlogName);
@@ -722,7 +802,11 @@ namespace URLNotesGrabberCORE
                         command.Parameters.AddWithValue("@DateModified", DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss"));
                         command.Parameters.AddWithValue("@DateCreated", DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss"));
 
-                    int rowsInserted = command.ExecuteNonQuery();
+                        rowsInserted = command.ExecuteNonQuery();
+                    }
+
+                    transaction.Commit();
+                }
 
                 if (rowsInserted == 1)
                 {
@@ -730,7 +814,7 @@ namespace URLNotesGrabberCORE
                     Console.ForegroundColor = ConsoleColor.Green;
                     Console.WriteLine("{2}\t{0}\t{1}", UnixTimeStampToDateTime(timestamp), noteBlogName, type);
                     Console.ForegroundColor = previousColor;
-                        Thread.Sleep(250); // Brief pause to make new notes more noticeable in the console output
+                    Thread.Sleep(125); // Brief pause to make new notes more noticeable in the console output
                 }
                 else
                 {
@@ -755,11 +839,10 @@ namespace URLNotesGrabberCORE
                     catch { }
                 }
             }
-            }
             catch (Exception ex)
             {
                 // Breakpoint here
-                if (ex.Message != "constraint failed\r\nUNIQUE constraint failed: Notes.RootBlogName, Notes.PostID, Notes.TimeStamp, Notes.Type, Notes.NoteBlogName")
+                if (!IsNotesDuplicateKey(ex))
                 {
                     Console.WriteLine(ex.Message);
                     Console.WriteLine("^^^^^ - SHORTCUT");
@@ -775,14 +858,21 @@ namespace URLNotesGrabberCORE
         ///
         /// </summary>
         /// <param name="withoutNotesOnly"></param>
+        /// <param name="ignoreRefreshCooldown">Drops the age gate on the periodic re-queue branch (--force).</param>
+        /// <param name="fromDate">Lower bound on the *original post's* PostDate for the periodic re-queue branch (--fromDate). Independent of ignoreRefreshCooldown -- applies whether or not --force is also given.</param>
+        /// <param name="toDate">Upper bound on the *original post's* PostDate for the periodic re-queue branch (--toDate). Same independence from ignoreRefreshCooldown as fromDate.</param>
         /// <param name="DBPath"></param>
         /// <returns>blogName, postID, lastNoteTimestamp, notesGatheredTimestamp</returns>
-        public static List<Tuple<string, long, long, long>> GetPosts(bool withoutNotesOnly = false, DateTime? beforeDate = null, string? DBPath = null)
+        public static List<Tuple<string, long, long, long>> GetPosts(bool withoutNotesOnly = false, DateTime? beforeDate = null, string? blogName = null, bool ignoreRefreshCooldown = false, DateTime? fromDate = null, DateTime? toDate = null, string? DBPath = null)
         {
             DBPath ??= GetDefaultDbPath();
             using SQLiteConnection connection = new SQLiteConnection("Data Source=" + DBPath);
             List<Tuple<string, long, long, long>> posts = new List<Tuple<string, long, long, long>>();
 
+            // A blog filter matches BlogName exactly: the column is BINARY-collated and leads the
+            // Posts primary key, so "= @blogName" rides that index instead of scanning 1.18M rows.
+            bool filterByBlog = !string.IsNullOrWhiteSpace(blogName);
+
             try
             {
                 connection.Open();
@@ -798,31 +888,49 @@ namespace URLNotesGrabberCORE
                         beforeDateFilter = $"WHERE (U.NotesGatheredDateTime < {unixTimestamp} OR U.NotesGatheredDateTime IS NULL)" + Environment.NewLine;
                     }
 
-                    sql = "WITH PostsWithCount AS" + Environment.NewLine +
+                    // Blogs.IsActive is the crawler's work-selection flag (Rolodex removal sets it to 0)
-                          "(" + Environment.NewLine +
+                    // and is independent of Posts.IsActive/Notes.IsActive -- deactivating a blog does not
-                          "    SELECT " + Environment.NewLine +
+                    // touch its posts' own IsActive column. AndIsActive("Posts", ...) above therefore does
-                          "        P.BlogName," + Environment.NewLine +
+                    // not catch a deactivated blog; this join against the source rows is what does, so a
-                          "        P.PostID," + Environment.NewLine +
+                    // blog taken IsActive = 0 in Blogs stops being re-queued by --collect 1 even if its
-                          "        1925013599 AS LatestNoteTimestamp," + Environment.NewLine +
+                    // posts were never individually marked inactive. Blogs.BlogName is that table's PRIMARY
-                          "        P.NotesGatheredDateTime," + Environment.NewLine +
+                    // KEY, so the join rides an index rather than scanning it.
-                          "        COUNT(P.PostID) OVER(PARTITION BY P.BlogName) AS CNT," + Environment.NewLine +
+                    //
-                          "        P.HasNotesGathered," + Environment.NewLine +
+                    // Same WHERE/AND juggling WhereIsActive does, extended to the optional blog predicate:
-                          "        P.NotFound," + Environment.NewLine +
+                    // either clause may be absent, so the first one present has to open the WHERE.
-                          "        P.PostDate" + Environment.NewLine +
+                    string sourceClause = AndIsActive("Posts", "P", DBPath) + " AND COALESCE(BL.IsActive, 1) = 1" + (filterByBlog ? " AND P.BlogName = @blogName" : string.Empty);
-                          "    FROM Posts P" + WhereIsActive("Posts", "P", DBPath) + Environment.NewLine +
+                    string sourceFilter = sourceClause.Length == 0 ? string.Empty : " WHERE" + sourceClause.Substring(" AND".Length);
-                          ")," + Environment.NewLine +
+
-                          "Unioned AS" + Environment.NewLine +
+                    // The zomb-eh branch re-queues that blog's *already collected* posts every 3 days. It needs
-                          "(" + Environment.NewLine +
+                    // no blog-filter or IsActive handling of its own: it reads PostsWithCount, which the source
-                          "    SELECT " + Environment.NewLine +
+                    // filter above -- Blogs.IsActive included -- has already scoped, so it contributes its rows
-                          "        BlogName," + Environment.NewLine +
+                    // only when zomb-eh itself is still IsActive = 1 there. That keeps a filtered worklist a
-                          "        PostID," + Environment.NewLine +
+                    // strict subset of the unfiltered one -- "--collect 1 X" returns exactly the rows
-                          "        LatestNoteTimestamp," + Environment.NewLine +
+                    // "--collect 1" would have returned for X.
-                          "        NotesGatheredDateTime," + Environment.NewLine +
+                    //
-                          "        CNT," + Environment.NewLine +
+                    // --force drops the age gate only. NotFound = 0 and the IsActive/blog scoping above still
-                          "        PostDate" + Environment.NewLine +
+                    // apply: the flag is "re-collect early", not "collect rows every other path excludes".
-                          "    FROM PostsWithCount" + Environment.NewLine +
+                    string refreshCooldownClause = ignoreRefreshCooldown
-                          "    WHERE NotFound = 0" + Environment.NewLine +
+                        ? string.Empty
-                          "      AND HasNotesGathered = 0" + Environment.NewLine +
+                        : "      AND NotesGatheredDateTime < unixepoch('now', 'localtime', '-3 days')" + Environment.NewLine;
+
+                    // --fromDate bounds the *original post's* PostDate, not the re-collect cooldown --
+                    // it stacks with refreshCooldownClause instead of replacing it, so it applies the
+                    // same way whether or not --force also dropped the cooldown. A NULL PostDate never
+                    // satisfies ">=" and is excluded, same as an unfiltered run would still include it
+                    // (there's nothing to compare here, so this only narrows, never widens, the result).
+                    string fromDateClause = fromDate.HasValue
+                        ? "      AND PostDate >= @fromDate" + Environment.NewLine
+                        : string.Empty;
+
+                    // --toDate is the same deal, mirrored: stacks alongside fromDateClause/
+                    // refreshCooldownClause rather than replacing either, so --fromDate and --toDate
+                    // can be given together (or alone) and both hold with or without --force.
+                    string toDateClause = toDate.HasValue
+                        ? "      AND PostDate <= @toDate" + Environment.NewLine
+                        : string.Empty;
+
+                    string refreshBranch =
                           "" + Environment.NewLine +
                           "    UNION " + Environment.NewLine +
                           "" + Environment.NewLine +
@@ -836,7 +944,36 @@ namespace URLNotesGrabberCORE
                           "    FROM PostsWithCount" + Environment.NewLine +
                           "    WHERE BlogName = 'zomb-eh'" + Environment.NewLine +
                           "      AND NotFound = 0" + Environment.NewLine +
-                          "      AND NotesGatheredDateTime < unixepoch('now', 'localtime', '-3 days')" + Environment.NewLine +
+                          refreshCooldownClause +
+                          fromDateClause +
+                          toDateClause;
+
+                    sql = "WITH PostsWithCount AS" + Environment.NewLine +
+                          "(" + Environment.NewLine +
+                          "    SELECT " + Environment.NewLine +
+                          "        P.BlogName," + Environment.NewLine +
+                          "        P.PostID," + Environment.NewLine +
+                          "        1925013599 AS LatestNoteTimestamp," + Environment.NewLine +
+                          "        P.NotesGatheredDateTime," + Environment.NewLine +
+                          "        COUNT(P.PostID) OVER(PARTITION BY P.BlogName) AS CNT," + Environment.NewLine +
+                          "        P.HasNotesGathered," + Environment.NewLine +
+                          "        P.NotFound," + Environment.NewLine +
+                          "        P.PostDate" + Environment.NewLine +
+                          "    FROM Posts P LEFT JOIN Blogs BL ON BL.BlogName = P.BlogName" + sourceFilter + Environment.NewLine +
+                          ")," + Environment.NewLine +
+                          "Unioned AS" + Environment.NewLine +
+                          "(" + Environment.NewLine +
+                          "    SELECT " + Environment.NewLine +
+                          "        BlogName," + Environment.NewLine +
+                          "        PostID," + Environment.NewLine +
+                          "        LatestNoteTimestamp," + Environment.NewLine +
+                          "        NotesGatheredDateTime," + Environment.NewLine +
+                          "        CNT," + Environment.NewLine +
+                          "        PostDate" + Environment.NewLine +
+                          "    FROM PostsWithCount" + Environment.NewLine +
+                          "    WHERE NotFound = 0" + Environment.NewLine +
+                          "      AND HasNotesGathered = 0" + Environment.NewLine +
+                          refreshBranch +
                           ")" + Environment.NewLine +
                           "SELECT" + Environment.NewLine +
                           "    U.BlogName," + Environment.NewLine +
@@ -850,6 +987,10 @@ namespace URLNotesGrabberCORE
                 }
                 else
                 {
+                    // The LEFT OUTER JOIN to Notes that used to sit here has been dropped rather
+                    // than ported. Nothing was selected from it, a LEFT JOIN cannot remove a row,
+                    // and the GROUP BY below collapsed the rows it duplicated -- so it could not
+                    // affect the result, and it cost a join against 1.18M rows on every pass.
                     sql = "SELECT " +
                                  "  MAX(Posts.BlogName) as BlogName, " + Environment.NewLine +
                                  "  Posts.PostID, " + Environment.NewLine +
@@ -859,11 +1000,14 @@ namespace URLNotesGrabberCORE
                                  "FROM " + Environment.NewLine +
                                  "   Posts " + Environment.NewLine +
                                  "      LEFT OUTER JOIN " + Environment.NewLine +
-                                 "   Notes ON Notes.RootBlogName = Posts.BlogName AND Notes.PostID = Posts.PostID " + Environment.NewLine +
-                                 "      LEFT OUTER JOIN " + Environment.NewLine +
                                  "   (      select BlogName, count(PostID) as CNT from Posts" + WhereIsActive("Posts", "", DBPath) + " group by BlogName)  CNT on CNT.blogName = Posts.BlogName " +
                                  "WHERE NotFound = 0 " + AndIsActive("Posts", "Posts", DBPath) + Environment.NewLine;
 
+                    if (filterByBlog)
+                    {
+                        sql += " AND Posts.BlogName = @blogName " + Environment.NewLine;
+                    }
+
                     if (beforeDate.HasValue)
                     {
                         long unixTimestamp = new DateTimeOffset(beforeDate.Value).ToUnixTimeSeconds();
@@ -883,6 +1027,16 @@ namespace URLNotesGrabberCORE
 
                 using (SQLiteCommand command = new SQLiteCommand(sql, connection))
                 {
+                    if (filterByBlog)
+                        command.Parameters.AddWithValue("@blogName", blogName);
+
+                    // Only ever referenced by the zomb-eh refresh branch, which only exists when
+                    // withoutNotesOnly is true -- harmless to bind unconditionally otherwise.
+                    if (fromDate.HasValue)
+                        command.Parameters.AddWithValue("@fromDate", fromDate.Value.ToString("yyyy-MM-dd HH:mm:ss"));
+                    if (toDate.HasValue)
+                        command.Parameters.AddWithValue("@toDate", toDate.Value.ToString("yyyy-MM-dd HH:mm:ss"));
+
                     using (SQLiteDataReader reader = command.ExecuteReader())
                     {
                         while (reader.Read())
@@ -971,7 +1125,11 @@ namespace URLNotesGrabberCORE
             {
                 connection.Open();
 
-                string sql = "SELECT	distinct RootBlogName as blogName, postID FROM Notes WHERE Notes.type = 'reply'" + AndIsActive("Notes", "Notes", DBPath) + " order by RootBlogName, PostID";
+                string sql = "SELECT DISTINCT BN.BlogName as blogName, N.PostID" +
+                             " FROM Notes N" +
+                             " INNER JOIN BlogNames BN ON BN.BlogId = N.RootBlogId" +
+                             " WHERE N.TypeId = (SELECT TypeId FROM NoteTypes WHERE Type = 'reply')" + AndIsActive("Notes", "N", DBPath) +
+                             " ORDER BY BN.BlogName, N.PostID";
 
                 using (SQLiteCommand command = new SQLiteCommand(sql, connection))
                 {
@@ -1011,12 +1169,15 @@ namespace URLNotesGrabberCORE
             {
                 connection.Open();
 
-                string sql = @"SELECT DISTINCT Notes.RootBlogName as blogName, Notes.PostID,
+                // Grouped on the integer rather than the name: the group key is what gets sorted,
-                             MAX(Notes.timestamp) as LatestTimestamp
+                // and BN.BlogName comes along for free off the join.
-                             FROM Notes 
+                string sql = @"SELECT BN.BlogName as blogName, N.PostID,
-                             WHERE Notes.type = 'reply'
+                             MAX(N.TimeStamp) as LatestTimestamp
-                             AND (Notes.replyText IS NULL OR Notes.replyText = '' OR Notes.replyText = '.')" + AndIsActive("Notes", "Notes", DBPath) + @"
+                             FROM Notes N
-                             GROUP BY Notes.RootBlogName, Notes.PostID
+                             INNER JOIN BlogNames BN ON BN.BlogId = N.RootBlogId
+                             WHERE N.TypeId = (SELECT TypeId FROM NoteTypes WHERE Type = 'reply')
+                             AND (N.replyText IS NULL OR N.replyText = '' OR N.replyText = '.')" + AndIsActive("Notes", "N", DBPath) + @"
+                             GROUP BY N.RootBlogId, N.PostID
                              ORDER BY LatestTimestamp ASC
                              LIMIT @limit";
 
@@ -1058,11 +1219,15 @@ namespace URLNotesGrabberCORE
             {
                 connection.Open();
 
-                string sql = @"SELECT DISTINCT P.BlogName, P.PostID, MAX(N.timestamp) as LatestTimestamp
+                // Posts carries only BlogName, so this is the one join to Notes that has to go
+                // through BlogNames -- there is no Posts.BlogId to hop on. The name predicate is
+                // pushed into the 20k-row lookup, which then feeds integers to the Notes key.
+                string sql = @"SELECT DISTINCT P.BlogName, P.PostID, MAX(N.TimeStamp) as LatestTimestamp
                               FROM Posts P
-                              INNER JOIN Notes N ON N.PostID = P.PostID AND N.RootBlogName = P.BlogName
+                              INNER JOIN BlogNames RBN ON RBN.BlogName = P.BlogName
+                              INNER JOIN Notes N ON N.RootBlogId = RBN.BlogId AND N.PostID = P.PostID
                               WHERE P.NotFound = 0
-                                AND N.type = 'reply'
+                                AND N.TypeId = (SELECT TypeId FROM NoteTypes WHERE Type = 'reply')
                                 AND (N.replyText IS NULL OR N.replyText = '' OR N.replyText = '.')" + AndIsActive("Posts", "P", DBPath) + AndIsActive("Notes", "N", DBPath) + @"
                               GROUP BY P.BlogName, P.PostID
                               ORDER BY LatestTimestamp ASC";
@@ -1160,6 +1325,11 @@ namespace URLNotesGrabberCORE
             {
                 connection.Open();
                 string sql;
+
+                // The two Notes branches below join on Blogs.BlogId, which is NULL for the 168k
+                // registry rows that have never appeared in a note. The inner join drops them,
+                // which is correct here -- both branches already require a note to exist -- but
+                // it is the wrong shape for anything that lists the registry.
                 if (!string.IsNullOrEmpty(specificBlog))
                 {
                     // Specific blog: always process, bypass cooldown
@@ -1179,12 +1349,12 @@ namespace URLNotesGrabberCORE
                                    COALESCE(B.LikesCursor, 0),
                                    COALESCE(B.LikesNewestTimestamp, 0)
                               FROM Blogs B
-                              INNER JOIN Notes N ON N.NoteBlogName = B.BlogName
+                              INNER JOIN Notes N ON N.NoteBlogId = B.BlogId
                              WHERE N.TimeStamp >= 1535778000
-                               AND N.rootBlogName = B.BlogName
+                               AND N.RootBlogId = B.BlogId
                                AND B.IsActive = 1" + AndIsActive("Notes", "N", DBPath) + @"
                              GROUP BY B.BlogName
-                             ORDER BY MIN(N.Timestamp);";
+                             ORDER BY MIN(N.TimeStamp);";
                 }
                 else
                 {
@@ -1194,9 +1364,9 @@ namespace URLNotesGrabberCORE
                                    COALESCE(B.LikesCursor, 0),
                                    COALESCE(B.LikesNewestTimestamp, 0)
                               FROM Blogs B
-                              INNER JOIN Notes N ON N.NoteBlogName = B.BlogName
+                              INNER JOIN Notes N ON N.NoteBlogId = B.BlogId
                              WHERE N.TimeStamp >= 1535778000
-                               AND N.rootBlogName = B.BlogName
+                               AND N.RootBlogId = B.BlogId
                                AND B.IsActive = 1" + AndIsActive("Notes", "N", DBPath) + @"
                                AND (
                                     B.LikesPulled = 0
@@ -1204,7 +1374,7 @@ namespace URLNotesGrabberCORE
                                     < (CAST(strftime('%s','now') AS INTEGER) - (@cooldownDays * 86400))
                                )
                              GROUP BY B.BlogName
-                             ORDER BY MIN(N.Timestamp);";
+                             ORDER BY MIN(N.TimeStamp);";
                 }
 
                 using (SQLiteCommand command = new SQLiteCommand(sql, connection))
@@ -1244,11 +1414,14 @@ namespace URLNotesGrabberCORE
             try
             {
                 connection.Open();
+                // Blogs is reached in one integer hop off Blogs.BlogId, not through BlogNames --
+                // that would add a hop and end in the text comparison the migration removed.
+                // The negated form is only correct because Notes.TypeId is NOT NULL.
                 string sql = "";
                 if (reblogsOnly)
-                    sql = "SELECT NoteBlogName as blogName, count(*) FROM notes INNER JOIN blogs ON blogs.BlogName = notes.NoteBlogName WHERE blogs.IsActive = @isActive" + AndIsActive("Notes", "notes", DBPath) + " AND type IN ('reblog', 'reply', 'posted') AND HasBeenOutput = 0 GROUP BY NoteBlogName ORDER BY count(*) DESC, BlogName LIMIT @top";
+                    sql = "SELECT B.BlogName as blogName, count(*) FROM Notes N INNER JOIN Blogs B ON B.BlogId = N.NoteBlogId WHERE B.IsActive = @isActive" + AndIsActive("Notes", "N", DBPath) + " AND N.TypeId IN (SELECT TypeId FROM NoteTypes WHERE Type IN ('reblog', 'reply', 'posted')) AND B.HasBeenOutput = 0 GROUP BY N.NoteBlogId ORDER BY count(*) DESC, B.BlogName LIMIT @top";
                 else
-                    sql = "SELECT NoteBlogName as blogName, count(*) FROM notes INNER JOIN blogs ON blogs.BlogName = notes.NoteBlogName WHERE blogs.IsActive = @isActive" + AndIsActive("Notes", "notes", DBPath) + " AND type NOT IN ('reblog', 'reply', 'posted') AND HasBeenOutput = 0 GROUP BY NoteBlogName ORDER BY count(*) DESC, BlogName LIMIT @top";
+                    sql = "SELECT B.BlogName as blogName, count(*) FROM Notes N INNER JOIN Blogs B ON B.BlogId = N.NoteBlogId WHERE B.IsActive = @isActive" + AndIsActive("Notes", "N", DBPath) + " AND N.TypeId NOT IN (SELECT TypeId FROM NoteTypes WHERE Type IN ('reblog', 'reply', 'posted')) AND B.HasBeenOutput = 0 GROUP BY N.NoteBlogId ORDER BY count(*) DESC, B.BlogName LIMIT @top";
 
                 using (SQLiteCommand command = new SQLiteCommand(sql, connection))
                 {
@@ -1287,9 +1460,9 @@ namespace URLNotesGrabberCORE
                 connection.Open();
                 string sql = "";
                 if (reblogsOnly)
-                    sql = "SELECT NoteBlogName as blogName, count(*) FROM notes INNER JOIN blogs ON blogs.BlogName = notes.NoteBlogName WHERE blogs.IsActive = @isActive" + AndIsActive("Notes", "notes", DBPath) + " AND type IN ('reblog', 'reply', 'posted') AND HasBeenOutput = 0 GROUP BY NoteBlogName ORDER BY count(*) DESC, BlogName LIMIT @top";
+                    sql = "SELECT B.BlogName as blogName, count(*) FROM Notes N INNER JOIN Blogs B ON B.BlogId = N.NoteBlogId WHERE B.IsActive = @isActive" + AndIsActive("Notes", "N", DBPath) + " AND N.TypeId IN (SELECT TypeId FROM NoteTypes WHERE Type IN ('reblog', 'reply', 'posted')) AND B.HasBeenOutput = 0 GROUP BY N.NoteBlogId ORDER BY count(*) DESC, B.BlogName LIMIT @top";
                 else
-                    sql = "SELECT NoteBlogName as blogName, count(*) FROM notes INNER JOIN blogs ON blogs.BlogName = notes.NoteBlogName WHERE blogs.IsActive = @isActive" + AndIsActive("Notes", "notes", DBPath) + " AND HasBeenOutput = 0 GROUP BY NoteBlogName ORDER BY count(*) DESC, BlogName LIMIT @top";
+                    sql = "SELECT B.BlogName as blogName, count(*) FROM Notes N INNER JOIN Blogs B ON B.BlogId = N.NoteBlogId WHERE B.IsActive = @isActive" + AndIsActive("Notes", "N", DBPath) + " AND B.HasBeenOutput = 0 GROUP BY N.NoteBlogId ORDER BY count(*) DESC, B.BlogName LIMIT @top";
 
                 using (SQLiteCommand command = new SQLiteCommand(sql, connection))
                 {
@@ -1528,7 +1701,10 @@ namespace URLNotesGrabberCORE
             {
                 connection.Open();
 
-                string sql = "UPDATE Notes SET timestamp = @timestamp, DateModified = @dateModified WHERE rootBlogName = @rootBlogName AND noteBlogName = @noteBlogName AND PostID = @postID AND IFNULL(timestamp, 0) <> @timestamp";
+                string sql = "UPDATE Notes SET TimeStamp = @timestamp, DateModified = @dateModified " +
+                             "WHERE RootBlogId = (SELECT BlogId FROM BlogNames WHERE BlogName = @rootBlogName) " +
+                             "AND NoteBlogId = (SELECT BlogId FROM BlogNames WHERE BlogName = @noteBlogName) " +
+                             "AND PostID = @postID AND IFNULL(TimeStamp, 0) <> @timestamp";
                 using (SQLiteCommand command = new SQLiteCommand(sql, connection))
                 {
                     command.Parameters.AddWithValue("@timestamp", timestamp);
@@ -1542,7 +1718,7 @@ namespace URLNotesGrabberCORE
             catch (Exception ex)
             {
                 // Breakpoint here
-                if (ex.Message != "constraint failed\r\nUNIQUE constraint failed: Notes.RootBlogName, Notes.PostID, Notes.TimeStamp, Notes.Type, Notes.NoteBlogName")
+                if (!IsNotesDuplicateKey(ex))
                 {
                     Console.WriteLine(ex.Message);
                     Console.WriteLine("^^^^^ - SHORTCUT");
@@ -1552,9 +1728,10 @@ namespace URLNotesGrabberCORE
             return false;
         }
 
-        public static void UpdatePost(string blogName, long postID, string reblogURL, string postDate, string postURL, string slug, string reblogKey, string reblogName, string summary, string quote, string body, string tags, string link, string photoURL, string photoCaption, string downloadedFiles, string audioCaption, string question, string answer, string title, bool hasImage, bool byLikes = false, string? DBPath = null, string? rootBlogName = null, string? rootURL = null)
+        public static void UpdatePost(string blogName, long postID, string reblogURL, string postDate, string postURL, string slug, string reblogKey, string reblogName, string summary, string quote, string body, string tags, string link, string photoURL, string photoCaption, string downloadedFiles, string audioCaption, string question, string answer, string title, bool hasImage, bool byLikes = false, string? DBPath = null, string? rootBlogName = null, string? rootURL = null, string? postType = null)
         {
             DBPath ??= GetDefaultDbPath();
+            postType = PostTypes.Normalize(postType);
 
             SQLiteConnection connection;
             bool ownsConnection;
@@ -1602,6 +1779,10 @@ namespace URLNotesGrabberCORE
                 sql += "RootBlogName = CASE WHEN @rootBlogName IS NULL OR @rootBlogName = '' OR @rootBlogName = '.' THEN RootBlogName ELSE @rootBlogName END, ";
                 sql += "RootURL = CASE WHEN @rootURL IS NULL OR @rootURL = '' OR @rootURL = '.' THEN RootURL ELSE @rootURL END, ";
                 sql += "hasImage = @hasImage, ";
+                // Fill in a missing type, never overwrite one. A type derived by --ingest from a
+                // real export filename is authoritative; this path's type is only as good as the
+                // folder it was crawled from, so it must not win over an existing value.
+                sql += "PostType = IFNULL(PostType, @postType), ";
                 sql += "ByLikes = MAX(IFNULL(ByLikes, 0), @byLikes) ";
                 sql += " WHERE BlogName = @BlogName AND PostID = @PostID AND (";
                 sql += "(@postDate <> '.' AND IFNULL(postDate, '') <> @postDate) OR ";
@@ -1625,7 +1806,10 @@ namespace URLNotesGrabberCORE
                 sql += "IFNULL(hasImage, 0) <> @hasImage OR ";
                 sql += "(@byLikes = 1 AND IFNULL(ByLikes, 0) = 0) OR ";
                 sql += "((@rootBlogName IS NOT NULL AND @rootBlogName <> '' AND @rootBlogName <> '.') AND IFNULL(RootBlogName, '') <> @rootBlogName) OR ";
-                sql += "((@rootURL IS NOT NULL AND @rootURL <> '' AND @rootURL <> '.') AND IFNULL(RootURL, '') <> @rootURL)";
+                sql += "((@rootURL IS NOT NULL AND @rootURL <> '' AND @rootURL <> '.') AND IFNULL(RootURL, '') <> @rootURL) OR ";
+                // Without this the SET above is unreachable for a row whose content is already
+                // current: the UPDATE would not fire, and the type would stay NULL forever.
+                sql += "(PostType IS NULL AND @postType IS NOT NULL)";
                 sql += ")";
 
                 using (SQLiteCommand command = new SQLiteCommand(sql, connection))
@@ -1653,6 +1837,7 @@ namespace URLNotesGrabberCORE
                     command.Parameters.AddWithValue("@rootURL", string.IsNullOrWhiteSpace(rootURL) ? (object)DBNull.Value : rootURL);
                     command.Parameters.AddWithValue("@hasImage", hasImage ? 1 : 0);
                     command.Parameters.AddWithValue("@byLikes", byLikes ? 1 : 0);
+                    command.Parameters.AddWithValue("@postType", (object?)postType ?? DBNull.Value);
                     command.Parameters.AddWithValue("@BlogName", blogName);
                     command.Parameters.AddWithValue("@PostID", postID);
 
@@ -1828,10 +2013,15 @@ namespace URLNotesGrabberCORE
             {
                 connection.Open();
 
-                //string sql = "UPDATE Notes SET replyText = @replyText WHERE rootBlogName = @rootBlogName AND PostID = @PostID AND noteBlogName = @noteBlogName AND TimeStamp = @TimeStamp AND Type = 'reply'";
                 // Match on (noteBlogName, TimeStamp ±5s) only - a reply by a given blog at a given timestamp is the same reply across the original post and every reblog of it, so this fans out across reblog chains in one shot. Tolerance absorbs the ~1s drift between what -collect stored and what mode=conversation returns now.
                 // Only fan out to rows that match the SELECT criteria in GetRepliesWithFilledText (NULL/empty/legacy-'.'). Never overwrite '?' (confirmed-empty) or already-fetched text.
-                string sql = "UPDATE Notes SET replyText = @replyText, DateModified = @dateModified WHERE noteBlogName = @noteBlogName AND ABS(TimeStamp - @TimeStamp) <= 5 AND Type = 'reply' AND (replyText IS NULL OR replyText = '' OR replyText = '.') AND (replyText IS NULL OR replyText <> @replyText)";
+                // The ABS() term cannot use an index on TimeStamp, before or after the integer schema; the NoteBlogId probe is what keeps this off a full scan.
+                string sql = "UPDATE Notes SET replyText = @replyText, DateModified = @dateModified " +
+                             "WHERE NoteBlogId = (SELECT BlogId FROM BlogNames WHERE BlogName = @noteBlogName) " +
+                             "AND ABS(TimeStamp - @TimeStamp) <= 5 " +
+                             "AND TypeId = (SELECT TypeId FROM NoteTypes WHERE Type = 'reply') " +
+                             "AND (replyText IS NULL OR replyText = '' OR replyText = '.') " +
+                             "AND (replyText IS NULL OR replyText <> @replyText)";
                 using (SQLiteCommand command = new SQLiteCommand(sql, connection))
                 {
                     command.Parameters.AddWithValue("@replyText", replyText ?? "?");
@@ -1872,7 +2062,11 @@ namespace URLNotesGrabberCORE
             {
                 connection.Open();
 
-                string sql = "UPDATE Notes SET replyText = @replyText, DateModified = @dateModified WHERE rootBlogName = @rootBlogName AND PostID = @PostID AND Type = 'reply' AND IFNULL(replyText, '.') <> @replyText";
+                string sql = "UPDATE Notes SET replyText = @replyText, DateModified = @dateModified " +
+                             "WHERE RootBlogId = (SELECT BlogId FROM BlogNames WHERE BlogName = @rootBlogName) " +
+                             "AND PostID = @PostID " +
+                             "AND TypeId = (SELECT TypeId FROM NoteTypes WHERE Type = 'reply') " +
+                             "AND IFNULL(replyText, '.') <> @replyText";
                 using (SQLiteCommand command = new SQLiteCommand(sql, connection))
                 {
                     command.Parameters.AddWithValue("@replyText", replyText ?? ".");
@@ -1953,6 +2147,8 @@ namespace URLNotesGrabberCORE
                         cmd.ExecuteNonQuery();
                     Console.WriteLine("[Migration] Added PostType column to Posts table");
                 }
+
+                BackfillMissingPostTypes(connection);
             }
             catch (Exception ex)
             {
@@ -1960,6 +2156,65 @@ namespace URLNotesGrabberCORE
             }
         }
 
+        /// <summary>
+        /// Types rows that carry no PostType, inferring it from which content columns they hold.
+        ///
+        /// These are posts harvested from notes and likes rather than read out of a TumblThree
+        /// export, so no filename ever described them and --ingest can never reach them: it only
+        /// types a post it meets inside a real .txt. Content is the only signal they have.
+        ///
+        /// Runs on every migration pass and is idempotent -- it only touches PostType IS NULL,
+        /// so a row typed once is never revisited. Rows whose columns give no signal at all stay
+        /// NULL and are skipped by OutputMode.
+        ///
+        /// Mirrors PostTypes.InferFromContent; the two must agree. Notably HasImage is not
+        /// consulted, because most text posts carry it.
+        /// </summary>
+        private static void BackfillMissingPostTypes(SQLiteConnection connection)
+        {
+            const string set = @"
+                UPDATE Posts SET PostType = CASE
+                    WHEN Has(Question) AND Has(Answer) THEN 'answers'
+                    WHEN Has(Quote)                    THEN 'quotes'
+                    WHEN Has(Link)                     THEN 'links'
+                    WHEN Has(AudioCaption)             THEN 'audios'
+                    WHEN Has(Body)                     THEN 'texts'
+                    WHEN Has(PhotoURL) OR Has(PhotoCaption) THEN 'images'
+                    ELSE NULL END
+                WHERE PostType IS NULL";
+
+            // SQLite has no user-defined predicate here, so expand the "field supplied" test
+            // ("." is the not-supplied sentinel used throughout the export format) inline.
+            string sql = System.Text.RegularExpressions.Regex.Replace(
+                set, @"Has\((\w+)\)", "TRIM(IFNULL($1, '')) NOT IN ('', '.')");
+
+            try
+            {
+                long before;
+                using (var count = new SQLiteCommand("SELECT COUNT(*) FROM Posts WHERE PostType IS NULL", connection))
+                    before = Convert.ToInt64(count.ExecuteScalar());
+
+                if (before == 0) return;
+
+                int changed;
+                using (var cmd = new SQLiteCommand(sql, connection))
+                    changed = cmd.ExecuteNonQuery();
+
+                long after;
+                using (var count = new SQLiteCommand("SELECT COUNT(*) FROM Posts WHERE PostType IS NULL", connection))
+                    after = Convert.ToInt64(count.ExecuteScalar());
+
+                if (changed > 0 || after != before)
+                    Console.WriteLine($"[Migration] Backfilled PostType for {before - after} post(s); {after} still untyped (no content signal).");
+            }
+            catch (Exception ex)
+            {
+                // A failed backfill must not stop the run: untyped rows are skipped on export,
+                // which is inconvenient, not corrupting.
+                Console.WriteLine($"[Migration] PostType backfill failed: {ex.Message}");
+            }
+        }
+
         // INSERT-or-UPDATE for a post arriving from a Tumblr text-file export.
         // On collision, only content columns + PostType + DateModified are updated;
         // engagement columns (ByLikes, RootBlogName, RootURL, HasNotesGathered, NotFound,
@@ -1990,6 +2245,10 @@ namespace URLNotesGrabberCORE
             string? DBPath = null)
         {
             DBPath ??= GetDefaultDbPath();
+            // Central guarantee: whatever a caller believes, only a canonical type reaches the
+            // column. PostType is used as an output filename, so this is the invariant that keeps
+            // a stray value from becoming a stray file.
+            postType = PostTypes.Normalize(postType);
             try { AddBlog(blogName, false, DBPath); } catch { }
 
             SQLiteConnection connection;

URLNotesGrabberCORE/IngestMode.cs

@@ -85,7 +85,25 @@ namespace URLNotesGrabberCORE
                     {
                         string rawBlogName = Path.GetFileName(Path.GetDirectoryName(file) ?? "unknown");
                         string blogName = Regex.Replace(rawBlogName, @"_\d+$", "");
-                        string postType = Path.GetFileNameWithoutExtension(file);
+
+                        // The filename becomes the row's PostType, and PostType later becomes an
+                        // output filename -- so an unrecognized name here would mint a new type and
+                        // a new file from any stray .txt that happens to sit in the tree. Only the
+                        // eight real export files are ingestable.
+                        //
+                        // This is also what breaks the Unknown.txt cycle: OutputMode used to write
+                        // untyped rows to Unknown.txt, and this scan would read it straight back
+                        // and stamp those rows with the literal type "Unknown", making the file
+                        // regenerate itself forever.
+                        string? resolvedPostType = PostTypes.FromFileName(file);
+                        if (resolvedPostType == null)
+                        {
+                            filesSkipped++;
+                            continue;
+                        }
+                        // Non-nullable from here so the local Flush() below stays warning-clean:
+                        // nullable flow analysis does not reach into local functions.
+                        string postType = resolvedPostType;
 
                         if (targetBlog != null && !string.Equals(blogName, targetBlog, StringComparison.OrdinalIgnoreCase))
                         {

URLNotesGrabberCORE/LegacyPostsDbImporter.cs

@@ -117,7 +117,12 @@ namespace URLNotesGrabberCORE
                                 question: reader.IsDBNull(18) ? null : reader.GetString(18),
                                 answer: reader.IsDBNull(19) ? null : reader.GetString(19),
                                 title: reader.IsDBNull(20) ? null : reader.GetString(20),
-                                postType: reader.IsDBNull(21) ? null : reader.GetString(21),
+                                // A legacy Posts.db predating the PostType column hands back NULL
+                                // here, and on the INSERT branch that NULL is stored -- reseeding
+                                // exactly the untyped rows the backfill exists to clear. Normalize
+                                // so an unrecognized legacy value cannot become a filename either;
+                                // the backfill types whatever comes through as null.
+                                postType: PostTypes.Normalize(reader.IsDBNull(21) ? null : reader.GetString(21)),
                                 hasImage: hasImage);
                             postsUpserted++;
                             if (postsUpserted % 500 == 0)

URLNotesGrabberCORE/OutputMode.cs

@@ -65,10 +65,20 @@ namespace URLNotesGrabberCORE
                 var posts = DataAccess.GetAllPostsForBlog(blogName);
                 Console.WriteLine($"  Found {posts.Count} post(s) for this blog.");
 
-                var grouped = posts.GroupBy(p => p.PostType ?? "Unknown");
+                // A post's type becomes a filename, so only a recognized type may be written. The
+                // old `PostType ?? "Unknown"` invented Unknown.txt for untyped rows, which --ingest
+                // then read back as a type named "Unknown" -- the two regenerated each other.
+                // Untyped rows are skipped instead: after the backfill these are only rows with no
+                // content signal at all, so nothing meaningful is lost, and nothing is invented.
+                var typed = posts.Where(p => PostTypes.Normalize(p.PostType) != null).ToList();
+                int untyped = posts.Count - typed.Count;
+                if (untyped > 0)
+                    Console.WriteLine($"    Skipping {untyped} post(s) with no recognized PostType.");
+
+                var grouped = typed.GroupBy(p => PostTypes.Normalize(p.PostType)!);
                 foreach (var typeGroup in grouped)
                 {
-                    string postType = typeGroup.Key ?? "Unknown";
+                    string postType = typeGroup.Key;
                     string outputFilePath = Path.Combine(folder, $"{postType}.txt");
                     var ordered = typeGroup.OrderBy(p => p.Date).ToList();
                     Console.WriteLine($"    Writing {ordered.Count} post(s) to {postType}.txt");

URLNotesGrabberCORE/PostTypes.cs

@@ -0,0 +1,123 @@
+using System;
+using System.Collections.Generic;
+using System.IO;
+
+namespace URLNotesGrabberCORE
+{
+    /// <summary>
+    /// The single source of truth for Posts.PostType values.
+    ///
+    /// PostType exists so --output can write one .txt per type. Because the type becomes a
+    /// *filename*, an unvalidated value is not a cosmetic problem: it creates a file. That is
+    /// how "Unknown.txt" came about -- OutputMode used `PostType ?? "Unknown"` as a filename,
+    /// --ingest then read that file straight back and derived the literal type "Unknown" from
+    /// its name, and the pair would have kept regenerating each other indefinitely.
+    ///
+    /// So every path that produces a type routes through <see cref="Normalize"/>, which admits
+    /// only the eight known names and returns null for anything else. A null type is safe:
+    /// OutputMode skips those rows rather than inventing a file for them.
+    /// </summary>
+    public static class PostTypes
+    {
+        // The canonical set. These are exactly the TumblThree .txt basenames, which is what
+        // makes an ingested filename usable as a type without translation.
+        public const string Texts = "texts";
+        public const string Answers = "answers";
+        public const string Quotes = "quotes";
+        public const string Links = "links";
+        public const string Conversations = "conversations";
+        public const string Images = "images";
+        public const string Videos = "videos";
+        public const string Audios = "audios";
+
+        private static readonly HashSet<string> Known = new HashSet<string>(
+            new[] { Texts, Answers, Quotes, Links, Conversations, Images, Videos, Audios },
+            StringComparer.OrdinalIgnoreCase);
+
+        // Tumblr's legacy post format (npf=false) names types in the singular. The likes API is
+        // the one source that reports a type directly rather than via a filename, so it is the
+        // only place this mapping is needed.
+        private static readonly Dictionary<string, string> ApiTypeMap = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
+        {
+            ["text"] = Texts,
+            ["photo"] = Images,
+            ["quote"] = Quotes,
+            ["link"] = Links,
+            ["chat"] = Conversations,
+            ["answer"] = Answers,
+            ["audio"] = Audios,
+            ["video"] = Videos,
+        };
+
+        /// <summary>
+        /// Returns the canonical type name, or null if the value is not one of the eight.
+        /// Returning null rather than passing the value through is the whole point: an
+        /// unrecognized string must never reach a filename.
+        /// </summary>
+        public static string? Normalize(string? candidate)
+        {
+            if (string.IsNullOrWhiteSpace(candidate)) return null;
+            string trimmed = candidate.Trim();
+            return Known.TryGetValue(trimmed, out string? canonical) ? canonical : null;
+        }
+
+        /// <summary>
+        /// Type for a post read out of a TumblThree export file, taken from the filename
+        /// ("texts.txt" -> "texts"). Anything else in the folder -- README.txt, a stray
+        /// triage file, or a previously written Unknown.txt -- normalizes to null and is
+        /// rejected by the caller.
+        /// </summary>
+        public static string? FromFileName(string? path)
+        {
+            if (string.IsNullOrWhiteSpace(path)) return null;
+            return Normalize(Path.GetFileNameWithoutExtension(path));
+        }
+
+        /// <summary>
+        /// Type for a post from the likes API, whose legacy-format `type` field is singular.
+        /// Null when the field is absent or unrecognized -- the access is dynamic, so a missing
+        /// field yields null at runtime rather than failing to compile.
+        /// </summary>
+        public static string? FromApiType(string? apiType)
+        {
+            if (string.IsNullOrWhiteSpace(apiType)) return null;
+            return ApiTypeMap.TryGetValue(apiType.Trim(), out string? mapped) ? mapped : null;
+        }
+
+        /// <summary>
+        /// Last-resort type inferred from which content columns a row actually carries. Used
+        /// only to backfill rows written before any type was recorded; a filename or an API
+        /// type is always preferred over this.
+        ///
+        /// The order matters and is derived from the already-typed rows, where the column
+        /// signatures are effectively disjoint: answers carry Question+Answer and no Body,
+        /// images carry photo columns and no Body, texts carry Body and no photo columns.
+        ///
+        /// HasImage is deliberately NOT consulted: it is set on 12,420 of 19,828 known text
+        /// posts, so it says nothing about the post's type.
+        ///
+        /// conversations cannot be separated from texts this way -- both carry only Body -- so
+        /// a chat post with no other signal is labelled texts. A later --ingest that meets the
+        /// post in a real conversations.txt corrects it.
+        /// </summary>
+        public static string? InferFromContent(string? question, string? answer, string? quote,
+            string? link, string? audioCaption, string? body, string? photoUrl, string? photoCaption)
+        {
+            if (HasValue(question) && HasValue(answer)) return Answers;
+            if (HasValue(quote)) return Quotes;
+            if (HasValue(link)) return Links;
+            if (HasValue(audioCaption)) return Audios;
+            if (HasValue(body)) return Texts;
+            if (HasValue(photoUrl) || HasValue(photoCaption)) return Images;
+            return null;
+        }
+
+        // "." is the codebase-wide "field not supplied" sentinel in export records, so it
+        // counts as absent here just as it does in UpdatePost's CASE guards.
+        private static bool HasValue(string? value)
+        {
+            if (string.IsNullOrWhiteSpace(value)) return false;
+            return value.Trim() != ".";
+        }
+    }
+}

URLNotesGrabberCORE/Program.cs

@@ -44,6 +44,8 @@ namespace URLNotesGrabberCORE
             bool apiExplicitlySet = false;
             string startFromBlogName = string.Empty;
             bool forceIgnoreCooldown = false;
+            DateTime? fromDate = null;
+            DateTime? toDate = null;
             List<string> filteredArgs = new List<string>();
             for (int i = 0; i < args.Length; i++)
             {
@@ -60,6 +62,34 @@ namespace URLNotesGrabberCORE
                     continue;
                 }
 
+                if (string.Equals(args[i], "--fromDate", StringComparison.OrdinalIgnoreCase))
+                {
+                    if (i + 1 < args.Length && DateTime.TryParse(args[i + 1], out DateTime parsedFromDate))
+                    {
+                        fromDate = parsedFromDate;
+                        i++;
+                    }
+                    else
+                    {
+                        Console.WriteLine("--Missing or unparseable date after --fromDate. Ignoring.--");
+                    }
+                    continue;
+                }
+
+                if (string.Equals(args[i], "--toDate", StringComparison.OrdinalIgnoreCase))
+                {
+                    if (i + 1 < args.Length && DateTime.TryParse(args[i + 1], out DateTime parsedToDate))
+                    {
+                        toDate = parsedToDate;
+                        i++;
+                    }
+                    else
+                    {
+                        Console.WriteLine("--Missing or unparseable date after --toDate. Ignoring.--");
+                    }
+                    continue;
+                }
+
                 if (string.Equals(args[i], "--api3", StringComparison.OrdinalIgnoreCase))
                 {
                     apiSectionName = "TumblrApi3";
@@ -148,6 +178,12 @@ namespace URLNotesGrabberCORE
             if (args.Length == 0) //Traverse folder structure to add posts and thus blogs to DB
             {
                 int postsAdded = 0;
+                // This is the mode that actually gets run day to day, so the schema migration and
+                // the PostType backfill have to happen here too. They used to hang off --ingest,
+                // --output and friends only, which meant the untyped rows this traversal creates
+                // could sit unrepaired indefinitely while the one command everyone runs skipped
+                // the fix entirely. Idempotent, so paying it on every run costs nothing.
+                DataAccess.EnsureTTFileHelperColumnsExist();
                 try
                 {
                     DataAccess.EnableImportModePragmas();
@@ -220,7 +256,7 @@ namespace URLNotesGrabberCORE
 
                         if (args.Length < 2)
                         {
-                            Console.WriteLine("--Expected WITHOUTNOTESONLY (0, 1) [OPTIONAL: BEFOREDATE]--");
+                            Console.WriteLine("--Expected WITHOUTNOTESONLY (0, 1) [OPTIONAL: BEFOREDATE] [OPTIONAL: BLOGNAME]--");
                             exitCode = 2;
                             break;
                         }
@@ -242,28 +278,63 @@ namespace URLNotesGrabberCORE
                             Console.WriteLine("Without Notes Only: {0}\t{1}", withoutNotesOnly, args[1]);
                         }
 
-                        // Parse optional beforeDate parameter
+                        // Trailing arguments are the optional cutoff date and the optional blog filter, in
-                        if (args.Length >= 3 && !string.IsNullOrEmpty(args[2]))
+                        // either order. A token that parses as a date is the cutoff; anything else is a blog
+                        // name -- which is why an unparseable token is no longer an error here. "--blog=name"
+                        // forces the blog reading for the rare name that would otherwise parse as a date.
+                        string? collectBlogName = null;
+                        bool badCollectArg = false;
+
+                        for (int i = 2; i < args.Length; i++)
                         {
-                            if (DateTime.TryParse(args[2], out DateTime parsedDate))
+                            string arg = args[i];
+                            if (string.IsNullOrWhiteSpace(arg))
+                                continue;
+
+                            if (arg.StartsWith("--blog=", StringComparison.OrdinalIgnoreCase))
+                            {
+                                collectBlogName = arg.Substring("--blog=".Length);
+                                if (string.IsNullOrWhiteSpace(collectBlogName))
+                                {
+                                    Console.WriteLine("ERROR: --blog= requires a blog name");
+                                    badCollectArg = true;
+                                    break;
+                                }
+                            }
+                            else if (!explicitDateSupplied && DateTime.TryParse(arg, out DateTime parsedDate))
                             {
                                 beforeDate = parsedDate;
                                 explicitDateSupplied = true;
                                 Console.WriteLine($"Filter: Collecting notes for posts with NotesGatheredDateTime < {beforeDate}");
                             }
+                            else if (collectBlogName == null)
+                            {
+                                collectBlogName = arg;
+                            }
                             else
                             {
-                                Console.WriteLine($"ERROR: Invalid date format '{args[2]}'");
+                                Console.WriteLine($"ERROR: Unexpected argument '{arg}'");
-                                exitCode = 2;
+                                badCollectArg = true;
                                 break;
                             }
                         }
 
+                        if (badCollectArg)
+                        {
+                            exitCode = 2;
+                            break;
+                        }
+
+                        if (collectBlogName != null)
+                            Console.WriteLine($"Filter: Collecting notes for posts by '{collectBlogName}' only");
+
                         // Mode 0 (full re-check) with no explicit date is a *managed* run: freeze the cutoff and
                         // persist it so an interrupted run resumes against the same cutoff and a completed run stops
-                        // instead of restarting. Mode 1 and explicit-date runs keep their existing behavior.
+                        // instead of restarting. Mode 1, explicit-date and blog-scoped runs keep their existing
+                        // behavior -- a single blog covers a slice of the worklist, so letting it write the shared
+                        // run state would mark the whole re-check complete after collecting one blog.
                         bool managedCollectRun = false;
-                        if (!withoutNotesOnly && !explicitDateSupplied)
+                        if (!withoutNotesOnly && !explicitDateSupplied && collectBlogName == null)
                         {
                             DataAccess.EnsureCollectRunStateTableExists();
                             var runState = DataAccess.GetCollectRunState();
@@ -281,7 +352,22 @@ namespace URLNotesGrabberCORE
                             managedCollectRun = true;
                         }
 
-                        exitCode = CollectNotes(settings.GetValue<string>("PathOutput"), withoutNotesOnly, beforeDate, managedCollectRun).GetAwaiter().GetResult();
+                        if (forceIgnoreCooldown)
+                            Console.WriteLine(withoutNotesOnly
+                                ? "--force: ignoring the periodic re-collect cooldown; already-collected posts in scope are re-queued now"
+                                : "--force: no effect in mode 0 - a full re-check already re-collects every post");
+
+                        if (fromDate.HasValue)
+                            Console.WriteLine(withoutNotesOnly
+                                ? $"--fromDate: only re-queuing already-collected posts originally posted on/after {fromDate.Value} (applies with or without --force)"
+                                : "--fromDate: no effect in mode 0 - it only bounds the periodic re-queue branch");
+
+                        if (toDate.HasValue)
+                            Console.WriteLine(withoutNotesOnly
+                                ? $"--toDate: only re-queuing already-collected posts originally posted on/before {toDate.Value} (applies with or without --force)"
+                                : "--toDate: no effect in mode 0 - it only bounds the periodic re-queue branch");
+
+                        exitCode = CollectNotes(settings.GetValue<string>("PathOutput"), withoutNotesOnly, beforeDate, managedCollectRun, collectBlogName, forceIgnoreCooldown, fromDate, toDate).GetAwaiter().GetResult();
                         break;
 
                     case "--blogsR": //collect notes from all posts
@@ -413,7 +499,7 @@ namespace URLNotesGrabberCORE
 
             Console.WriteLine("--blogs\t For each Blog in DB, write blogname to file");
 
-            Console.WriteLine("--collect [0|1] [datetime]\t Collect Notes from API. 1=only posts without notes. 0=full re-check of all posts: a single resumable pass (interrupt & relaunch to resume; stops when complete, retrigger for a new pass). Optional datetime overrides the cutoff and runs as a one-off (bypasses resume tracking).");
+            Console.WriteLine("--collect [0|1] [datetime] [blogname]\t Collect Notes from API. 1=only posts without notes. 0=full re-check of all posts: a single resumable pass (interrupt & relaunch to resume; stops when complete, retrigger for a new pass). Optional datetime overrides the cutoff and runs as a one-off (bypasses resume tracking). Optional blogname restricts the run to that blog (exact, case-sensitive match) and also runs as a one-off; e.g. \"--collect 1 zomb-eh\". datetime and blogname may be given in either order - use --blog=name if a blog name would otherwise parse as a date. Add --force to ignore the periodic re-collect cooldown and re-queue already-collected posts immediately (mode 1 only). Add --fromDate <datetime> / --toDate <datetime> to only re-queue already-collected posts originally posted on/after / on/before that date (mode 1 only; either or both may be given; applies with or without --force).");
 
             Console.WriteLine("--blogsR\t For each Note that is a REPLY, write blogname to file ");
 
@@ -425,7 +511,11 @@ namespace URLNotesGrabberCORE
 
             Console.WriteLine("--likes\t Fetch likes: initial backfill for new blogs, incremental refresh for blogs past cooldown. Optional blog name forces single-blog run.");
 
-            Console.WriteLine("--force\t (with --likes) Ignore cooldown and refresh every fully-backfilled blog");
+            Console.WriteLine("--force\t Ignore refresh cooldowns: with --likes, refresh every fully-backfilled blog; with --collect 1, re-queue already-collected posts without waiting out their cooldown");
+
+            Console.WriteLine("--fromDate <datetime>\t With --collect 1, only re-queue already-collected posts originally posted on/after <datetime>. Independent of --force - applies whether or not the cooldown is also bypassed.");
+
+            Console.WriteLine("--toDate <datetime>\t With --collect 1, only re-queue already-collected posts originally posted on/before <datetime>. Independent of --force; may be combined with --fromDate for a range.");
 
             Console.WriteLine("--urldump\t Scan all posts' text columns and extract suspected URLs to configured file");
 
@@ -1012,6 +1102,15 @@ namespace URLNotesGrabberCORE
                             string reblogKey = post.reblog_key?.ToString() ?? ".";
                             string link = ".";
 
+                            // No file backs a liked post, so the filename trick used everywhere
+                            // else cannot apply here. GrabLikes requests npf=false, and in the
+                            // legacy format `type` is the discriminator that decides which content
+                            // fields a post carries -- singular there, mapped to our plural names.
+                            // liked_posts is List<dynamic>, so this is resolved at runtime and a
+                            // missing field yields null rather than a compile error; an absent or
+                            // unrecognized value leaves the type NULL instead of guessing.
+                            string? apiPostType = PostTypes.FromApiType(post.type?.ToString() as string);
+
                             // Only insert if any of the data contains strings from ContainsList
                             bool shouldInsert = false;
                             string matchedFieldName = string.Empty;
@@ -1055,7 +1154,8 @@ if (shouldInsert)
                                 DataAccess.AddPost(authorBlog, postID, reblogURL, date, postURL, slug, reblogKey,
                                                         reblogName, summary, quote, body, tags, link, photoURL,
                                                         photoCaption, downloadedFiles, audioCaption, question, answer,
-                                                        title, hasImage, true, rootBlogName: rootBlogName, rootURL: rootURL);
+                                                        title, hasImage, true, rootBlogName: rootBlogName, rootURL: rootURL,
+                                                        postType: apiPostType);
                             }
                         }
 
@@ -1294,9 +1394,19 @@ if (shouldInsert)
         // blipping on one post. Past this, skipping post-by-post would just hammer a closed door.
         const int MaxConsecutiveTransient = 10;
 
-        static async Task<int> CollectNotes(string outPath, bool withoutNotesOnly = true, DateTime? beforeDate = null, bool managedRun = false)
+        static async Task<int> CollectNotes(string outPath, bool withoutNotesOnly = true, DateTime? beforeDate = null, bool managedRun = false, string? blogName = null, bool ignoreRefreshCooldown = false, DateTime? fromDate = null, DateTime? toDate = null)
         {
-            List<Tuple<string, long, long, long>> posts = DataAccess.GetPosts(withoutNotesOnly, beforeDate);
+            List<Tuple<string, long, long, long>> posts = DataAccess.GetPosts(withoutNotesOnly, beforeDate, blogName, ignoreRefreshCooldown, fromDate, toDate);
+
+            if (posts.Count == 0 && !string.IsNullOrWhiteSpace(blogName))
+            {
+                // BlogName is matched exactly, so a typo or a case mismatch looks identical to "nothing
+                // left to collect". Say so rather than reporting a silent, instant success.
+                Console.WriteLine($"No posts to collect for blog '{blogName}'. Either it is fully collected, or the name does not match a stored blog (the match is case-sensitive).");
+                if (withoutNotesOnly && !ignoreRefreshCooldown)
+                    Console.WriteLine("Already-collected posts are re-queued only once their cooldown elapses; add --force to re-collect them now.");
+                return 0;
+            }
 
             // Posts attempted (with a definitive, non-throttle result) during *this* process. Guarantees a single
             // attempt pass: once every remaining post has been attempted, the loop stops instead of spinning on a
@@ -1392,7 +1502,7 @@ if (shouldInsert)
                         }
 
                         // Re-fetch the updated list after processing the current post
-                        posts = DataAccess.GetPosts(withoutNotesOnly, beforeDate);
+                        posts = DataAccess.GetPosts(withoutNotesOnly, beforeDate, blogName, ignoreRefreshCooldown, fromDate, toDate);
                     }
                 }
 
@@ -1465,7 +1575,15 @@ if (shouldInsert)
                         string normalizedDirectoryName = NormalizeBlogFolderName(new DirectoryInfo(path).Name);
                         bool isAtOrAfterStart = string.IsNullOrWhiteSpace(startFromBlogName) || string.Compare(normalizedDirectoryName, startFromBlogName, StringComparison.OrdinalIgnoreCase) >= 0;
 
+                        // The filename is the post type ("texts.txt" -> "texts"), so only the eight
+                        // known export files are post sources. Everything else in a blog folder is
+                        // either not a post file at all (README.txt, url lists, triage scratch) or
+                        // is our own derived output -- Unknown.txt above all, which must never be
+                        // read back in as a source or it perpetuates itself.
+                        string? filePostType = PostTypes.FromFileName(file);
+
                         if (file.EndsWith(".txt", StringComparison.OrdinalIgnoreCase)
+                            && filePostType != null
                             && (string.IsNullOrEmpty(blogName) || path.IndexOf(blogName, StringComparison.OrdinalIgnoreCase) >= 0)
                             && isAtOrAfterStart)
                         {
@@ -1498,7 +1616,7 @@ if (shouldInsert)
                                                 DataAccess.AddPost(curDir, long.Parse(reblog.postID), reblog.reblogURL, reblog.date, reblog.postURL, reblog.slug, reblog.reblogKey,
                                                                     reblog.reblogName, reblog.summary, reblog.quote, reblog.body, reblog.tags, reblog.link, reblog.photoURL,
                                                                     reblog.photoCaption, reblog.downloadedFiles, reblog.audioCaption, reblog.question, reblog.answer,
-                                                                    reblog.title, false, rootURL: reblog.rootURL);
+                                                                    reblog.title, false, rootURL: reblog.rootURL, postType: filePostType);
                                                 recordImportStopwatch.Stop();
 
                                                 postsAdded++;
@@ -1646,7 +1764,7 @@ if (shouldInsert)
                                             DataAccess.AddPost(curDir, long.Parse(reblog.postID), reblog.reblogURL, reblog.date, reblog.postURL, reblog.slug, reblog.reblogKey,
                                                                     reblog.reblogName, reblog.summary, reblog.quote, reblog.body, reblog.tags, reblog.link, reblog.photoURL,
                                                                     reblog.photoCaption, reblog.downloadedFiles, reblog.audioCaption, reblog.question, reblog.answer,
-                                                                    reblog.title, true, rootURL: reblog.rootURL);
+                                                                    reblog.title, true, rootURL: reblog.rootURL, postType: filePostType);
                                             recordImportStopwatch.Stop();
 
                                             postsAdded++;

URLNotesGrabberCORE/TL.db.md

@@ -4,12 +4,25 @@ The SQLite database behind **URLNotesGrabberCORE** and its sibling crawlers, and
 [Rolodex](https://git.basso.land/jim/Rolodex) reads.
 
 Everything below was read out of the live file, not inferred from code. Counts are as of
-**2026-07-29**; re-run the queries at the bottom to refresh them.
+**2026-08-07**; re-run the queries at the bottom to refresh them.
 
 - Journal mode: **WAL** — `TL.db-wal` and `TL.db-shm` live beside the file and are part of
   the database. Copying `TL.db` alone gives you whatever was last checkpointed, not the
   current state.
-- Page size: 4096.
+- Page size: 4096. File size: 148 MB.
+
+> ### ⚠ Breaking change, 2026-08-07: `Notes` holds integer IDs, not names
+>
+> `Notes.RootBlogName`, `Notes.NoteBlogName` and `Notes.Type` **no longer exist**. They
+> are now `RootBlogId`, `NoteBlogId` and `TypeId`, resolved through the new `BlogNames`
+> and `NoteTypes` tables. Any query naming the old columns fails outright.
+>
+> There is no compatibility view. See [porting to the integer
+> schema](#porting-to-the-integer-schema) for the old-to-new translation of every query
+> shape the applications use.
+>
+> Applied by `../normalize-notes.sql`, which took the file from 207 MB to 148 MB. An
+> earlier change the same day (`../shrink-db.sql`) took it from 267 MB to 207 MB.
 
 ---
 
@@ -17,13 +30,20 @@ Everything below was read out of the live file, not inferred from code. Counts a
 
 | Table | Rows | What it is |
 |---|--:|---|
-| `Blogs` | 144,367 | The crawl registry — one row per known blog, plus crawl-state flags |
+| `Blogs` | 188,620 | The crawl registry — one row per known blog, plus crawl-state flags |
-| `Posts` | 14,589 | Stored post content. Only 3,602 blogs actually have any |
+| `Posts` | 22,468 | Stored post content. Only 3,867 blogs actually have any |
-| `Notes` | 1,189,604 | The engagement graph: `NoteBlogName` acted on `(RootBlogName, PostID)` |
+| `Notes` | 1,182,333 | The engagement graph: `NoteBlogId` acted on `(RootBlogId, PostID)` |
 
-The engagement graph is the interesting part. 31,888 distinct blogs appear as engagers —
+…supported by two lookup tables that exist only to keep `Notes` small:
-far more than the 3,602 that have stored posts — which is what makes this a social graph
+
-rather than a post archive.
+| Table | Rows | What it is |
+|---|--:|---|
+| `BlogNames` | 20,430 | `BlogId` ⇄ `BlogName`. The ID authority for everything in `Notes` |
+| `NoteTypes` | 5 | `TypeId` ⇄ `Type`. `like`, `reblog`, `reply`, `posted`, `post_attribution` |
+
+The engagement graph is the interesting part. 20,311 distinct blogs appear as engagers —
+far more than the 3,867 that have stored posts — which is what makes this a social graph
+rather than a post archive. Only 2,771 blogs appear as the *root* of a note.
 
 ### `Blogs`
 
@@ -42,23 +62,45 @@ CREATE TABLE "Blogs" (
     LikesLastRefreshed      INTEGER DEFAULT 0,
     LikesLastNewCount       INTEGER DEFAULT 0,
     TTFolderPath            TEXT,
+    BlogId                  INTEGER,
     PRIMARY KEY("BlogName")
 );
+
+CREATE INDEX ix_Blogs_BlogId ON Blogs (BlogId);
 ```
 
-`BlogName` is the primary key, so it is the only indexed way in. There is no index on any
+`BlogName` is the primary key, so it is the only indexed way in by name. There is no index
-flag or date — filtering or sorting on those scans all 144k rows, which is affordable
+on any flag or date — filtering or sorting on those scans all 188k rows, which is
-here and is not on `Notes`.
+affordable here and is not on `Notes`.
 
-Flag distribution: `IsActive = 1` on 144,366 of 144,367 rows, `HasBeenOutput = 1` on
+**`BlogId` is new as of 2026-08-07 and is the join key to `Notes`.** It exists so that
-5,369, `ByLikes = 1` on 2. `IsActive` carries a second meaning as of Rolodex — see
+`Notes` can reach `Blogs` in a single integer hop rather than going through `BlogNames`
+and ending in a text comparison:
+
+```sql
+-- what you want
+FROM Blogs B JOIN Notes N ON N.NoteBlogId = B.BlogId
+
+-- not this
+FROM Blogs B JOIN BlogNames BN ON BN.BlogName = B.BlogName
+             JOIN Notes N      ON N.NoteBlogId = BN.BlogId
+```
+
+**`BlogId` is NULL on 168,202 of 188,620 rows** — every blog that has never appeared in a
+note. That is the large majority, and it is not an error: the registry is far bigger than
+the engagement graph. An inner join on `BlogId` therefore silently drops those blogs,
+which is usually what you want for engagement queries and is wrong for registry listings.
+
+Flag distribution: `IsActive = 1` on 188,601 of 188,620 rows, `HasBeenOutput = 1` on
+5,059, `ByLikes = 1` on 2. `IsActive` carries a second meaning as of Rolodex — see
 [`Blogs.IsActive`](#blogsisactive--now-written-by-two-applications) below.
 
 The columns after `DateCreated` were added later by `ALTER TABLE`, which is why they carry
-no quoting in the stored DDL. That is the normal way this schema grows.
+no quoting in the stored DDL. That is the normal way this schema grows, and `BlogId` is
+the newest example.
 
-**`DateAdded` is not written consistently.** 126,423 rows hold ISO `yyyy-MM-dd HH:mm:ss`;
+**`DateAdded` is not written consistently.** 170,677 rows hold ISO `yyyy-MM-dd HH:mm:ss`;
-17,944 hold US-format `M/d/yy` from a bulk import. As text those two sort into different
+17,943 hold US-format `M/d/yy` from a bulk import. As text those two sort into different
 parts of the table, so anything ordering or range-filtering on this column has to
 normalise first — see `DateSql` in Rolodex.
 
@@ -100,74 +142,352 @@ CREATE TABLE "Posts" (
 );
 ```
 
-**The key is `(BlogName, PostID)`, not `PostID`.** This matters more than it looks: 325
+**The key is `(BlogName, PostID)`, not `PostID`.** This matters more than it looks: 345
 post IDs exist under more than one blog, so an ID on its own is both ambiguous *and*
 unindexed. Any lookup should carry the blog name, and a batch lookup should group by blog
 so it stays on the leading column of the key.
 
 Notable:
 
-- **`PostType` is `NULL` on all 14,589 rows.** The column exists but nothing has ever
+- **`PostType` is now mostly populated: 20,679 of 22,468 rows, leaving 1,789 `NULL`.**
-  populated it. Treat it as unpopulated rather than as a type discriminator.
+  This reverses what earlier revisions of this document said — the column really was empty
-- `HasImage = 1` on 14,268 rows — nearly all of them. It records that the post *had* a
+  on every row, and something has since started writing it. Anything that treated it as
-  picture, not that a usable URL was kept, so it is not a reliable predictor that anything
+  permanently unset, or derived the type from post content instead, should be re-examined
-  will render.
+  against the live data. Rolodex still derives it.
+- `HasImage = 1` on 14,026 rows. It records that the post *had* a picture, not that a
+  usable URL was kept, so it is not a reliable predictor that anything will render.
 - `PhotoURL` is largely unused; in practice the image markup lives inside `Body`.
-- `NotFound = 1` on 4,663 rows — posts that have since been deleted upstream.
+- `NotFound = 1` on 4,712 rows — posts that have since been deleted upstream.
 - The content columns (`Body`, `Quote`, `Question`, `Answer`, …) are the heavy ones. List
   views should not select them.
 
 ### `Notes`
 
 ```sql
-CREATE TABLE "Notes" (
+CREATE TABLE Notes (
-    "RootBlogName"    TEXT,
+    RootBlogId      INTEGER NOT NULL,
-    "PostID"          INTEGER,
+    PostID          INTEGER NOT NULL,
-    "NoteBlogName"    TEXT,
+    NoteBlogId      INTEGER NOT NULL,
-    "TimeStamp"       INTEGER,
+    TimeStamp       INTEGER NOT NULL,
-    "Type"            TEXT,
+    TypeId          INTEGER NOT NULL,
-    "replyText"       TEXT DEFAULT '.',
+    replyText       TEXT,
-    "DatetimeCrawled" TEXT DEFAULT '2/12/26 12am',
+    DatetimeCrawled TEXT,
-    "DateModified"    TEXT,
+    DateModified    TEXT,
-    "DateCreated"     TEXT,
+    DateCreated     TEXT,
-    PRIMARY KEY("RootBlogName","PostID","TimeStamp","Type","NoteBlogName")
+    IsActive        INTEGER NOT NULL DEFAULT 1,
-);
+    PRIMARY KEY (RootBlogId, PostID, TimeStamp, TypeId, NoteBlogId)
+) WITHOUT ROWID;
 
-CREATE INDEX "Notes_idx_06e01ae3" ON "Notes" ("TimeStamp" DESC);
+CREATE INDEX ix_Notes_NoteBlogId ON Notes (NoteBlogId);
-CREATE INDEX "ix_NoteBlogName01"  ON "Notes" ("NoteBlogName");
 ```
 
+**Integer IDs since 2026-08-07 — this is the breaking change.** `RootBlogName`,
+`NoteBlogName` and `Type` are gone, replaced by `RootBlogId`, `NoteBlogId` and `TypeId`.
+Resolve them through [`BlogNames`](#blognames) and [`NoteTypes`](#notetypes), or join
+straight to `Blogs` on `BlogId`. The old names were text repeated on 1.18 million rows,
+in the table *and* in every index over it; the swap took the file from 207 MB to 148 MB.
+
+The **primary key column order is deliberately unchanged**, so the leading-prefix access
+patterns callers already depend on still hold: `(RootBlogId)` and `(RootBlogId, PostID)`
+remain cheap prefixes, exactly as `(RootBlogName)` and `(RootBlogName, PostID)` were.
+
+Two nulls-and-defaults differences from the old DDL, both intentional:
+
+- `replyText` and `DatetimeCrawled` **no longer carry column defaults**. The old table
+  defaulted them to `'.'` and `'2/12/26 12am'`, which is how 1.1M rows acquired
+  placeholder values nobody wrote. New rows now get `NULL` unless a writer supplies
+  something. The crawler names both columns explicitly, so its behaviour is unchanged.
+- The five key columns are now `NOT NULL`. They always were in practice.
+
+**`WITHOUT ROWID`, since earlier the same day.** The rows live in the primary key's
+b-tree rather than in a rowid table with a separate key index beside it. Two consequences
+matter before adding an index here:
+
+- There is no `rowid` on this table. `SELECT rowid FROM Notes` is an error, and no
+  code in any of the three apps relied on it.
+- A secondary index carries the whole five-column primary key as its row reference
+  instead of a compact rowid, so indexes here are **expensive** — though far less so
+  than before, now that the key is five integers rather than three integers and two
+  strings. `ix_Notes_NoteBlogId` costs 27 MB; its text predecessor cost 58 MB.
+
+**`Notes_idx_06e01ae3` on `TimeStamp DESC` was dropped at the same time.** It cost
+14 MB as a rowid index and would have cost 58 MB after the conversion. It was worth
+neither: the crawler's only `TimeStamp` filter (`>= 1535778000`) excludes 786 rows
+of 1.18M, Rolodex's default Notes sort carries a three-column tiebreaker that forces
+a full sort regardless, and the reply-matching `UPDATE` uses `ABS(TimeStamp - ?) <= 5`,
+which no index on `TimeStamp` can serve. The one path that got slower is Rolodex's
+Notes page with a date-range filter: 60 ms to 164 ms.
+
+See `../shrink-db.sql` for the full rationale and the applied result.
+
+**`DatetimeCrawled` is `NULL` on 1,148,077 rows, and that is the honest value.** Those
+rows previously stored the literal string `'2/12/26 12am'` — this column's own DDL
+default, written as a bulk backfill placeholder rather than as a crawl time. They were
+set to `NULL` on 2026-08-07, which is what consumers already displayed them as: the
+string parses as a date in neither format this schema writes.
+
+Note the trap: **the `DEFAULT '2/12/26 12am'` clause is still in the DDL above.** Any
+`INSERT` that omits this column writes the placeholder straight back. The crawler names
+it explicitly on every insert, so nothing reintroduces it today, but a new writer that
+forgets to would — which is why consumers should keep treating an unparseable value here
+as "unknown" rather than assuming `NULL` is now the only such marker.
+
 One row per engagement event. `TimeStamp` is **unix seconds** — unlike every date column
 elsewhere in the schema, which are text.
 
-| `Type` | Rows | Share |
+At 1.18M rows this is the table that dictates how the whole database has to be queried:
-|---|--:|--:|
-| `like` | 947,955 | 79.7% |
-| `reblog` | 224,323 | 18.9% |
-| `reply` | 15,201 | 1.3% |
-| `posted` | 2,106 | 0.2% |
-| `post_attribution` | 19 | — |
-
-At 1.19M rows this is the table that dictates how the whole database has to be queried:
 
 - **Nothing should run an unbounded `SELECT` or a bare `COUNT(*)` here.** A count scans
   the lot on every call.
-- The only fast access paths are the primary key's leading columns (`RootBlogName`, then
+- The only fast access paths are the primary key's leading columns (`RootBlogId`, then
-  `PostID`) and `ix_NoteBlogName01` on `NoteBlogName`. "Notes received by a blog" and
+  `PostID`) and `ix_Notes_NoteBlogId` on `NoteBlogId`. "Notes received by a blog" and
   "notes given by a blog" are both cheap; almost nothing else is.
-- Ordering by anything but `TimeStamp` is a full sort of whatever the filters leave.
+- **Every** ordering here is a full sort of whatever the filters leave, `TimeStamp`
-- `replyText` is `'.'` on 1,174,706 rows — only `reply` notes carry real text.
+  included. Filter first, then sort.
+- `replyText` is `'.'` on 1,167,464 rows — only `reply` notes carry real text. Those
+  dots are inherited from the old column default; new rows get `NULL` instead.
+
+**Resolve IDs by filtering the lookup, not by scanning `Notes`.** The lookup tables are
+tiny and uniquely indexed, so pushing a name predicate into them costs nothing and lets
+the `Notes` index do the work:
+
+```sql
+-- good: BlogNames resolves the name, then the index is searched
+SELECT * FROM Notes
+ WHERE NoteBlogId = (SELECT BlogId FROM BlogNames WHERE BlogName = ?);
+
+-- also good, same plan
+SELECT n.* FROM Notes n
+  JOIN BlogNames b ON b.BlogId = n.NoteBlogId
+ WHERE b.BlogName = ?;
+```
+
+### `BlogNames`
+
+```sql
+CREATE TABLE BlogNames (
+    BlogId   INTEGER PRIMARY KEY,
+    BlogName TEXT NOT NULL UNIQUE
+);
+```
+
+20,430 rows — every name appearing in `Notes` as either participant, and nothing else.
+This is the **ID authority**: `Notes.RootBlogId` and `Notes.NoteBlogId` both point here,
+and `Blogs.BlogId` is a copy of the value for the blogs that have one.
+
+**12 of these names have no `Blogs` row.** The registry has never been a superset of the
+engagement graph and still is not, so resolving an ID through `Blogs` rather than
+`BlogNames` will occasionally find nothing. Use `BlogNames` when you need the name itself
+and `Blogs` when you need registry columns.
+
+IDs are assigned by SQLite and are **stable**: they are stored in over a million `Notes`
+rows. Never renumber them. A blog that is renamed upstream should get a new row, not an
+edit to an existing one, unless every `Notes` reference is migrated with it.
+
+### `NoteTypes`
+
+```sql
+CREATE TABLE NoteTypes (
+    TypeId INTEGER PRIMARY KEY,
+    Type   TEXT NOT NULL UNIQUE
+);
+```
+
+| `TypeId` | `Type` | Rows | Share |
+|--:|---|--:|--:|
+| 1 | `like` | 945,167 | 79.9% |
+| 2 | `reblog` | 219,203 | 18.5% |
+| 3 | `reply` | 15,345 | 1.3% |
+| 4 | `posted` | 2,617 | 0.2% |
+| 5 | `post_attribution` | 1 | — |
+
+The set is fixed in practice, but it is a table rather than a `CHECK` constraint so that
+adding a type is an `INSERT` and not a schema migration. **The IDs above are stored in
+`Notes` and must not be reassigned.**
+
+Five rows means the lookup is effectively free; write `t.Type = 'reblog'` and let SQLite
+resolve it, or hardcode the ID if you prefer — both are fine, but hardcoding ties your
+code to this table's contents, so prefer the join in anything long-lived.
+
+---
+
+## Porting to the integer schema
+
+Everything here was checked against the live 148 MB file. There were 14 affected call
+sites in `DataAccess.cs` and 16 in `RolodexRepository.cs`. TumblThree needs no changes —
+its single statement touches `Blogs.IsActive` and `BlogName` only.
+
+**`DataAccess.cs` is ported.** All 14 sites now read the integer schema, `AddNote`
+registers names and types before inserting, and `verify-db-schema.sql` reports a
+pre-migration file rather than letting the app fail on it. `RolodexRepository.cs` lives in
+the [Rolodex](https://git.basso.land/jim/Rolodex) repository and is not covered by that
+work. One site was dropped rather than translated: the `LEFT JOIN Notes` in `GetPosts`
+selected nothing and was collapsed by the query's own `GROUP BY`, so it could not affect
+the result.
+
+### Column mapping
+
+| Was | Is now | Resolve via |
+|---|---|---|
+| `Notes.RootBlogName` | `Notes.RootBlogId` | `BlogNames.BlogId` → `.BlogName` |
+| `Notes.NoteBlogName` | `Notes.NoteBlogId` | `BlogNames.BlogId` → `.BlogName` |
+| `Notes.Type` | `Notes.TypeId` | `NoteTypes.TypeId` → `.Type` |
+| `ix_NoteBlogName01` | `ix_Notes_NoteBlogId` | — |
+
+`PostID`, `TimeStamp`, `replyText`, `DatetimeCrawled`, `DateModified`, `DateCreated` and
+`IsActive` are unchanged.
+
+### Filtering by a blog name
+
+```sql
+-- was
+WHERE NoteBlogName = @Name
+
+-- now, either form; both search ix_Notes_NoteBlogId after a unique-index lookup
+WHERE NoteBlogId = (SELECT BlogId FROM BlogNames WHERE BlogName = @Name)
+-- or
+JOIN BlogNames b ON b.BlogId = n.NoteBlogId WHERE b.BlogName = @Name
+```
+
+Measured 73 ms against 63 ms for the old text form on the busiest blog — the extra hop is
+a unique-index probe on a 20k-row table and does not show.
+
+### Joining `Notes` to `Blogs`
+
+This is the join to get right; it is the most common shape in both applications.
+
+```sql
+-- was
+FROM Blogs B INNER JOIN Notes N ON N.NoteBlogName = B.BlogName
+
+-- now: one integer hop, using the new Blogs.BlogId
+FROM Blogs B INNER JOIN Notes N ON N.NoteBlogId = B.BlogId
+```
+
+Do **not** route this through `BlogNames` — that adds a hop and ends in the text
+comparison the change was meant to remove.
+
+### Selecting a name back out
+
+```sql
+-- was
+SELECT NoteBlogName AS blogName, COUNT(*) FROM Notes ... GROUP BY NoteBlogName
+
+-- now
+SELECT bn.BlogName AS blogName, COUNT(*)
+  FROM Notes n JOIN BlogNames bn ON bn.BlogId = n.NoteBlogId
+ ... GROUP BY bn.BlogName
+```
+
+Group by `n.NoteBlogId` instead of `bn.BlogName` when you only need the name for display —
+grouping on the integer is cheaper and the name comes along for free.
+
+### Filtering by type
+
+```sql
+-- was
+WHERE type IN ('reblog', 'reply', 'posted')
+
+-- now
+WHERE TypeId IN (SELECT TypeId FROM NoteTypes WHERE Type IN ('reblog','reply','posted'))
+-- or, equivalently
+JOIN NoteTypes t ON t.TypeId = n.TypeId WHERE t.Type IN ('reblog','reply','posted')
+```
+
+`WHERE TypeId IN (2,3,4)` also works and is marginally faster, but hardcodes this table's
+contents into application code. Prefer the lookup outside of hot paths.
+
+Note the negated form needs care: `type NOT IN ('reblog','reply','posted')` becomes
+`TypeId NOT IN (SELECT TypeId FROM NoteTypes WHERE Type IN (...))`, which is correct only
+because `TypeId` is `NOT NULL`.
+
+### Inserting a note
+
+The crawler must ensure both names have IDs first. `INSERT OR IGNORE` on `BlogNames` is
+the whole of it — no read-back, no round trip, safe to run every time:
+
+```sql
+INSERT OR IGNORE INTO BlogNames (BlogName) VALUES (@rootBlogName);
+INSERT OR IGNORE INTO BlogNames (BlogName) VALUES (@noteBlogName);
+
+INSERT OR IGNORE INTO Notes
+       (RootBlogId, PostID, NoteBlogId, TimeStamp, TypeId,
+        DatetimeCrawled, DateModified, DateCreated)
+SELECT (SELECT BlogId FROM BlogNames WHERE BlogName = @rootBlogName),
+       @PostID,
+       (SELECT BlogId FROM BlogNames WHERE BlogName = @noteBlogName),
+       @TimeStamp,
+       (SELECT TypeId FROM NoteTypes WHERE Type = @Type),
+       @DatetimeCrawled, @DateModified, @DateCreated;
+```
+
+Verified: a genuinely new note inserts, and re-running the identical statement inserts 0.
+Run all three statements in one transaction so a crash cannot leave a name registered
+with no note.
+
+**The duplicate-key error message has changed.** `DataAccess.cs` compares against the
+literal string
+
+```
+UNIQUE constraint failed: Notes.RootBlogName, Notes.PostID, Notes.TimeStamp, Notes.Type, Notes.NoteBlogName
+```
+
+at two call sites to decide whether to swallow an exception. SQLite now emits the *new*
+column names, so those comparisons no longer match and real errors will surface where
+they used to be silently ignored — or vice versa.
+
+Both sites now go through `IsNotesDuplicateKey` in `DataAccess.cs`, which matches on
+`UNIQUE constraint failed` plus `Notes.` rather than on the column list. A literal
+comparison is what broke here; the next rename should not break it again.
+
+### Updating notes
+
+Predicates translate the same way. The reply-matching update, which cannot use an index
+on `TimeStamp` either before or after:
+
+```sql
+-- now
+UPDATE Notes SET replyText = @replyText, DateModified = @dateModified
+ WHERE NoteBlogId = (SELECT BlogId FROM BlogNames WHERE BlogName = @noteBlogName)
+   AND ABS(TimeStamp - @TimeStamp) <= 5
+   AND TypeId = (SELECT TypeId FROM NoteTypes WHERE Type = 'reply')
+   AND (replyText IS NULL OR replyText = '' OR replyText = '.')
+   AND (replyText IS NULL OR replyText <> @replyText);
+```
+
+Rolodex's soft-delete updates need no change beyond the `WHERE` clause — they set
+`IsActive`, which is untouched.
+
+### Three traps
+
+**`Blogs.BlogId` is NULL on 168,202 of 188,620 rows.** Any inner join on it silently drops
+every blog that has never appeared in a note. Correct for engagement queries; wrong for
+registry listings, which need a `LEFT JOIN` or no join at all.
+
+**12 names in `BlogNames` have no `Blogs` row.** Resolving an ID to a name through `Blogs`
+will occasionally find nothing. Use `BlogNames` for names and `Blogs` for registry columns.
+
+**IDs are stable and must stay so.** `BlogNames.BlogId` and `NoteTypes.TypeId` are stored
+in over a million `Notes` rows. Never renumber. A blog renamed upstream gets a new row,
+not an edited one, unless every `Notes` reference migrates with it.
+
+---
 
 ### Referential integrity
 
 There are no foreign keys, and the tables do not perfectly agree:
 
 - 4 `Posts` rows name a blog with no `Blogs` row.
-- 15 of the 31,888 distinct engagers have no `Blogs` row.
+- 12 of the 20,430 names in `BlogNames` have no `Blogs` row.
 
 So a name appearing in `Notes` or `Posts` is not a guarantee that the registry knows about
 it. Joins from those tables back to `Blogs` should tolerate a miss.
 
+The integer schema does not fix this and was not meant to. `BlogNames` is deliberately
+built from `Notes` rather than from `Blogs`, precisely so that the 12 unregistered
+engagers keep their IDs and their rows. Had it been built from the registry, those notes
+would have been dropped by the migration's inner joins.
+
 ---
 
 ## The `'.'` placeholder convention
@@ -178,10 +498,14 @@ consumer.
 
 | Column | `'.'` rows |
 |---|--:|
-| `Notes.replyText` | 1,174,706 |
+| `Notes.replyText` | 1,167,464 |
-| `Posts.Title` | 13,144 |
+| `Posts.Title` | 12,562 |
 | `Posts.Body` | 172 |
 
+`Notes.replyText` and `Notes.DatetimeCrawled` **no longer carry column defaults** as of
+the integer migration, so new note rows get `NULL` rather than a placeholder. The dots
+already in `replyText` were not rewritten — cleaning is still required on read.
+
 Any query whose output reaches a human should collapse it:
 
 ```sql
@@ -214,9 +538,12 @@ Crawler bookkeeping. Rolodex ignores all of these.
 `DataAccess.cs` joins on it to decide what to collect:
 
 ```sql
-SELECT NoteBlogName, count(*) FROM notes
+-- shape only; the ported GetBlogs joins Blogs directly on BlogId and needs no BlogNames hop
-INNER JOIN blogs ON blogs.BlogName = notes.NoteBlogName
+SELECT bn.BlogName, count(*)
-WHERE blogs.IsActive = @isActive AND ...
+  FROM Notes n
+  JOIN Blogs     b  ON b.BlogId  = n.NoteBlogId
+  JOIN BlogNames bn ON bn.BlogId = n.NoteBlogId
+ WHERE b.IsActive = @isActive AND ...
 ```
 
 Nothing inside the crawler *writes* it — it is an input, set from outside.
@@ -262,12 +589,16 @@ under its Posts and Notes pages. Removing a blog hides the blog, not what it col
 
 ---
 
-## `Posts.IsActive` and `Notes.IsActive` — optional, and not in this database yet
+## `Posts.IsActive` and `Notes.IsActive` — present, and written from outside
 
-The same flag is being extended to the two content tables, with the same meaning: `0` is
+The same flag extends to the two content tables, with the same meaning: `0` is removed,
-removed, anything else — including `NULL` — is live. **Neither column exists in the live
+anything else — including `NULL` — is live. **Both columns now exist in the live `TL.db`**
-`TL.db` as of 2026-07-29**; the DDL quoted above for `Posts` and `Notes` is complete. Like
+and are included in the DDL quoted above. As of 2026-08-07, `Posts.IsActive = 0` on 5,900
-`Blogs.IsActive`, they are written from outside this crawler.
+rows and `Notes.IsActive = 0` on none. Like `Blogs.IsActive`, they are written from
+outside this crawler.
+
+On `Notes` the column is `INTEGER NOT NULL DEFAULT 1`, so a `NULL` cannot occur there;
+`Posts` and `Blogs` are laxer, which is why the predicate below still uses `COALESCE`.
 
 The crawler therefore treats both as optional, and as nothing it owns:
 
@@ -308,10 +639,18 @@ handled:
 ```sql
 SELECT 'Blogs', COUNT(*) FROM Blogs
 UNION ALL SELECT 'Posts',     COUNT(*) FROM Posts
-UNION ALL SELECT 'Notes', COUNT(*) FROM Notes;
+UNION ALL SELECT 'Notes',     COUNT(*) FROM Notes
+UNION ALL SELECT 'BlogNames', COUNT(*) FROM BlogNames;
 
--- note type mix
+-- note type mix (joins NoteTypes; Notes.Type no longer exists)
-SELECT Type, COUNT(*) FROM Notes GROUP BY Type ORDER BY 2 DESC;
+SELECT t.Type, COUNT(*)
+  FROM Notes n JOIN NoteTypes t ON t.TypeId = n.TypeId
+ GROUP BY t.Type ORDER BY 2 DESC;
+
+-- how much of the registry participates in the engagement graph
+SELECT COUNT(*) FILTER (WHERE BlogId IS NOT NULL) AS with_notes,
+       COUNT(*) FILTER (WHERE BlogId IS NULL)     AS without_notes
+  FROM Blogs;
 
 -- the two date shapes in Blogs.DateAdded
 SELECT CASE WHEN DateAdded LIKE '____-__-__%' THEN 'ISO' ELSE 'US' END, COUNT(*)
@@ -324,6 +663,13 @@ SELECT COUNT(*) FROM (
 -- rows that reference a blog the registry does not have
 SELECT COUNT(*) FROM Posts p
 WHERE NOT EXISTS (SELECT 1 FROM Blogs b WHERE b.BlogName = p.BlogName);
+
+SELECT COUNT(*) FROM BlogNames bn
+WHERE NOT EXISTS (SELECT 1 FROM Blogs b WHERE b.BlogName = bn.BlogName);
+
+-- space by object, to see where the file actually goes
+SELECT name, SUM(pgsize)/1024/1024 AS mb
+  FROM dbstat GROUP BY name ORDER BY SUM(pgsize) DESC;
 ```
 
 Open the file read-only so an inspection can never disturb a running crawl:

normalize-notes.sql

@@ -0,0 +1,143 @@
+-- normalize-notes.sql
+-- Replaces the repeated blog-name and type TEXT in Notes with integer IDs.
+-- Reduces TL.db from ~207 MB to ~148 MB (-29%).
+--
+-- THIS IS A BREAKING SCHEMA CHANGE. There is no compatibility layer. Every
+-- query in URLNotesGrabberCORE and Rolodex that names Notes.RootBlogName,
+-- Notes.NoteBlogName or Notes.Type stops working the moment this runs, and
+-- stays broken until those queries are rewritten. This was a deliberate choice
+-- over a view-plus-triggers shim, which was measured to work but cost 194 ms ->
+-- 321 ms on Rolodex's unfiltered Notes page.
+--
+-- TumblThree is unaffected. It touches only Blogs, and the column added to
+-- Blogs here is additive.
+--
+-- HOW TO RUN (DB Browser for SQLite):
+--   1. Stop all three apps. Pause NextCloud sync.
+--   2. Back up TL.db.
+--   3. Execute SQL, paste this file, run. Then Write Changes.
+--   4. Tools > Compact Database (VACUUM). Nothing shrinks until this finishes.
+
+--------------------------------------------------------------------------
+-- The shape this produces
+--------------------------------------------------------------------------
+--   BlogNames(BlogId, BlogName)   the ID authority: every name appearing in
+--                                 Notes as either participant. 20,430 rows.
+--                                 12 of these have no Blogs row -- the
+--                                 registry has never been a superset of the
+--                                 engagement graph, and still is not.
+--
+--   NoteTypes(TypeId, Type)       5 rows. Fixed set, but written as a table
+--                                 rather than a CHECK so a new type is an
+--                                 INSERT and not a schema migration.
+--
+--   Notes(...Id columns...)       integer FKs in place of text. WITHOUT ROWID,
+--                                 same 5-column key in the same column order.
+--
+--   Blogs.BlogId                  NEW additive column. Lets Notes join Blogs in
+--                                 one integer hop instead of going through
+--                                 BlogNames and comparing text at the end.
+--                                 NULL on the 168,202 blogs with no notes.
+
+PRAGMA foreign_keys = off;
+
+BEGIN;
+
+--------------------------------------------------------------------------
+-- STEP 1: the ID authority
+--------------------------------------------------------------------------
+CREATE TABLE BlogNames (
+    BlogId   INTEGER PRIMARY KEY,
+    BlogName TEXT NOT NULL UNIQUE
+);
+
+INSERT INTO BlogNames (BlogName)
+    SELECT RootBlogName FROM Notes
+    UNION
+    SELECT NoteBlogName FROM Notes;
+
+--------------------------------------------------------------------------
+-- STEP 2: the type lookup
+--------------------------------------------------------------------------
+CREATE TABLE NoteTypes (
+    TypeId INTEGER PRIMARY KEY,
+    Type   TEXT NOT NULL UNIQUE
+);
+
+-- IDs are assigned explicitly and must stay stable: they are stored in Notes.
+INSERT INTO NoteTypes (TypeId, Type) VALUES
+    (1, 'like'),
+    (2, 'reblog'),
+    (3, 'reply'),
+    (4, 'posted'),
+    (5, 'post_attribution');
+
+--------------------------------------------------------------------------
+-- STEP 3: rebuild Notes with integer keys
+--------------------------------------------------------------------------
+-- Column order of the primary key is unchanged from the text version, so the
+-- leading-column access patterns callers already rely on still hold:
+-- (RootBlogId) and (RootBlogId, PostID) remain cheap prefixes.
+CREATE TABLE NotesN (
+    RootBlogId      INTEGER NOT NULL,
+    PostID          INTEGER NOT NULL,
+    NoteBlogId      INTEGER NOT NULL,
+    TimeStamp       INTEGER NOT NULL,
+    TypeId          INTEGER NOT NULL,
+    replyText       TEXT,
+    DatetimeCrawled TEXT,
+    DateModified    TEXT,
+    DateCreated     TEXT,
+    IsActive        INTEGER NOT NULL DEFAULT 1,
+    PRIMARY KEY (RootBlogId, PostID, TimeStamp, TypeId, NoteBlogId)
+) WITHOUT ROWID;
+
+-- Inner joins are safe here: BlogNames was just built from these very columns,
+-- and NoteTypes covers all 5 values present. A row that failed to match would
+-- be silently dropped, which is what the row-count check at the bottom is for.
+INSERT INTO NotesN
+SELECT r.BlogId, n.PostID, b.BlogId, n.TimeStamp, t.TypeId,
+       n.replyText, n.DatetimeCrawled, n.DateModified, n.DateCreated, n.IsActive
+  FROM Notes n
+  JOIN BlogNames r ON r.BlogName = n.RootBlogName
+  JOIN BlogNames b ON b.BlogName = n.NoteBlogName
+  JOIN NoteTypes t ON t.Type     = n.Type;
+
+DROP TABLE Notes;
+ALTER TABLE NotesN RENAME TO Notes;
+
+-- Replaces ix_NoteBlogName01. Renamed because it indexes a different column now.
+CREATE INDEX ix_Notes_NoteBlogId ON Notes (NoteBlogId);
+
+--------------------------------------------------------------------------
+-- STEP 4: give Blogs the matching id
+--------------------------------------------------------------------------
+-- Additive: no existing column changes, so TumblThree's
+-- "UPDATE Blogs SET IsActive = 0 ... WHERE BlogName = ?" is untouched.
+ALTER TABLE Blogs ADD COLUMN BlogId INTEGER;
+
+UPDATE Blogs
+   SET BlogId = (SELECT bn.BlogId FROM BlogNames bn WHERE bn.BlogName = Blogs.BlogName);
+
+CREATE INDEX ix_Blogs_BlogId ON Blogs (BlogId);
+
+COMMIT;
+
+--------------------------------------------------------------------------
+-- STEP 5: Write Changes, then Tools > Compact Database
+--------------------------------------------------------------------------
+-- From the CLI instead:  sqlite3 TL.db "VACUUM;"
+
+--------------------------------------------------------------------------
+-- VERIFY
+--------------------------------------------------------------------------
+-- PRAGMA integrity_check;                         -- expect: ok
+-- SELECT COUNT(*) FROM Notes;                     -- expect: 1182333, unchanged
+-- SELECT COUNT(*) FROM BlogNames;                 -- expect: 20430
+-- SELECT COUNT(*) FROM Blogs WHERE BlogId IS NOT NULL;   -- expect: 20418
+--
+-- Losslessness was proven before this ran, by reconstructing the old text shape
+-- from the new schema and diffing it against the original both ways:
+--   SELECT COUNT(*) FROM (SELECT * FROM old.Notes EXCEPT SELECT * FROM Rebuilt);
+--   SELECT COUNT(*) FROM (SELECT * FROM Rebuilt EXCEPT SELECT * FROM old.Notes);
+-- Both returned 0 across all 1,182,333 rows and all 10 columns.

shrink-db.sql

@@ -0,0 +1,159 @@
+-- shrink-db.sql
+-- Reduces TL.db from ~267 MB to ~207 MB (-22%) with no application changes,
+-- and no visible change in any of the three apps that touch this file.
+--
+-- The three consumers, and what each one uses:
+--   URLNotesGrabberCORE  System.Data.SQLite 1.0.119  writes Notes, Posts, Blogs
+--   Rolodex (web)        Microsoft.Data.Sqlite 10.0  reads all three; soft-deletes via IsActive
+--   TumblThree           System.Data.SQLite.Core 1.0.119
+--                        one statement only, ManagerController.cs:972 --
+--                        "UPDATE Blogs SET IsActive = 0, DateModified = @DateModified
+--                         WHERE BlogName = @BlogName"
+--                        Nothing below touches the Blogs table, so TumblThree is
+--                        unaffected. (Its GlobalDatabaseService talks to TumblThree's
+--                        own separate FileEntries/BlogFiles database, not this file.)
+--
+-- WITHOUT ROWID needs SQLite >= 3.8.2 (Dec 2013). All three providers above are
+-- 2024-25 builds, an order of magnitude newer, so STEP 2 is readable by all of them.
+--
+-- Every figure below was measured on a copy of the live 267 MB file, and the
+-- result was checked against all three apps' access patterns:
+--   PRAGMA integrity_check ....... ok
+--   row counts ................... Notes 1182333, Posts 22468, Blogs 188620 (unchanged)
+--   Rolodex soft-delete UPDATE ... works
+--   Rolodex NoteBlogName filter .. still uses ix_NoteBlogName01
+--   crawler INSERT OR IGNORE ..... still dedupes (0 dupes admitted)
+--   TumblThree's UPDATE Blogs ..... untouched -- Blogs is not modified by this script
+--
+-- HOW TO RUN (DB Browser for SQLite):
+--   1. Stop ALL THREE apps: the crawler, the Rolodex web app, and TumblThree.
+--      Rolodex holds the file open and checkpoints the WAL, so it must be down,
+--      not just idle. TumblThree only opens the file for an instant when you
+--      delete a blog, but it can also launch the crawler on its own
+--      (UrlNotesGrabberService) -- so close it rather than merely avoiding it.
+--   2. Back up TL.db (copy the 267 MB file somewhere safe).
+--   3. Open TL.db, go to Execute SQL, paste STEP 1-3, run.
+--   4. Click "Write Changes".
+--   5. Run Tools > Compact Database. This is VACUUM; it will not run from the
+--      Execute SQL tab because DB Browser keeps a transaction open there.
+--      NOTHING SHRINKS ON DISK UNTIL THIS FINISHES.
+--
+-- Expected: steps 1-3 a couple of minutes, Compact a couple more.
+-- Free disk needed during Compact: ~270 MB for the temp copy.
+
+--------------------------------------------------------------------------
+-- STEP 1: drop the TimeStamp index          (required by STEP 2, not optional)
+--------------------------------------------------------------------------
+-- Measured cost/benefit:
+--
+--   * Rolodex's DEFAULT Notes view does not use it. Its sort carries the
+--     tiebreaker "RootBlogName, PostID, NoteBlogName", which forces a full sort
+--     regardless -- the query plan is byte-identical with and without the index.
+--     Sorting.cs:138 already assumes as much, and is right in practice.
+--   * The crawler's collect query (DataAccess.cs:1183) filters
+--     TimeStamp >= 1535778000, which excludes 786 of 1,182,333 rows (0.07%).
+--     A full index scan wearing a disguise. Same measured time without it.
+--   * The reply-matching UPDATE uses ABS(TimeStamp - ?) <= 5, which can never
+--     use an index on TimeStamp.
+--   * It DOES help exactly one path: Rolodex's Notes page with a date-range
+--     filter applied. 60 ms -> 164 ms. That is the whole of what is lost.
+--
+-- And it must go, because after STEP 2 it stops being cheap. A secondary index
+-- on a WITHOUT ROWID table carries the full 5-column primary key instead of a
+-- compact rowid, so this index grows 14 MB -> 58 MB. Keeping it lands the file
+-- at 265 MB instead of 207 MB -- i.e. it cancels the entire exercise to save
+-- 100 ms on one filtered view.
+
+DROP INDEX IF EXISTS Notes_idx_06e01ae3;
+
+--------------------------------------------------------------------------
+-- STEP 2: rebuild Notes as WITHOUT ROWID                       (-32 MB)
+--------------------------------------------------------------------------
+-- Notes has a 5-column composite primary key. In a rowid table SQLite stores
+-- that key twice: once in the table, once in sqlite_autoindex_Notes_1 (62 MB).
+-- WITHOUT ROWID stores the rows *in* the key's b-tree, so the copy disappears.
+--
+-- ix_NoteBlogName01 grows 25 -> 58 MB for the reason described above. Net -32 MB.
+-- It is kept because Rolodex filters on NoteBlogName and the crawler joins on it.
+--
+-- Safe: neither codebase references rowid on Notes (grep across both trees,
+-- zero matches). IsActive keeps its exact current declaration, which is what
+-- Rolodex's ActiveFlag predicate reads.
+
+PRAGMA foreign_keys = off;
+
+CREATE TABLE Notes_new (
+    "RootBlogName"      TEXT,
+    "PostID"            INTEGER,
+    "NoteBlogName"      TEXT,
+    "TimeStamp"         INTEGER,
+    "Type"              TEXT,
+    "replyText"         TEXT DEFAULT '.',
+    "DatetimeCrawled"   TEXT DEFAULT '2/12/26 12am',
+    "DateModified"      TEXT,
+    "DateCreated"       TEXT,
+    IsActive            INTEGER NOT NULL DEFAULT 1,
+    PRIMARY KEY("RootBlogName","PostID","TimeStamp","Type","NoteBlogName")
+) WITHOUT ROWID;
+
+INSERT INTO Notes_new
+SELECT RootBlogName, PostID, NoteBlogName, TimeStamp, Type,
+       replyText, DatetimeCrawled, DateModified, DateCreated, IsActive
+  FROM Notes;
+
+DROP TABLE Notes;
+ALTER TABLE Notes_new RENAME TO Notes;
+
+CREATE INDEX "ix_NoteBlogName01" ON "Notes" ("NoteBlogName");
+
+--------------------------------------------------------------------------
+-- STEP 3: clear the DatetimeCrawled placeholder                (-13 MB)
+--------------------------------------------------------------------------
+-- 1,148,077 of 1,182,333 rows hold the literal DDL default '2/12/26 12am' --
+-- a backfill placeholder, not a crawl time. SQLite stores all 12 bytes of it
+-- on every one of those rows.
+--
+-- This is UI-NEUTRAL in Rolodex, which is why it is safe despite Rolodex
+-- displaying the column. Rolodex reads and sorts it through DateSql.Sortable
+-- (DateRange.cs:67), whose CASE matches '____-__-__%' or the 8-character
+-- '__/__/__'. The 12-character '2/12/26 12am' matches neither, so Sortable
+-- already returns NULL for these rows and the page already renders an em dash
+-- and sorts them to the bottom. RolodexRepository.cs:957-963 documents exactly
+-- this. Writing a real NULL changes the bytes on disk, not the screen.
+--
+-- The crawler never reads the column back -- it only writes it on INSERT
+-- (DataAccess.cs:713, 721).
+
+UPDATE Notes SET DatetimeCrawled = NULL WHERE DatetimeCrawled = '2/12/26 12am';
+
+--------------------------------------------------------------------------
+-- DELIBERATELY NOT DONE: nulling Notes.DateCreated
+--------------------------------------------------------------------------
+-- An earlier draft of this script also cleared DateCreated = '2026-04-13'
+-- (a further -12 MB). Do not. Unlike DatetimeCrawled, that value DOES match
+-- Sortable's '____-__-__%' branch, so Rolodex renders it as a real date in the
+-- "Created" column on the Notes page and Post detail, and sorts by it. Nulling
+-- it would turn visible dates into em dashes and move rows in the sort order.
+
+--------------------------------------------------------------------------
+-- STEP 4: Write Changes, then Tools > Compact Database
+--------------------------------------------------------------------------
+-- Nothing above reclaims disk until VACUUM runs. From the sqlite3 CLI instead:
+--     sqlite3 TL.db "VACUUM;"
+
+--------------------------------------------------------------------------
+-- VERIFY (run after compacting; file should be ~207 MB)
+--------------------------------------------------------------------------
+-- PRAGMA integrity_check;
+--
+-- SELECT 'Notes' t, COUNT(*) n FROM Notes
+-- UNION ALL SELECT 'Posts', COUNT(*) FROM Posts
+-- UNION ALL SELECT 'Blogs', COUNT(*) FROM Blogs;
+--   -- expect 1182333 / 22468 / 188620, unchanged
+--
+-- SELECT name, SUM(pgsize)/1024/1024 AS mb
+--   FROM dbstat GROUP BY name ORDER BY SUM(pgsize) DESC;
+--   -- expect Notes 78, ix_NoteBlogName01 58, Posts 50, Blogs 14
+--
+-- APPLIED 2026-08-07. Actual result: 267.32 MB -> 207.17 MB, integrity_check ok,
+-- row counts unchanged, journal_mode still wal. VACUUM took 6 seconds.

verify-db-schema.sql

@@ -79,18 +79,35 @@ WITH expected(tbl, col, alter_stmt) AS (
     ('Blogs','LikesLastRefreshed',    'ALTER TABLE Blogs ADD COLUMN LikesLastRefreshed INTEGER DEFAULT 0;'),
     ('Blogs','LikesLastNewCount',     'ALTER TABLE Blogs ADD COLUMN LikesLastNewCount INTEGER DEFAULT 0;'),
     ('Blogs','TTFolderPath',          'ALTER TABLE Blogs ADD COLUMN TTFolderPath TEXT;'),
+    -- Blogs.BlogId (2026-08-07) is the single-hop join key into Notes. Deliberately NOT
+    -- auto-fixable: an added-but-empty BlogId makes every engagement join return zero
+    -- rows silently, which is worse than the hard error a missing column gives.
+    ('Blogs','BlogId',                'MANUAL REVIEW - see query 1d: run normalize-notes.sql'),
 
     -- Notes (base columns: manual review if missing)
-    ('Notes','RootBlogName',          'MANUAL REVIEW - base/PK column missing'),
+    -- Integer IDs since 2026-08-07. RootBlogName/NoteBlogName/Type are GONE, not renamed
+    -- in place -- a backup that still has them needs normalize-notes.sql, not an ALTER.
+    -- Query 1d below reports exactly that case.
+    ('Notes','RootBlogId',            'MANUAL REVIEW - see query 1d: pre-2026-08-07 name schema, or damaged'),
     ('Notes','PostID',                'MANUAL REVIEW - base/PK column missing'),
-    ('Notes','NoteBlogName',          'MANUAL REVIEW - base/PK column missing'),
+    ('Notes','NoteBlogId',            'MANUAL REVIEW - see query 1d: pre-2026-08-07 name schema, or damaged'),
     ('Notes','TimeStamp',             'MANUAL REVIEW - base/PK column missing'),
-    ('Notes','Type',                  'MANUAL REVIEW - base/PK column missing'),
+    ('Notes','TypeId',                'MANUAL REVIEW - see query 1d: pre-2026-08-07 name schema, or damaged'),
     ('Notes','DatetimeCrawled',       'MANUAL REVIEW - base column missing'),
     ('Notes','DateModified',          'MANUAL REVIEW - base column missing'),
     ('Notes','DateCreated',           'MANUAL REVIEW - base column missing'),
     -- Notes (additive migration column, auto-fixable)
-    ('Notes','replyText',             'ALTER TABLE Notes ADD COLUMN replyText TEXT DEFAULT ''.'';'),
+    -- No DEFAULT: the migrated schema dropped it, so new rows get NULL rather than a
+    -- placeholder. EnsureReplyTextColumnExists in DataAccess.cs adds it the same way.
+    ('Notes','replyText',             'ALTER TABLE Notes ADD COLUMN replyText TEXT;'),
+
+    -- BlogNames / NoteTypes (the lookup tables Notes resolves its IDs through, 2026-08-07).
+    -- Not auto-fixable: an empty BlogNames does not mean "add the table", it means the
+    -- Notes rows have nothing to resolve against. Rebuild with normalize-notes.sql.
+    ('BlogNames','BlogId',            'MANUAL REVIEW - see query 1d: run normalize-notes.sql'),
+    ('BlogNames','BlogName',          'MANUAL REVIEW - see query 1d: run normalize-notes.sql'),
+    ('NoteTypes','TypeId',            'MANUAL REVIEW - see query 1d: run normalize-notes.sql'),
+    ('NoteTypes','Type',              'MANUAL REVIEW - see query 1d: run normalize-notes.sql'),
 
     -- DailyAPICount (base columns)
     ('DailyAPICount','Date',          'MANUAL REVIEW - base/PK column missing'),
@@ -106,6 +123,8 @@ actual(tbl, col) AS (
   SELECT 'Posts',           name FROM pragma_table_info('Posts')
   UNION ALL SELECT 'Blogs',           name FROM pragma_table_info('Blogs')
   UNION ALL SELECT 'Notes',           name FROM pragma_table_info('Notes')
+  UNION ALL SELECT 'BlogNames',       name FROM pragma_table_info('BlogNames')
+  UNION ALL SELECT 'NoteTypes',       name FROM pragma_table_info('NoteTypes')
   UNION ALL SELECT 'DailyAPICount',   name FROM pragma_table_info('DailyAPICount')
   UNION ALL SELECT 'ApiKeyPoolState', name FROM pragma_table_info('ApiKeyPoolState')
   UNION ALL SELECT 'ApiKeyPoolMeta',  name FROM pragma_table_info('ApiKeyPoolMeta')
@@ -127,7 +146,7 @@ ORDER BY (e.alter_stmt LIKE 'ALTER%') DESC, e.tbl, e.col;
 -- 1b. MISSING TABLES: expected tables that don't exist at all in this DB.
 --     Zero rows = good.
 WITH expected_tables(tbl) AS (
-  VALUES ('Posts'),('Blogs'),('Notes'),('DailyAPICount'),
+  VALUES ('Posts'),('Blogs'),('Notes'),('BlogNames'),('NoteTypes'),('DailyAPICount'),
          ('ApiKeyPoolState'),('ApiKeyPoolMeta')
 )
 SELECT et.tbl AS missing_table
@@ -157,10 +176,12 @@ WITH expected(tbl, col) AS (
     ('Blogs','BlogName'),('Blogs','HasBeenOutput'),('Blogs','IsActive'),('Blogs','DateAdded'),
     ('Blogs','ByLikes'),('Blogs','DateModified'),('Blogs','DateCreated'),('Blogs','LikesPulled'),
     ('Blogs','LikesCursor'),('Blogs','LikesNewestTimestamp'),('Blogs','LikesLastRefreshed'),
-    ('Blogs','LikesLastNewCount'),('Blogs','TTFolderPath'),
+    ('Blogs','LikesLastNewCount'),('Blogs','TTFolderPath'),('Blogs','BlogId'),
-    ('Notes','RootBlogName'),('Notes','PostID'),('Notes','NoteBlogName'),('Notes','TimeStamp'),
+    ('Notes','RootBlogId'),('Notes','PostID'),('Notes','NoteBlogId'),('Notes','TimeStamp'),
-    ('Notes','Type'),('Notes','DatetimeCrawled'),('Notes','DateModified'),('Notes','DateCreated'),
+    ('Notes','TypeId'),('Notes','DatetimeCrawled'),('Notes','DateModified'),('Notes','DateCreated'),
     ('Notes','replyText'),('Notes','IsActive'),
+    ('BlogNames','BlogId'),('BlogNames','BlogName'),
+    ('NoteTypes','TypeId'),('NoteTypes','Type'),
     ('DailyAPICount','Date'),('DailyAPICount','APICount'),
     ('ApiKeyPoolState','KeyName'),('ApiKeyPoolState','RetryUntil'),
     ('ApiKeyPoolMeta','Id'),('ApiKeyPoolMeta','LastIndex')
@@ -169,6 +190,8 @@ actual(tbl, col) AS (
   SELECT 'Posts',           name FROM pragma_table_info('Posts')
   UNION ALL SELECT 'Blogs',           name FROM pragma_table_info('Blogs')
   UNION ALL SELECT 'Notes',           name FROM pragma_table_info('Notes')
+  UNION ALL SELECT 'BlogNames',       name FROM pragma_table_info('BlogNames')
+  UNION ALL SELECT 'NoteTypes',       name FROM pragma_table_info('NoteTypes')
   UNION ALL SELECT 'DailyAPICount',   name FROM pragma_table_info('DailyAPICount')
   UNION ALL SELECT 'ApiKeyPoolState', name FROM pragma_table_info('ApiKeyPoolState')
   UNION ALL SELECT 'ApiKeyPoolMeta',  name FROM pragma_table_info('ApiKeyPoolMeta')
@@ -181,6 +204,25 @@ WHERE e.col IS NULL
 ORDER BY a.tbl, a.col;
 
 
+-- 1d. PRE-MIGRATION DATABASE: a backup from before 2026-08-07, when Notes still
+--     stored names. Zero rows = good.
+--
+--     This is the one failure SECTION 2 cannot fix. Notes.RootBlogName /
+--     NoteBlogName / Type were replaced by RootBlogId / NoteBlogId / TypeId
+--     resolving through BlogNames and NoteTypes -- a data migration, not an
+--     ADD COLUMN. There is no compatibility view, so the current code fails
+--     outright ("no such column: RootBlogId") against such a file.
+--
+--     Fix: run normalize-notes.sql against a COPY of the backup, then re-run
+--     SECTION 1. Do not hand-add the ID columns: they would be empty, and an
+--     empty NoteBlogId is indistinguishable from a note by blog #0.
+SELECT 'Notes still stores names -- run normalize-notes.sql on a copy' AS pre_migration_schema,
+       group_concat(name, ', ')                                        AS legacy_columns_found
+FROM pragma_table_info('Notes')
+WHERE lower(name) IN ('rootblogname','noteblogname','type')
+HAVING COUNT(*) > 0;
+
+
 -- ============================================================================
 -- SECTION 2  --  FIX (opt-in, additive only)
 --
@@ -190,6 +232,9 @@ ORDER BY a.tbl, a.col;
 -- "duplicate column name" error and changes nothing -- just run the flagged
 -- subset. These are the 8 additive migration columns and nothing else; the
 -- likes high-water-mark reset is intentionally NOT included.
+--
+-- Nothing here addresses query 1d. The Notes integer schema is a data migration
+-- (normalize-notes.sql) and cannot be reached by adding columns.
 -- ============================================================================
 
 -- ALTER TABLE Posts ADD COLUMN PostType TEXT;
@@ -199,4 +244,4 @@ ORDER BY a.tbl, a.col;
 -- ALTER TABLE Blogs ADD COLUMN LikesLastRefreshed INTEGER DEFAULT 0;
 -- ALTER TABLE Blogs ADD COLUMN LikesLastNewCount INTEGER DEFAULT 0;
 -- ALTER TABLE Blogs ADD COLUMN TTFolderPath TEXT;
--- ALTER TABLE Notes ADD COLUMN replyText TEXT DEFAULT '.';
+-- ALTER TABLE Notes ADD COLUMN replyText TEXT;