jim/URLNotesGrabberCore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: revise TL.db notes for Rolodex's use of Blogs.IsActive

Browse Source 
Replaces the Blogs.IsDeleted section. Rolodex adds no column of its own;
it reuses the crawler's existing IsActive flag, so removing a blog in the
UI also stops it being collected.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
jim
2026-07-29 09:56:18 -05:00
co-authored by Claude Opus 5
parent a14debd5ed
commit eded5271ea
1 changed files with 45 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
URLNotesGrabberCORE/TL.db.md
+45 -43
View File
@@ -50,14 +50,12 @@ CREATE TABLE "Blogs" (
flag or date — filtering or sorting on those scans all 144k rows, which is affordable flag or date — filtering or sorting on those scans all 144k rows, which is affordable
here and is not on `Notes`. here and is not on `Notes`.
Flag distribution: `IsActive = 1` on 144,366 of 144,367 rows (effectively always true, so Flag distribution: `IsActive = 1` on 144,366 of 144,367 rows, `HasBeenOutput = 1` on
the Active filter is close to a no-op today), `HasBeenOutput = 1` on 5,369, `ByLikes = 1` 5,369, `ByLikes = 1` on 2. `IsActive` carries a second meaning as of Rolodex — see
on 2. [`Blogs.IsActive`](#blogsisactive--now-written-by-two-applications) below.
The columns after `DateCreated` were added later by `ALTER TABLE`, which is why they carry The columns after `DateCreated` were added later by `ALTER TABLE`, which is why they carry
no quoting in the stored DDL. That is the normal way this schema grows — and it is how no quoting in the stored DDL. That is the normal way this schema grows.
[`IsDeleted`](#blogsisdeleted--added-by-rolodex) gets there too, so expect that column in
the DDL of any database Rolodex has opened.
**`DateAdded` is not written consistently.** 126,423 rows hold ISO `yyyy-MM-dd HH:mm:ss`; **`DateAdded` is not written consistently.** 126,423 rows hold ISO `yyyy-MM-dd HH:mm:ss`;
17,944 hold US-format `M/d/yy` from a bulk import. As text those two sort into different 17,944 hold US-format `M/d/yy` from a bulk import. As text those two sort into different
@@ -210,53 +208,57 @@ Crawler bookkeeping. Rolodex ignores all of these.
--- ---
## `Blogs.IsDeleted` — added by Rolodex ## `Blogs.IsActive` — now written by two applications
`IsActive` has always been the crawler's work-selection flag. `GetBlogs` in
`DataAccess.cs` joins on it to decide what to collect:
```sql ```sql
ALTER TABLE Blogs ADD COLUMN IsDeleted INTEGER NOT NULL DEFAULT 0; SELECT NoteBlogName, count(*) FROM notes
INNER JOIN blogs ON blogs.BlogName = notes.NoteBlogName
WHERE blogs.IsActive = @isActive AND ...
``` ```
**This column is not written by the crawler.** Rolodex adds it automatically at startup if Nothing inside the crawler *writes* it — it is an input, set from outside.
it is missing, and uses it to hide a blog from its own UI without destroying anything:
- `0` — the blog is live. `NOT NULL DEFAULT 0` means every existing row, and every row the **Rolodex is now one of the things that sets it.** Removing a blog through the Rolodex UI
crawler creates afterwards, starts here. runs exactly this:
- `1` — somebody removed the blog through Rolodex.
Removal is a soft delete and nothing else changes: the row keeps every crawl flag it had, ```sql
and its `Posts` and `Notes` rows are untouched. Restoring is the same `UPDATE` setting the UPDATE Blogs SET IsActive = 0 WHERE BlogName = ?;
column back to `0`. ```
**Do not write `NULL` here.** Rolodex reads the column through `COALESCE(IsDeleted, 0)` Rolodex adds no column and changes no schema. It reuses this flag because the two meanings
precisely so a NULL cannot strand a row — without it a NULL satisfies neither the live were judged to be one decision: a blog you do not want in the browsing UI is a blog you do
test nor its negation, and the blog would disappear from the registry *and* from the not want to keep crawling. Removal therefore stops collection, and the Rolodex
removed list with no way back through the UI. The `NOT NULL` declaration is the real confirmation screen says so before anyone commits.
guard; the `COALESCE` is the belt to its braces.
- `1` (or absent/NULL) — live. Crawled, and visible in Rolodex.
- `0` — removed. Not crawled, hidden from the Rolodex registry, dashboard counts and
engagement rollups.
Restoring is the same `UPDATE` with a `1`. Nothing is destroyed either way: the blog's
`Posts` and `Notes` rows are never touched, and Rolodex deliberately keeps showing them
under its Posts and Notes pages. Removing a blog hides the blog, not what it collected.
### What other tools need to know ### What other tools need to know
1. **Adding the column is safe for `SELECT`, and for `INSERT` that names its columns.** It 1. **Setting `IsActive = 0` now also hides the blog from Rolodex**, and setting it back to
is *not* safe for `INSERT INTO Blogs VALUES (...)` without a column list — that form `1` makes it reappear. If another tool deactivates blogs in bulk, it is also removing
breaks the moment column count changes. If any crawler does this, give it an explicit them from the browsing UI — which may be exactly right, but it is no longer a
column list. crawler-only decision.
2. **The crawler should leave the column alone.** Writing to it would silently un-remove or 2. **Re-crawling a removed blog will not bring it back**, since nothing in the crawler
remove blogs behind Rolodex's back. writes the flag. An `INSERT OR REPLACE` on the `Blogs` row *would*, by resetting it to
3. **Re-crawling a removed blog will not bring it back.** An `INSERT OR REPLACE` on the the column default of `1`. Prefer an `UPDATE` of the specific columns, or
`Blogs` row *would*, by resetting the column to its default `0`. If that matters, `INSERT … ON CONFLICT DO UPDATE SET` naming only the columns being refreshed.
prefer an `UPDATE` of the specific columns, or `INSERT … ON CONFLICT DO UPDATE SET` 3. **NULL is treated as live.** The column is `INTEGER DEFAULT 1` with no `NOT NULL`, so
naming only the crawl columns. Rolodex reads it through `COALESCE(IsActive, 1)`. A NULL therefore leaves the blog
4. **A tool that lists blogs should decide whether it cares.** Rolodex filters visible rather than stranding it outside both the registry and the removed list, where
`WHERE COALESCE(IsDeleted, 0) = 0` everywhere a blog surfaces. A crawler probably no screen could reach it. Write `0` or `1`, not NULL.
should not — a blog hidden from a browsing UI is not necessarily one to stop crawling. 4. **Backing the feature out is a configuration change, not a migration.** Because there is
That is a deliberate choice, not an oversight. no Rolodex-owned column, setting `Rolodex__EnableBlogDeletion=false` is the whole of it;
there is nothing to drop. Any blogs already at `IsActive = 0` simply go back to being
Rolodex degrades if the column is absent — a read-only or `immutable=1` deployment cannot ordinary inactive blogs.
take the `ALTER` — by falling back to its previous behaviour of showing every blog. So
removing the column is a supported way to back the feature out:
```sql
ALTER TABLE Blogs DROP COLUMN IsDeleted;
```
--- ---