jim/URLNotesGrabberCore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: revise TL.db notes for Rolodex's use of Blogs.IsActive

Browse Source 
Replaces the Blogs.IsDeleted section. Rolodex adds no column of its own;
it reuses the crawler's existing IsActive flag, so removing a blog in the
UI also stops it being collected.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
jim
2026-07-29 09:56:18 -05:00
co-authored by Claude Opus 5
parent a14debd5ed
commit eded5271ea
1 changed files with 45 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
URLNotesGrabberCORE/TL.db.md
+45 -43
View File
@@ -50,14 +50,12 @@ CREATE TABLE "Blogs" (
flag or date — filtering or sorting on those scans all 144k rows, which is affordable
here and is not on `Notes`.
Flag distribution: `IsActive = 1` on 144,366 of 144,367 rows (effectively always true, so
the Active filter is close to a no-op today), `HasBeenOutput = 1` on 5,369, `ByLikes = 1`
on 2.
Flag distribution: `IsActive = 1` on 144,366 of 144,367 rows, `HasBeenOutput = 1` on
5,369, `ByLikes = 1` on 2. `IsActive` carries a second meaning as of Rolodex — see
[`Blogs.IsActive`](#blogsisactive--now-written-by-two-applications) below.
The columns after `DateCreated` were added later by `ALTER TABLE`, which is why they carry
no quoting in the stored DDL. That is the normal way this schema grows — and it is how
[`IsDeleted`](#blogsisdeleted--added-by-rolodex) gets there too, so expect that column in
the DDL of any database Rolodex has opened.
no quoting in the stored DDL. That is the normal way this schema grows.
**`DateAdded` is not written consistently.** 126,423 rows hold ISO `yyyy-MM-dd HH:mm:ss`;
17,944 hold US-format `M/d/yy` from a bulk import. As text those two sort into different
@@ -210,53 +208,57 @@ Crawler bookkeeping. Rolodex ignores all of these.
---
## `Blogs.IsDeleted` — added by Rolodex
## `Blogs.IsActive` — now written by two applications
`IsActive` has always been the crawler's work-selection flag. `GetBlogs` in
`DataAccess.cs` joins on it to decide what to collect:
```sql
ALTER TABLE Blogs ADD COLUMN IsDeleted INTEGER NOT NULL DEFAULT 0;
SELECT NoteBlogName, count(*) FROM notes
INNER JOIN blogs ON blogs.BlogName = notes.NoteBlogName
WHERE blogs.IsActive = @isActive AND ...
```
**This column is not written by the crawler.** Rolodex adds it automatically at startup if
it is missing, and uses it to hide a blog from its own UI without destroying anything:
Nothing inside the crawler *writes* it — it is an input, set from outside.
- `0` — the blog is live. `NOT NULL DEFAULT 0` means every existing row, and every row the
crawler creates afterwards, starts here.
- `1` — somebody removed the blog through Rolodex.
**Rolodex is now one of the things that sets it.** Removing a blog through the Rolodex UI
runs exactly this:
Removal is a soft delete and nothing else changes: the row keeps every crawl flag it had,
and its `Posts` and `Notes` rows are untouched. Restoring is the same `UPDATE` setting the
column back to `0`.
```sql
UPDATE Blogs SET IsActive = 0 WHERE BlogName = ?;
```
**Do not write `NULL` here.** Rolodex reads the column through `COALESCE(IsDeleted, 0)`
precisely so a NULL cannot strand a row — without it a NULL satisfies neither the live
test nor its negation, and the blog would disappear from the registry *and* from the
removed list with no way back through the UI. The `NOT NULL` declaration is the real
guard; the `COALESCE` is the belt to its braces.
Rolodex adds no column and changes no schema. It reuses this flag because the two meanings
were judged to be one decision: a blog you do not want in the browsing UI is a blog you do
not want to keep crawling. Removal therefore stops collection, and the Rolodex
confirmation screen says so before anyone commits.
- `1` (or absent/NULL) — live. Crawled, and visible in Rolodex.
- `0` — removed. Not crawled, hidden from the Rolodex registry, dashboard counts and
engagement rollups.
Restoring is the same `UPDATE` with a `1`. Nothing is destroyed either way: the blog's
`Posts` and `Notes` rows are never touched, and Rolodex deliberately keeps showing them
under its Posts and Notes pages. Removing a blog hides the blog, not what it collected.
### What other tools need to know
1. **Adding the column is safe for `SELECT`, and for `INSERT` that names its columns.** It
is *not* safe for `INSERT INTO Blogs VALUES (...)` without a column list — that form
breaks the moment column count changes. If any crawler does this, give it an explicit
column list.
2. **The crawler should leave the column alone.** Writing to it would silently un-remove or
remove blogs behind Rolodex's back.
3. **Re-crawling a removed blog will not bring it back.** An `INSERT OR REPLACE` on the
`Blogs` row *would*, by resetting the column to its default `0`. If that matters,
prefer an `UPDATE` of the specific columns, or `INSERT … ON CONFLICT DO UPDATE SET`
naming only the crawl columns.
4. **A tool that lists blogs should decide whether it cares.** Rolodex filters
`WHERE COALESCE(IsDeleted, 0) = 0` everywhere a blog surfaces. A crawler probably
should not — a blog hidden from a browsing UI is not necessarily one to stop crawling.
That is a deliberate choice, not an oversight.
Rolodex degrades if the column is absent — a read-only or `immutable=1` deployment cannot
take the `ALTER` — by falling back to its previous behaviour of showing every blog. So
removing the column is a supported way to back the feature out:
```sql
ALTER TABLE Blogs DROP COLUMN IsDeleted;
```
1. **Setting `IsActive = 0` now also hides the blog from Rolodex**, and setting it back to
`1` makes it reappear. If another tool deactivates blogs in bulk, it is also removing
them from the browsing UI — which may be exactly right, but it is no longer a
crawler-only decision.
2. **Re-crawling a removed blog will not bring it back**, since nothing in the crawler
writes the flag. An `INSERT OR REPLACE` on the `Blogs` row *would*, by resetting it to
the column default of `1`. Prefer an `UPDATE` of the specific columns, or
`INSERT … ON CONFLICT DO UPDATE SET` naming only the columns being refreshed.
3. **NULL is treated as live.** The column is `INTEGER DEFAULT 1` with no `NOT NULL`, so
Rolodex reads it through `COALESCE(IsActive, 1)`. A NULL therefore leaves the blog
visible rather than stranding it outside both the registry and the removed list, where
no screen could reach it. Write `0` or `1`, not NULL.
4. **Backing the feature out is a configuration change, not a migration.** Because there is
no Rolodex-owned column, setting `Rolodex__EnableBlogDeletion=false` is the whole of it;
there is nothing to drop. Any blogs already at `IsActive = 0` simply go back to being
ordinary inactive blogs.
---