jim/URLNotesGrabberCore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: stop "." export sentinel from clobbering real post content

Browse Source 
ReblogRecord (TraverseDirectory's .txt-export parser) and the --likes API
path both default every content field to the literal "." when their source
has no value for that field, then pass it straight into UpdatePost. A blog
with two export folders in different field formats -- a duplicate "_2"
folder, or an export whose field set changed over time -- sends one record
with real Title/Tags/Slug and another with those fields "." because that
format never had a line for them. Re-importing both on every run flipped
the row back and forth forever: net content never changed, but
DateModified moved on every pass since each write really did change a
column relative to the other write, just not relative to the true value.

Every content column in UpdatePost's SET list is now guarded the same way
RootBlogName/RootURL already were -- a "." parameter leaves the existing
value alone instead of overwriting it -- and the change-detection WHERE
clause carries the same exception, so a "."-only difference no longer
fires the UPDATE at all. Deliberately narrow: only the literal "." is the
sentinel, so an explicit empty string from a real record still overwrites.

Verified against a throwaway DB using the exact SQL text and parameter
binding from UpdatePost, reproducing the an-angry-wolf/adore-blk scenario
found in the live DB: re-importing conflicting "." records now writes zero
rows and leaves DateModified untouched, while a genuine content change
still fires and still moves it.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
jim
2026-08-05 09:17:20 -05:00
co-authored by Claude Opus 5
parent e1d2eb48c2
commit 83e35a2323
2 changed files with 70 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+23
View File
@@ -101,6 +101,29 @@ new post arriving for a known blog reopens `HasBeenOutput` but must leave `DateA
a new post is not a new blog, and rewriting the column both destroys the registration date
and makes every insert look like a change.
**`"."` in a `Posts` content field means "not supplied", not "empty".** `ReblogRecord`
(`TraverseDirectory`'s parser for the local `.txt` export tree) and the `--likes` API path
both default every content field to the literal string `"."` when their source has no value
for it, then pass that straight to `UpdatePost`. A blog with two export folders in different
field formats (a duplicate `_2` folder, or a Tumblr export whose field set changed over time)
sends one record with a real `Title`/`Tags`/`Slug` and another with those fields `"."`
because that format never had a line for them — and without a guard, re-importing both on
every run flips the row back and forth forever, bumping `DateModified` on every pass even
though the true content never changes.
- Every content column in `UpdatePost`'s `SET` list is guarded the same way `RootBlogName`/
`RootURL` already were: `col = CASE WHEN @col = '.' THEN col ELSE @col END`. A `"."`
parameter leaves the existing value alone instead of overwriting it
- The change-detection `WHERE` clause carries the same exception —
`(@col <> '.' AND IFNULL(col, '') <> @col) OR ...` — so a `"."`-only difference does not
make the statement fire at all, and `DateModified` stays put
- Deliberately narrow: only the literal `"."` is the sentinel. An explicit empty string from
a real record still overwrites, same as before this fix. `postID`, `BlogName`, `hasImage`,
`ByLikes` are not part of this convention and are unaffected
- If a new content field is added to `Posts`/`UpdatePost`, decide explicitly whether its
source can legitimately supply `"."` as "field absent" before deciding whether it needs
the same `CASE` treatment — don't assume every column needs it
### Testing
- No existing test suite; use xUnit if adding tests
- Test critical logic: `ApiKeyPool` init, color parsing, config persistence
URLNotesGrabberCORE/DataAccess.cs
+47 -36
View File
@@ -1532,49 +1532,60 @@ namespace URLNotesGrabberCORE
{
if (ownsConnection) connection.Open();
// "." is TraverseDirectory/ReblogRecord's sentinel for "this field had no
// matching line in this particular export file" -- not an empty value. A blog
// with two export files in different formats (e.g. an "_2" duplicate folder, or
// a Tumblr export whose field set changed over time) sends one record with a real
// Title and another with Title = "." for the same PostID, and re-importing both
// on every run must not let the "not supplied" record blank out what the other
// one has. Every content field below is CASE-guarded the same way RootBlogName/
// RootURL already were, and the change-detection ignores "." too so a "."-only
// difference doesn't fire the UPDATE (and bump DateModified) on its own. Only "."
// is treated as the sentinel -- an explicit empty string from a real field still
// overwrites, same as before.
string sql = "UPDATE Posts SET ";
sql += "postDate = @postDate, ";
sql += "reblogURL = @reblogURL, ";
sql += "postURL = @postURL, ";
sql += "slug = @slug, ";
sql += "reblogKey = @reblogKey, ";
sql += "reblogName = @reblogName, ";
sql += "summary = @summary, ";
sql += "quote = @quote, ";
sql += "body = @body, ";
sql += "tags = @tags, ";
sql += "link = @link, ";
sql += "photoURL = @photoURL, ";
sql += "photoCaption = @photoCaption, ";
sql += "downloadedFiles = @downloadedFiles, ";
sql += "audioCaption = @audioCaption, ";
sql += "question = @question, ";
sql += "answer = @answer, ";
sql += "title = @title, ";
sql += "postDate = CASE WHEN @postDate = '.' THEN postDate ELSE @postDate END, ";
sql += "reblogURL = CASE WHEN @reblogURL = '.' THEN reblogURL ELSE @reblogURL END, ";
sql += "postURL = CASE WHEN @postURL = '.' THEN postURL ELSE @postURL END, ";
sql += "slug = CASE WHEN @slug = '.' THEN slug ELSE @slug END, ";
sql += "reblogKey = CASE WHEN @reblogKey = '.' THEN reblogKey ELSE @reblogKey END, ";
sql += "reblogName = CASE WHEN @reblogName = '.' THEN reblogName ELSE @reblogName END, ";
sql += "summary = CASE WHEN @summary = '.' THEN summary ELSE @summary END, ";
sql += "quote = CASE WHEN @quote = '.' THEN quote ELSE @quote END, ";
sql += "body = CASE WHEN @body = '.' THEN body ELSE @body END, ";
sql += "tags = CASE WHEN @tags = '.' THEN tags ELSE @tags END, ";
sql += "link = CASE WHEN @link = '.' THEN link ELSE @link END, ";
sql += "photoURL = CASE WHEN @photoURL = '.' THEN photoURL ELSE @photoURL END, ";
sql += "photoCaption = CASE WHEN @photoCaption = '.' THEN photoCaption ELSE @photoCaption END, ";
sql += "downloadedFiles = CASE WHEN @downloadedFiles = '.' THEN downloadedFiles ELSE @downloadedFiles END, ";
sql += "audioCaption = CASE WHEN @audioCaption = '.' THEN audioCaption ELSE @audioCaption END, ";
sql += "question = CASE WHEN @question = '.' THEN question ELSE @question END, ";
sql += "answer = CASE WHEN @answer = '.' THEN answer ELSE @answer END, ";
sql += "title = CASE WHEN @title = '.' THEN title ELSE @title END, ";
sql += "DateModified = @dateModified, ";
sql += "RootBlogName = CASE WHEN @rootBlogName IS NULL OR @rootBlogName = '' OR @rootBlogName = '.' THEN RootBlogName ELSE @rootBlogName END, ";
sql += "RootURL = CASE WHEN @rootURL IS NULL OR @rootURL = '' OR @rootURL = '.' THEN RootURL ELSE @rootURL END, ";
sql += "hasImage = @hasImage, ";
sql += "ByLikes = MAX(IFNULL(ByLikes, 0), @byLikes) ";
sql += " WHERE BlogName = @BlogName AND PostID = @PostID AND (";
sql += "IFNULL(postDate, '') <> @postDate OR ";
sql += "IFNULL(reblogURL, '') <> @reblogURL OR ";
sql += "IFNULL(postURL, '') <> @postURL OR ";
sql += "IFNULL(slug, '') <> @slug OR ";
sql += "IFNULL(reblogKey, '') <> @reblogKey OR ";
sql += "IFNULL(reblogName, '') <> @reblogName OR ";
sql += "IFNULL(summary, '') <> @summary OR ";
sql += "IFNULL(quote, '') <> @quote OR ";
sql += "IFNULL(body, '') <> @body OR ";
sql += "IFNULL(tags, '') <> @tags OR ";
sql += "IFNULL(link, '') <> @link OR ";
sql += "IFNULL(photoURL, '') <> @photoURL OR ";
sql += "IFNULL(photoCaption, '') <> @photoCaption OR ";
sql += "IFNULL(downloadedFiles, '') <> @downloadedFiles OR ";
sql += "IFNULL(audioCaption, '') <> @audioCaption OR ";
sql += "IFNULL(question, '') <> @question OR ";
sql += "IFNULL(answer, '') <> @answer OR ";
sql += "IFNULL(title, '') <> @title OR ";
sql += "(@postDate <> '.' AND IFNULL(postDate, '') <> @postDate) OR ";
sql += "(@reblogURL <> '.' AND IFNULL(reblogURL, '') <> @reblogURL) OR ";
sql += "(@postURL <> '.' AND IFNULL(postURL, '') <> @postURL) OR ";
sql += "(@slug <> '.' AND IFNULL(slug, '') <> @slug) OR ";
sql += "(@reblogKey <> '.' AND IFNULL(reblogKey, '') <> @reblogKey) OR ";
sql += "(@reblogName <> '.' AND IFNULL(reblogName, '') <> @reblogName) OR ";
sql += "(@summary <> '.' AND IFNULL(summary, '') <> @summary) OR ";
sql += "(@quote <> '.' AND IFNULL(quote, '') <> @quote) OR ";
sql += "(@body <> '.' AND IFNULL(body, '') <> @body) OR ";
sql += "(@tags <> '.' AND IFNULL(tags, '') <> @tags) OR ";
sql += "(@link <> '.' AND IFNULL(link, '') <> @link) OR ";
sql += "(@photoURL <> '.' AND IFNULL(photoURL, '') <> @photoURL) OR ";
sql += "(@photoCaption <> '.' AND IFNULL(photoCaption, '') <> @photoCaption) OR ";
sql += "(@downloadedFiles <> '.' AND IFNULL(downloadedFiles, '') <> @downloadedFiles) OR ";
sql += "(@audioCaption <> '.' AND IFNULL(audioCaption, '') <> @audioCaption) OR ";
sql += "(@question <> '.' AND IFNULL(question, '') <> @question) OR ";
sql += "(@answer <> '.' AND IFNULL(answer, '') <> @answer) OR ";
sql += "(@title <> '.' AND IFNULL(title, '') <> @title) OR ";
sql += "IFNULL(hasImage, 0) <> @hasImage OR ";
sql += "(@byLikes = 1 AND IFNULL(ByLikes, 0) = 0) OR ";
sql += "((@rootBlogName IS NOT NULL AND @rootBlogName <> '' AND @rootBlogName <> '.') AND IFNULL(RootBlogName, '') <> @rootBlogName) OR ";