ReblogRecord (TraverseDirectory's .txt-export parser) and the --likes API
path both default every content field to the literal "." when their source
has no value for that field, then pass it straight into UpdatePost. A blog
with two export folders in different field formats -- a duplicate "_2"
folder, or an export whose field set changed over time -- sends one record
with real Title/Tags/Slug and another with those fields "." because that
format never had a line for them. Re-importing both on every run flipped
the row back and forth forever: net content never changed, but
DateModified moved on every pass since each write really did change a
column relative to the other write, just not relative to the true value.
Every content column in UpdatePost's SET list is now guarded the same way
RootBlogName/RootURL already were -- a "." parameter leaves the existing
value alone instead of overwriting it -- and the change-detection WHERE
clause carries the same exception, so a "."-only difference no longer
fires the UPDATE at all. Deliberately narrow: only the literal "." is the
sentinel, so an explicit empty string from a real record still overwrites.
Verified against a throwaway DB using the exact SQL text and parameter
binding from UpdatePost, reproducing the an-angry-wolf/adore-blk scenario
found in the live DB: re-importing conflicting "." records now writes zero
rows and leaves DateModified untouched, while a genuine content change
still fires and still moves it.
Co-Authored-By: Claude Opus 5 <[email protected]>
Seven UPDATE statements wrote DateModified unconditionally, so re-crawling
or re-ingesting identical content marked Blogs, Posts and Notes rows as
modified. Each now carries a WHERE guard covering every column in its SET
list, so SQLite matches zero rows on a no-op.
Guarded: AddPost's insert-failure fallback and blog stamp, AddNote's blog
stamp, UpdateBlogLikesNewestTimestamp, UpdateNoteReplyText,
UpsertPostFromTextFile, SetBlogTTFolderPath, UpdatePostContentFields.
Also:
- Blogs.DateAdded is no longer rewritten when a new post arrives for a
known blog. A new post is not a new blog, and rewriting the column both
destroyed the registration date and made every insert look like a change.
- Posts.NotesGatheredDateTime is crawl bookkeeping that moves on every
pass, so it no longer moves DateModified on its own. It is still written
each pass, but the timestamp is wrapped in a CASE on the pre-UPDATE
HasNotesGathered value so only the flag flipping counts.
These statements now return 0 rows for "found but unchanged" as well as
"not found"; CorrectMode's postsUpdated tally consequently counts rows
actually changed, matching what its dry-run diff reports.
Co-Authored-By: Claude Opus 5 <[email protected]>
A non-JSON response body (CDN 403/5xx HTML, empty body, transport error)
never reached the Tumblr API, so it says nothing about the post being
fetched. These were recorded as FAILURE, which consumed the post's single
attempt for the pass and cleared the API key's rate-limit flag on the way
through.
Classify them as Root.transientFailure and retry in place (1s/4s/10s)
before skipping. Skipped posts stay unmarked in the DB so a later launch
retries them. Ten consecutive transient failures now aborts the pass
rather than skipping post-by-post against an edge refusing all traffic.
Also:
- MarkAvailable() only on a response that reached the API, and it is now
a no-op when the key was not flagged (was writing to the DB and logging
on every single call)
- Only a real 429 counts as a rate limit; stop inferring one from
X-RateLimit-* headers, which Tumblr sends on every response
- Limiters pace with AcquireAsync instead of AttemptAcquire, which did
not wait and aborted the run once a window was saturated
- Throttle --collect and --likes from 300/min to 60/min
- Log one line per transient failure instead of the HTML body and stack
trace; keep full detail only for a 2xx that fails to parse
- --collect returns exit 3 when a pass ends incomplete
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Note TraverseDirectory's recognized field prefixes, multi-line
Body/Downloaded files continuation, and how RootURL now gets
populated from both the .txt Reblog root url line and the API-based
--likes flow.
jimandClaude Opus 5