fix: retry transient CDN failures instead of failing the post

A non-JSON response body (CDN 403/5xx HTML, empty body, transport error)
never reached the Tumblr API, so it says nothing about the post being
fetched. These were recorded as FAILURE, which consumed the post's single
attempt for the pass and cleared the API key's rate-limit flag on the way
through.

Classify them as Root.transientFailure and retry in place (1s/4s/10s)
before skipping. Skipped posts stay unmarked in the DB so a later launch
retries them. Ten consecutive transient failures now aborts the pass
rather than skipping post-by-post against an edge refusing all traffic.

Also:
- MarkAvailable() only on a response that reached the API, and it is now
  a no-op when the key was not flagged (was writing to the DB and logging
  on every single call)
- Only a real 429 counts as a rate limit; stop inferring one from
  X-RateLimit-* headers, which Tumblr sends on every response
- Limiters pace with AcquireAsync instead of AttemptAcquire, which did
  not wait and aborted the run once a window was saturated
- Throttle --collect and --likes from 300/min to 60/min
- Log one line per transient failure instead of the HTML body and stack
  trace; keep full detail only for a 2xx that fails to parse
- --collect returns exit 3 when a pass ends incomplete

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
jim
2026-07-22 12:10:49 -05:00
co-authored by Claude Opus 4.8
parent 4df73367fb
commit 003a504d5e
4 changed files with 200 additions and 185 deletions
+4
View File
@@ -85,6 +85,10 @@ namespace URLNotesGrabberCORE
public int retryInSeconds { get; set; }
public string rawJson { get; set; }
// The request never reached the Tumblr API (transport error, or an edge/CDN response with a
// non-JSON body). Says nothing about the post, so the caller should retry rather than fail it.
public bool transientFailure { get; set; }
}
// Classes for Posts API endpoint (for reply_text)