fix: retry transient CDN failures instead of failing the post
A non-JSON response body (CDN 403/5xx HTML, empty body, transport error) never reached the Tumblr API, so it says nothing about the post being fetched. These were recorded as FAILURE, which consumed the post's single attempt for the pass and cleared the API key's rate-limit flag on the way through. Classify them as Root.transientFailure and retry in place (1s/4s/10s) before skipping. Skipped posts stay unmarked in the DB so a later launch retries them. Ten consecutive transient failures now aborts the pass rather than skipping post-by-post against an edge refusing all traffic. Also: - MarkAvailable() only on a response that reached the API, and it is now a no-op when the key was not flagged (was writing to the DB and logging on every single call) - Only a real 429 counts as a rate limit; stop inferring one from X-RateLimit-* headers, which Tumblr sends on every response - Limiters pace with AcquireAsync instead of AttemptAcquire, which did not wait and aborted the run once a window was saturated - Throttle --collect and --likes from 300/min to 60/min - Log one line per transient failure instead of the HTML body and stack trace; keep full detail only for a 2xx that fails to parse - --collect returns exit 3 when a pass ends incomplete Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
@@ -85,6 +85,10 @@ namespace URLNotesGrabberCORE
|
||||
public int retryInSeconds { get; set; }
|
||||
|
||||
public string rawJson { get; set; }
|
||||
|
||||
// The request never reached the Tumblr API (transport error, or an edge/CDN response with a
|
||||
// non-JSON body). Says nothing about the post, so the caller should retry rather than fail it.
|
||||
public bool transientFailure { get; set; }
|
||||
}
|
||||
|
||||
// Classes for Posts API endpoint (for reply_text)
|
||||
|
||||
Reference in New Issue
Block a user