66 lines
3.9 KiB
Markdown
66 lines
3.9 KiB
Markdown
# AGENTS.md
|
|
|
|
## Project Overview
|
|
- **Primary Language**: C# (.NET 8 Console Application)
|
|
- **Key Libraries**: RestSharp, Newtonsoft.Json, System.Data.SQLite, Microsoft.Extensions.Configuration
|
|
- **Purpose**: Tumblr API data harvester for collecting notes, posts, likes, and replies, storing results in SQLite.
|
|
|
|
## Architectural Patterns
|
|
- CLI entry point in `Program.cs` with workflow orchestration
|
|
- `DataAccess.cs`: Database operations, `ApiKeyPool` (API key management), `APIAccess` (Tumblr client)
|
|
- `ResponseNotes.cs`: Tumblr API response models
|
|
- Round-robin API key rotation with rate-limit tracking
|
|
- Automatic console color assignment per API key for output differentiation
|
|
- No-argument mode (`Program.TraverseDirectory`) ingests `.txt` blog export files into `Posts` via `DataAccess.AddPost`. Recognized field prefixes live in `TraverseDirectoryFieldPrefixes`; `Body:` and `Downloaded files:` collect every following line up to the next recognized prefix (multi-line values). `RootURL` is populated from a `Reblog root url:` line the same way it's populated from the API-based `--likes` flow — both paths converge on `DataAccess.AddPost`'s `rootURL` parameter, which `UpdatePost` only overwrites when the incoming value is non-empty (existing `RootURL` is preserved otherwise)
|
|
|
|
## Developer Guidelines
|
|
|
|
### Code Formatting
|
|
- 4-space indentation, no tabs, match existing C# style
|
|
- PascalCase for public members, camelCase for locals
|
|
- Minimize code comments unless explicitly requested
|
|
- Use only existing project libraries; no new dependencies without confirmation
|
|
- Match accessibility modifiers (`public` for models, `internal` for helpers)
|
|
|
|
### Error Handling
|
|
- Wrap file/network operations in `try-catch`
|
|
- Log non-critical errors (e.g., config write failures) with `[Warning]` prefix
|
|
- Preserve console color state: use save/restore pattern for temporary color changes
|
|
- API rate limits must use `ApiKeyPool.MarkRateLimited()`/`MarkAvailable()`
|
|
|
|
### API Failure Classification
|
|
Tumblr sits behind a CDN that returns HTML error pages (403, 5xx) which never reach the API. These
|
|
say nothing about the item being fetched, so they must not be recorded as per-item failures.
|
|
|
|
- A response body that will not parse as JSON did not come from the API. Flag it with
|
|
`Root.transientFailure`, never as `FAILURE`
|
|
- Transient failures retry in place (`TransientBackoffSeconds`) before the item is skipped; a skipped
|
|
item stays unmarked in the DB so a later launch retries it
|
|
- `MaxConsecutiveTransient` consecutive transient failures aborts the pass rather than skipping
|
|
item-by-item against an edge that is refusing all traffic
|
|
- Only call `ApiKeyPool.MarkAvailable()` on a response that actually reached the API. A transport or
|
|
CDN failure says nothing about the key's standing and must not clear its flag
|
|
- Only a real HTTP 429 (or `meta.status == 429`) counts as a rate limit. Do not infer one from the
|
|
presence of `X-RateLimit-*` headers, which Tumblr sends on every response
|
|
- Rate limiters must pace with `await AcquireAsync()`. `AttemptAcquire()` does not wait, so a
|
|
saturated window aborts the run instead of throttling it
|
|
- Long-running commands return exit 3 when a pass ends incomplete (rate-limit pause, breaker trip, or
|
|
skipped items), so a caller can distinguish that from a clean run
|
|
|
|
### Testing
|
|
- No existing test suite; use xUnit if adding tests
|
|
- Test critical logic: `ApiKeyPool` init, color parsing, config persistence
|
|
- Avoid testing one-off CLI workflows
|
|
|
|
### Git Commit Messages
|
|
- Imperative mood ("Add feature" not "Added feature")
|
|
- Prefix with type: `feat:`, `fix:`, `chore:`, `docs:`
|
|
- Keep messages under 72 characters
|
|
- Never commit sensitive data (API keys/tokens)
|
|
|
|
### API Key Color Rules
|
|
- Unconfigured keys auto-assign colors from a preset palette
|
|
- Auto-assigned colors persist to `appsettings.json`
|
|
- All output for an active key uses its assigned color
|
|
- Temporary color changes (e.g., errors) must restore the key's color afterward
|