jim
7bf270e47d
The zomb-eh periodic re-queue branch in GetPosts had no lower bound on the post's original PostDate -- it re-queued every already-collected zomb-eh post past the 3-day cooldown, regardless of age. Add --fromDate <datetime> to bound that branch by PostDate >= the given date. It stacks with the existing cooldown clause rather than replacing it, so it applies the same way whether or not --force also drops the cooldown.
60 lines
2.7 KiB
Markdown
60 lines
2.7 KiB
Markdown
# AI Development Instructions for URLNotesGrabberCORE
|
|
|
|
This document provides essential context for AI agents working with URLNotesGrabberCORE, a .NET Core application designed to process and analyze blog notes and interactions.
|
|
|
|
## Project Architecture
|
|
|
|
### Core Components
|
|
- **Program.cs**: Entry point and command handler for various operations (parsing, testing, collecting notes)
|
|
- **DataAccess.cs**: SQLite database operations for storing blog and note data
|
|
- **ResponseNotes.cs**: Data models for API responses
|
|
- **appsettings.json**: Configuration for paths and application settings
|
|
|
|
### Data Flow
|
|
1. Input text files are processed from configured input path
|
|
2. Blog data is extracted and stored in SQLite database
|
|
3. API calls collect notes/interactions for each blog post
|
|
4. Results are written to configured output paths
|
|
|
|
## Key Development Workflows
|
|
|
|
### Building and Running
|
|
```powershell
|
|
dotnet build
|
|
dotnet run # Process all files in input directory
|
|
dotnet run -- --parse [blogname] # Process specific blog
|
|
dotnet run -- --test [blogname] [postID] # Test API for specific post
|
|
```
|
|
|
|
### Command-Line Interface
|
|
- `--parse [blogname]`: Parse text files for specific blog
|
|
- `--test [blogname] [postID]`: Test API note collection
|
|
- `--posts`: Export post blogs to file
|
|
- `--blogs`: Export blog list to file
|
|
- `--collect [0|1] [datetime] [blogname]`: Collect notes for posts in DB. Optional `blogname` restricts the run to one blog (exact match), e.g. `--collect 1 zomb-eh`. Add `--force` to ignore the periodic re-collect cooldown so already-collected posts are re-queued immediately (mode 1 only). Add `--fromDate <datetime>` to only re-queue already-collected posts whose original PostDate is on/after that date (mode 1 only; applies with or without `--force`)
|
|
- `--blogsR`: Export reply blogs to file
|
|
- `--blogsO [start] [stop]`: Export blogs within range
|
|
|
|
## Project Conventions
|
|
|
|
### Configuration
|
|
- All paths and settings are managed in `appsettings.json`
|
|
- Command-line args override config file settings
|
|
- SQLite database is used for persistent storage
|
|
|
|
### API Integration
|
|
- Uses RestSharp for API calls with OAuth authentication
|
|
- Rate limiting implemented via `System.Threading.RateLimiting`
|
|
- Resilient HTTP handling with `Microsoft.Extensions.Http.Resilience`
|
|
|
|
### Data Models
|
|
- `ReblogRecord`: Core data structure for blog post information
|
|
- `Note`: Represents interaction data from API responses
|
|
- Default field values are "." (period) to handle null cases
|
|
|
|
## Integration Points
|
|
- SQLite Database: Primary data store (`System.Data.SQLite`)
|
|
- REST API: External blog platform API (OAuth authentication)
|
|
- File System: Input/Output for text file processing
|
|
|
|
For questions or clarifications, please refer to the codebase or request updates to these instructions. |