URLNotesGrabberCore/.github/copilot-instructions.md

AI Development Instructions for URLNotesGrabberCORE

This document provides essential context for AI agents working with URLNotesGrabberCORE, a .NET Core application designed to process and analyze blog notes and interactions.

Project Architecture

Core Components

• Program.cs: Entry point and command handler for various operations (parsing, testing, collecting notes)
• DataAccess.cs: SQLite database operations for storing blog and note data
• ResponseNotes.cs: Data models for API responses
• appsettings.json: Configuration for paths and application settings

Data Flow

1. Input text files are processed from configured input path
2. Blog data is extracted and stored in SQLite database
3. API calls collect notes/interactions for each blog post
4. Results are written to configured output paths

Key Development Workflows

Building and Running

dotnet build
dotnet run            # Process all files in input directory
dotnet run -- -parse [blogname]  # Process specific blog
dotnet run -- -test [blogname] [postID]  # Test API for specific post

Command-Line Interface

• -parse [blogname]: Parse text files for specific blog
• -test [blogname] [postID]: Test API note collection
• -posts: Export post blogs to file
• -blogs: Export blog list to file
• -collect: Collect notes for all posts in DB
• -blogsR: Export reply blogs to file
• -blogsO [start] [stop]: Export blogs within range

Project Conventions

Configuration

• All paths and settings are managed in appsettings.json
• Command-line args override config file settings
• SQLite database is used for persistent storage

API Integration

• Uses RestSharp for API calls with OAuth authentication
• Rate limiting implemented via System.Threading.RateLimiting
• Resilient HTTP handling with Microsoft.Extensions.Http.Resilience

Data Models

• ReblogRecord: Core data structure for blog post information
• Note: Represents interaction data from API responses
• Default field values are "." (period) to handle null cases

Integration Points

• SQLite Database: Primary data store (System.Data.SQLite)
• REST API: External blog platform API (OAuth authentication)
• File System: Input/Output for text file processing

For questions or clarifications, please refer to the codebase or request updates to these instructions.