jim/URLNotesGrabberCore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Improve import speed, add URL dump, and CLI options

Browse Source 
- Add SQLite import mode pragmas for faster bulk inserts
- Implement -urldump command to extract all URLs from posts
- Add -start [blogname] CLI option for partial traversal
- Support toggling file logging and record import logging via config
- Only update DB rows if values change to reduce writes
- Add DateCreated fields to relevant tables
- Enhance logging and output formatting
- Refactor directory traversal for better control and reporting
This commit is contained in:
jim
2026-04-15 15:14:43 -05:00
parent 390e1284cb
commit 059c9cd527
4 changed files with 581 additions and 238 deletions
Download Patch File Download Diff File Expand all files Collapse all files
URLNotesGrabberCORE/appsettings.json
+4 -1
View File
@@ -6,9 +6,12 @@
"PathOutputPosts": "u:\\jim\\Documents\\Web Copies\\blogs\\GetPosts.txt",
"PathOutputBlogs": "u:\\jim\\Documents\\Web Copies\\blogs\\GetBlogs.txt",
"PathOutputReplies": "u:\\jim\\Documents\\Web Copies\\blogs\\GetReplies.txt",
"PathOutputUrls": "u:\\jim\\Documents\\Web Copies\\blogs\\GetUrls.txt",
"PathDB": "u:\\jim\\Documents\\Web Copies\\blogs\\TL.db",
"ContainsList": "zombaee,zomb-eh,ahzombae,thebugandme,lovingbabybug,h4rdspot",
"PostIDToExclude": "730084470076686336,726309940037287936,639974102120235008,184045126218"
"PostIDToExclude": "730084470076686336,726309940037287936,639974102120235008,184045126218",
"EnableFileLogging": false,
"LogTraversalRecordImports": false
},
"TumblrApi": {
"ConsumerKey": "PtsBCGumcsgihyynxUh8b47Jfmi7uXhEIOU46bmhdsUSJ5mEP3",