fix: make --output and --updatepaths tell the truth about TTFolderPath
--output iterated all 156k active Blogs rows and printed a "does not exist or is not set" skip line for each, which is nearly every blog in the crawl registry -- only the few hundred downloaded locally ever have a folder. The signal was buried in six figures of noise. GetAllBlogsWithTTFolderPath now selects only active blogs carrying a non-empty path, so --output processes export targets and nothing else. When none exist it says so once, names the database it read, points at --updatepaths, and returns non-zero instead of reporting success. A stored path this machine cannot see is now reported separately from an unset one, with the path shown, because the two are fixed in different places. Paths are trimmed before Directory.Exists, which stray whitespace in a .tumblr FileDownloadLocation would otherwise defeat. Both writers counted optimistically. UpdateBlogPathsRunner printed its per-blog success line and incremented its total from the metadata file parsing, never checking whether the UPDATE matched a row; LegacyPostsDbImporter counted a blog as copied even when the legacy TTFolderPath was NULL. Either could report full success having written nothing -- which is consistent with TL.db holding zero populated paths across all 156,492 active blogs despite 20,679 posts having merged. SetBlogTTFolderPath now returns whether a row changed, and both callers report written / already-correct / no-matching-row separately. Verified against a throwaway database: no-paths case, export case (stale .txt rotated to .bak, per-PostType files, date-sorted), missing-folder case, --updatepaths honest counts, and an idempotent rerun reporting already-correct. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
@@ -29,6 +29,8 @@ namespace URLNotesGrabberCORE
|
||||
Console.WriteLine($"Reading legacy posts.db: {legacyDbPath}");
|
||||
|
||||
int blogsCopied = 0;
|
||||
int blogPathsWritten = 0;
|
||||
int blogsWithoutPath = 0;
|
||||
int postsUpserted = 0;
|
||||
int errors = 0;
|
||||
|
||||
@@ -48,7 +50,13 @@ namespace URLNotesGrabberCORE
|
||||
if (string.IsNullOrWhiteSpace(blogName)) continue;
|
||||
try
|
||||
{
|
||||
DataAccess.SetBlogTTFolderPath(blogName, ttFolderPath);
|
||||
// A legacy row whose TTFolderPath was already NULL copies nothing.
|
||||
// Counting it as "copied" is what hid the fact that this import has
|
||||
// never populated a single path.
|
||||
if (string.IsNullOrWhiteSpace(ttFolderPath))
|
||||
blogsWithoutPath++;
|
||||
else if (DataAccess.SetBlogTTFolderPath(blogName, ttFolderPath.Trim()))
|
||||
blogPathsWritten++;
|
||||
blogsCopied++;
|
||||
}
|
||||
catch (Exception ex)
|
||||
@@ -58,7 +66,7 @@ namespace URLNotesGrabberCORE
|
||||
}
|
||||
}
|
||||
}
|
||||
Console.WriteLine($" Blogs copied: {blogsCopied}");
|
||||
Console.WriteLine($" Blogs seen: {blogsCopied}, TTFolderPath written: {blogPathsWritten}, legacy rows with no path: {blogsWithoutPath}");
|
||||
|
||||
// 2) Copy Posts
|
||||
try
|
||||
@@ -138,7 +146,8 @@ namespace URLNotesGrabberCORE
|
||||
}
|
||||
|
||||
Console.WriteLine($"\n========== Legacy import summary ==========");
|
||||
Console.WriteLine($"Blogs copied: {blogsCopied}");
|
||||
Console.WriteLine($"Blogs seen: {blogsCopied}");
|
||||
Console.WriteLine($"Paths written: {blogPathsWritten} (legacy rows with no path: {blogsWithoutPath})");
|
||||
Console.WriteLine($"Posts upserted: {postsUpserted}");
|
||||
Console.WriteLine($"Errors: {errors}");
|
||||
return errors == 0 ? 0 : 2;
|
||||
|
||||
Reference in New Issue
Block a user