diff --git a/README.md b/README.md index 8d5a8d9..98f12f4 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,9 @@ file's own container header for its real resolution, frame rate, duration, and codecs — no ffmpeg/MediaInfo dependency required — filling in `{resolution}`/ `{video codec}`/`{audio codec}` whenever the filename itself didn't already carry that tag, plus `{width}`/`{height}`/`{fps}`/`{duration}`/`{file size}`, which have no -filename-tag equivalent at all; see "Real media-file probing" below. The Align +filename-tag equivalent at all; see "Real media-file probing" below. A separate **Hash +Files** button computes real `{crc32}`/`{md5}`/`{sha1}` checksums on demand (deliberately +not automatic, since it reads a file's entire contents — see "Hash Files" below). The Align Up/Down buttons reorder the (shared) file list — the two panes always stay positionally paired — and up to 9 naming schemes can be saved as presets, recalled with a click or a `1`-`9` keypress (persisted locally, same pattern as @@ -240,14 +242,18 @@ src-tauri/ Tauri app shell: exposes aurora-curator-core as commands `browse_episodes` command backing the Episodes panel — series search + one season's episode list, via a `build_tv_client` helper now shared with `metadata.rs`'s `fetch_metadata`, so both read the - same stored TV-source setting and credential), `sfv.rs` (three - commands backing the SFV panel — `compute_checksums` over a batch - of loaded files with per-file error isolation, `save_checksum_file` - via a native "Save File" dialog, and `verify_checksum_file`, which - reads and parses a picked `.sfv`/`.md5`/`.sha1` file and resolves - each entry's bare filename relative to that file's own parent - directory — all wrapping `aurora-curator-core::sfv`, which does the - actual algorithm/format work), + same stored TV-source setting and credential), `sfv.rs` (four + commands backing the SFV panel and the Rename panel's Hash Files + button — `compute_checksums` over a batch of loaded files with + per-file error isolation, `save_checksum_file` via a native "Save + File" dialog, `verify_checksum_file`, which reads and parses a + picked `.sfv`/`.md5`/`.sha1` file and resolves each entry's bare + filename relative to that file's own parent directory, and + `compute_naming_hashes` — all four wrapping + `aurora-curator-core::sfv`, which does the actual algorithm/format + work; `compute_naming_hashes` calls the crate's + `compute_all_checksums` rather than `compute_checksum` three times, + streaming each file once instead of three), `history.rs` (rename history/undo log), `presets.rs` (up to 9 saved naming schemes) — all three stored as JSON in the OS app-config directory — `list.rs` (a @@ -601,6 +607,24 @@ its own **Resolution**/**Video codec** chip rows (populated from the same merged filename-tag-or-probed value) so a batch can be narrowed by either, not just by extension/kind/name/season. +## Hash Files + +A naming-scheme binding source, same idea as real media-file probing above, but for +`{crc32}`/`{md5}`/`{sha1}` — reuses the SFV panel's own hasher +(`aurora_curator_core::sfv`) rather than a separate implementation. Unlike Match's other +local steps (filename parsing, container-header probing), this is deliberately **not** +wired into Match automatically: a checksum needs the file's *entire* contents streamed +through, not just a small header or filename, so hashing a batch of large video files +can take real time — running it unasked on every Match click would make Match +unpredictably slow for schemes that don't even use a hash field. Instead, the Rename +panel's New Names list has its own **Hash Files** button next to Fetch Data; clicking it +computes all three checksums for every loaded file in one pass per file (not three +separate passes) and fills in `{crc32}`/`{md5}`/`{sha1}` for whichever files succeeded, +reporting how many hashed vs. failed (a file that's moved or deleted between loading and +hashing, a permissions problem, etc. — kept per-file, so one bad file doesn't block the +rest of the batch). The naming-scheme editor's field-reference list has a "File hashing" +group with all three as clickable buttons. + ## Porting roadmap (not yet done) Roughly in priority order — each references the original FileBot source file it's @@ -619,10 +643,13 @@ ported from, for anyone picking this up: for the two container families it supports (MP4/M4V/MOV and MKV/WebM). Probed bindings are now wired into all three panels that do local matching — Rename's Match step, and the Filter and List panels' own scans (Filter also gained - Resolution/Video codec filter chips from it). Still missing: probing for other - containers (.avi, .wmv, ...), audio-only metadata (bitrate, sample rate, channel - layout), and file hashing (`{crc32}` reuses the SFV panel's own hasher today but - isn't wired into naming-scheme bindings yet). + Resolution/Video codec filter chips from it). File hashing is done too — + `{crc32}`/`{md5}`/`{sha1}` reuse the SFV panel's own hasher (all three computed in a + single pass over the file, rather than reading it three times), wired to the Rename + panel's own "Hash Files" button rather than Match, since a full-file read is much + slower than everything else Match does and shouldn't run unasked for every batch + (see "Hash Files" below). Still missing: probing for other containers (.avi, .wmv, + ...) and audio-only metadata (bitrate, sample rate, channel layout). 2. **String/expression helper methods** — `net.filebot.format.ExpressionFormatMethods` (636 lines): FileBot-specific Groovy string extensions like `.pad()`, `.sortName()`. Ported so far (`crates/aurora-curator-core/src/format/functions.rs`): `any`, `all_of`, diff --git a/crates/aurora-curator-core/src/sfv/mod.rs b/crates/aurora-curator-core/src/sfv/mod.rs index 1dbc53a..ce72efc 100644 --- a/crates/aurora-curator-core/src/sfv/mod.rs +++ b/crates/aurora-curator-core/src/sfv/mod.rs @@ -98,6 +98,48 @@ pub fn compute_checksum(path: &Path, kind: ChecksumKind) -> io::Result { } } +/// All three checksums for one file, computed together - see [`compute_all_checksums`]. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct AllChecksums { + pub crc32: String, + pub md5: String, + pub sha1: String, +} + +/// Same job as calling [`compute_checksum`] three times (once per [`ChecksumKind`]), +/// but streams the file exactly once instead of three times - used for the naming-scheme +/// `{crc32}`/`{md5}`/`{sha1}` fields (see `src-tauri/src/sfv.rs`'s `compute_naming_hashes`), +/// where a scheme referencing more than one hash field would otherwise hash the same +/// (often multi-gigabyte video) file redundantly. The SFV panel itself still calls +/// [`compute_checksum`] directly - it only ever needs one algorithm at a time, picked by +/// the user, so tripling its I/O for the other two nobody asked for would be wasteful in +/// the other direction. +pub fn compute_all_checksums(path: &Path) -> io::Result { + let mut file = File::open(path)?; + let mut buf = [0u8; READ_BUFFER_SIZE]; + + let mut crc32 = crc32fast::Hasher::new(); + let mut md5 = Md5::new(); + let mut sha1 = Sha1::new(); + + loop { + let n = file.read(&mut buf)?; + if n == 0 { + break; + } + let chunk = &buf[..n]; + crc32.update(chunk); + md5.update(chunk); + sha1.update(chunk); + } + + Ok(AllChecksums { + crc32: format!("{:08X}", crc32.finalize()), + md5: format!("{:x}", md5.finalize()), + sha1: format!("{:x}", sha1.finalize()), + }) +} + /// One entry parsed out of an existing checksum file: the file name it references /// (not yet resolved to a real path - checksum files reference files by name only, /// always relative to wherever the checksum file itself lives) and the checksum text @@ -293,6 +335,44 @@ mod tests { assert!(compute_checksum(&missing, ChecksumKind::Crc32).is_err()); } + #[test] + fn compute_all_checksums_matches_the_known_test_vectors_all_at_once() { + // Same "123456789" CRC32 vector as above, computed alongside MD5/SHA-1 of the + // same bytes in one pass - pins that streaming all three hashers together over + // the same chunks produces the identical result as hashing separately. + let dir = temp_dir(); + let path = dir.join("check.txt"); + fs::write(&path, b"123456789").unwrap(); + let all = compute_all_checksums(&path).unwrap(); + assert_eq!(all.crc32, "CBF43926"); + assert_eq!(all.md5, "25f9e794323b453885f5181f1b624d0b"); // md5("123456789") + assert_eq!(all.sha1, "f7c3bc1d808e04732adf679965ccc34ca7ae3441"); // sha1("123456789") + } + + #[test] + fn compute_all_checksums_agrees_with_compute_checksum_on_a_multi_chunk_file() { + // Bigger than READ_BUFFER_SIZE (64 KiB) so both the individual and combined + // paths each have to loop over more than one read() - catches a bug where one + // of the three hashers only got the first chunk (e.g. an accidental `break` + // inside one match arm instead of the shared loop). + let dir = temp_dir(); + let path = dir.join("big.bin"); + let bytes: Vec = (0..200_000u32).map(|i| (i % 251) as u8).collect(); + fs::write(&path, &bytes).unwrap(); + + let all = compute_all_checksums(&path).unwrap(); + assert_eq!(all.crc32, compute_checksum(&path, ChecksumKind::Crc32).unwrap()); + assert_eq!(all.md5, compute_checksum(&path, ChecksumKind::Md5).unwrap()); + assert_eq!(all.sha1, compute_checksum(&path, ChecksumKind::Sha1).unwrap()); + } + + #[test] + fn compute_all_checksums_errors_when_the_file_does_not_exist() { + let dir = temp_dir(); + let missing = dir.join("nope.txt"); + assert!(compute_all_checksums(&missing).is_err()); + } + #[test] fn parses_a_typical_sfv_file_skipping_comments_and_blank_lines() { let content = "; Generated by Aurora Curator\n\nShow.S01E01.mkv CBF43926\nShow.S01E02.mkv 12345678\n"; diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs index c69b75c..319b996 100644 --- a/src-tauri/src/lib.rs +++ b/src-tauri/src/lib.rs @@ -127,6 +127,7 @@ pub fn run() { sfv::compute_checksums, sfv::save_checksum_file, sfv::verify_checksum_file, + sfv::compute_naming_hashes, list::export_text_list, probe::probe_media_files, history::list_rename_history, diff --git a/src-tauri/src/sfv.rs b/src-tauri/src/sfv.rs index 7efd882..183d3ed 100644 --- a/src-tauri/src/sfv.rs +++ b/src-tauri/src/sfv.rs @@ -53,6 +53,51 @@ pub async fn compute_checksums(paths: Vec, kind: ChecksumKind) -> Vec, + pub md5: Option, + pub sha1: Option, + pub error: Option, +} + +/// Computes CRC32 + MD5 + SHA-1 for every path in `paths` in one pass per file (see +/// `aurora_curator_core::sfv::compute_all_checksums`). Same blocking-thread treatment +/// as `compute_checksums` - real file I/O, potentially large video files - and +/// deliberately a separate, explicit action from "Match" rather than something that +/// runs automatically: hashing a whole file is much slower than Match's other local +/// steps (filename parsing, container-header probing), which only read a small header, +/// not the entire file. +#[tauri::command] +pub async fn compute_naming_hashes(paths: Vec) -> Vec { + tauri::async_runtime::spawn_blocking(move || { + paths + .into_iter() + .map(|path| match sfv::compute_all_checksums(Path::new(&path)) { + Ok(all) => NamingHashOutcome { + path, + crc32: Some(all.crc32), + md5: Some(all.md5), + sha1: Some(all.sha1), + error: None, + }, + Err(err) => NamingHashOutcome { path, crc32: None, md5: None, sha1: None, error: Some(err.to_string()) }, + }) + .collect() + }) + .await + .unwrap_or_default() +} + /// One row the frontend already has (a computed checksum for a loaded file) - the /// input shape `save_checksum_file` builds a sidecar file's text content from. #[derive(Debug, Clone, serde::Deserialize)] diff --git a/src/components/Icons.tsx b/src/components/Icons.tsx index a71b490..9861d90 100644 --- a/src/components/Icons.tsx +++ b/src/components/Icons.tsx @@ -153,3 +153,11 @@ export function KeyIcon() { ); } + +export function HashIcon() { + return ( + + + + ); +} diff --git a/src/components/RenamePanel.tsx b/src/components/RenamePanel.tsx index 77df744..fe9bef2 100644 --- a/src/components/RenamePanel.tsx +++ b/src/components/RenamePanel.tsx @@ -5,6 +5,7 @@ import { DownIcon, FetchIcon, GearIcon, + HashIcon, HistoryIcon, KeyIcon, LoadIcon, @@ -91,6 +92,19 @@ interface ProbeOutcome { probe: MediaProbe; } +// Mirrors aurora-curator-core's `NamingHashOutcome`. `crc32`/`md5`/`sha1` are `null` +// together with a non-null `error` when hashing that particular file failed (moved/ +// deleted between loading and hashing, a permissions problem, etc) - same per-file +// failure shape as `MediaProbe`/`ProbeOutcome` above, so one bad file doesn't block the +// rest of the batch. +interface NamingHashOutcome { + path: string; + crc32: string | null; + md5: string | null; + sha1: string | null; + error: string | null; +} + function pad2(n: number): string { return n.toString().padStart(2, "0"); } @@ -498,6 +512,12 @@ export default function RenamePanel() { // Match button stays disabled for the whole combined operation rather than just the // first half of it. const [matching, setMatching] = useState(false); + // Busy state for "Hash Files" (see `runHashFiles`) - deliberately its own state, + // separate from `matching`/`fetching`, since hashing a whole file is much slower + // than either of those (a full read of potentially multi-gigabyte video files, + // rather than a small filename/header/network round trip) and isn't run + // automatically as part of Match - the user opts into it explicitly. + const [hashing, setHashing] = useState(false); // Rename history / undo. `null` batches = not loaded yet (also used to show a // loading state the first time the History panel is opened). const [historyOpen, setHistoryOpen] = useState(false); @@ -1368,6 +1388,67 @@ export default function RenamePanel() { } }; + // "Hash Files": computes real CRC32/MD5/SHA-1 checksums for every loaded file and + // fills in the {crc32}/{md5}/{sha1} naming-scheme fields - reuses the SFV panel's + // own hasher (`aurora_curator_core::sfv::compute_all_checksums`, via the + // `compute_naming_hashes` command) rather than a separate implementation. Unlike + // Match's other local steps, this isn't wired into Match automatically: hashing + // reads a file's *entire* contents (potentially many gigabytes for video), unlike + // filename parsing or container-header probing, which only touch a small header - + // running it on every Match click for every loaded file, whether or not the naming + // scheme even uses a hash field, would make Match unpredictably slow. So this is its + // own explicit action, independent of media identification (a hash needs nothing + // but a real file on disk - no season/episode guess, no online lookup). + const runHashFiles = async () => { + const targets = items.filter((i) => !i.isSample); + if (targets.length === 0) { + setStatus("Load some files first, then Hash Files will compute their CRC32/MD5/SHA-1 checksums."); + window.clearTimeout((runHashFiles as any)._t); + (runHashFiles as any)._t = window.setTimeout(() => setStatus(null), 3200); + return; + } + + setHashing(true); + try { + const results = await invoke("compute_naming_hashes", { + paths: targets.map((item) => item.id), + }); + const byPath = new Map(results.map((r) => [r.path, r] as const)); + // Count directly from `results`, not inside the `setItems` updater below - same + // rule as runMatching/runFetch/confirmRename above (React doesn't run a + // functional updater synchronously, so a counter mutated in there wouldn't be + // ready yet for the status message right after). + const hashed = results.filter((r) => r.error === null).length; + const failed = results.length - hashed; + + setItems((prev) => + prev.map((item) => { + const result = byPath.get(item.id); + if (!result || result.error !== null || !result.crc32 || !result.md5 || !result.sha1) return item; + return { + ...item, + bindings: { + ...item.bindings, + crc32: result.crc32, + md5: result.md5, + sha1: result.sha1, + }, + }; + }) + ); + + const parts = [`Hashed ${hashed} of ${targets.length} file${targets.length === 1 ? "" : "s"}`]; + if (failed > 0) parts.push(`${failed} failed`); + setStatus(parts.join(", ") + "."); + } catch (err) { + setStatus(`Hashing failed: ${err}`); + } finally { + setHashing(false); + window.clearTimeout((runHashFiles as any)._t); + (runHashFiles as any)._t = window.setTimeout(() => setStatus(null), 5000); + } + }; + const removeSelected = () => { if (!selected) return notImplemented("Excluding items"); setItems((prev) => { @@ -1571,7 +1652,7 @@ export default function RenamePanel() {