Remove log warning on archive hash collision

TL;DR: The warning added a lot more noise than expected, doesn't really
add value, and improving it isn't worth the effort.

The warning was logged when the same pair of folder hash and file hash
appeared more than once within the set of archive files being loaded
(as the function was used, the set would be the archives loaded by a
single plugin).

Hashes are used directly instead of asset file paths because the paths
are not necessarily present in BSA files, and while they are present in
BA2 files (at least, I'm not aware of any option to omit them or any
files that do omit them), hashes are calculated from the file paths they
contain for consistency with the approach needed for BSA files.

Without the file paths it's not possible to determine if the repeated
hashes represent the same asset file (possibly containing different
data) or different files that have colliding hashes.

I had assumed that it would be unusual for a single plugin to load more
than one archive file containing the same asset file path, so any
repeated hash pairs would likely indicate hash collisions, but feedback
from Pickysaurus on behalf of Vortex users indicates that's not true,
and that logging all the warnings significantly slows down fully loading
plugins.

There are a few ways that the logged warning could be improved: tracking
which archive file existing hashes were inserted from would make it
easier to identify the pair of archive files that might need
investigating; reading BA2 files could defer transforming their asset file
paths into hashes until after all the files for a plugin have been read
(or even past that, to account for hash collisions between different
plugins' assets); and reading BSA files could opportunistically store the
asset file paths if they are present, and fall back to comparing using
hashes if not.

However, even if the warning was logged for only true positive hash
collisions, this is all in service of a sorting heuristic that is only
used when adding overlap edges and a pair of plugins do not have
overlapping records but do both load assets, and a collision would mean
that a plugin might seem to load fewer assets than it does, and could
also appear to overlap with a plugin that doesn't actually load assets
with the same file paths. That in turn might result in the two plugins
loading in one order instead of the other, causing one's assets to
override the other's. If that's a problem, then it can be fixed using
load after metadata, and you can only really tell if it's a problem by
spotting something wrong in game, so the warning doesn't really add much
value, and although it indicates that there might be a problem, there's
enough conditions between the warning and there actually being an issue
that logging it as a warning is excessive anyway.
This commit is contained in:
Oliver Hamlet
2026-03-13 18:40:49 +00:00
parent 4fe4daad83
commit 8cb826bf65
2 changed files with 22 additions and 30 deletions
+11
View File
@@ -2,6 +2,17 @@
Version History
***************
0.29.1 - Unreleased
===================
Removed
-------
- A warning is no longer logged when two archive files loaded by the same
plugin both appear to contain the same asset file path. The removed log
message had the format ``The folder and file with hashes <folder hash> and
<file hash> in "<archive path>" are present in another Bethesda archive.``.
0.29.0 - 2026-02-04
===================
+11 -30
View File
@@ -9,7 +9,6 @@ use super::error::{ArchiveParsingError, ArchivePathParsingError};
use crate::{
escape_ascii,
logging::{self, format_details},
plugin::has_ascii_extension,
};
use super::{ba2, bsa};
@@ -35,41 +34,23 @@ pub(crate) fn assets_in_archives(archive_paths: &[PathBuf]) -> BTreeMap<u64, BTr
}
};
let warn_on_hash_collisions = should_warn_on_hash_collisions(archive_path);
for (folder_hash, file_hashes) in assets {
let entry_file_hashes = archive_assets.entry(folder_hash).or_default();
for file_hash in file_hashes {
if !entry_file_hashes.insert(file_hash) && warn_on_hash_collisions {
logging::warn!(
"The folder and file with hashes {:x} and {:x} in \"{}\" are present in another Bethesda archive.",
folder_hash,
file_hash,
escape_ascii(archive_path)
);
}
}
// If two archives contain the same combination of folder hash and file
// hash, they will be deduplicated. Since each asset is only identified
// by a hash pair, collisions are possible, but since BSAs don't
// necessarily contain asset file paths, it's not necessarily possible
// to tell if a collision is for two different file paths or not, and
// logging all collisions is too noisy.
for (folder_hash, mut file_hashes) in assets {
archive_assets
.entry(folder_hash)
.or_default()
.append(&mut file_hashes);
}
}
archive_assets
}
fn should_warn_on_hash_collisions(archive_path: &Path) -> bool {
if !has_ascii_extension(archive_path, "ba2") {
return true;
}
let filename = archive_path
.file_name()
.unwrap_or_default()
.to_string_lossy()
.to_ascii_lowercase();
filename.starts_with("fallout4 - ") || filename.starts_with("dlcultrahighresolution - ")
}
fn get_assets_in_archive(
archive_path: &Path,
) -> Result<BTreeMap<u64, BTreeSet<u64>>, ArchivePathParsingError> {