mirror of
https://github.com/loot/libloot.git
synced 2026-07-27 14:16:01 -07:00
Add tie-break plugin graph edges more efficiently
Instead of brute-forcing the existence of a Hamiltonian path, attempt to create one by adding edges between each pair of adjacent vertices in the old load order wherever possible without introducing a cycle, and otherwise move plugins as necessary. The bidirectional BFS has been refactored to use a visitor so that the implementation can be shared between checking if a path exists and finding a path without always incurring the cost of the latter. This may result in a different sorted load order than the previous method, but that shouldn't make a practical difference as tie-break edges are only added between unrelated plugins. For a load order containing 1619 plugins, this reduces the time taken to sort them from around 400s to around 51s.
This commit is contained in:
Oliver Hamlet
+76
-17
@@ -24,10 +24,24 @@ the subrecords of the ``TES4`` header record.
|
||||
Create plugin graph vertices
|
||||
=================================
|
||||
|
||||
Once loaded, a directed graph is created and the plugins are added to it in
|
||||
lexicographical order as vertices. Any metadata a plugin has in the masterlist
|
||||
and userlist are then merged into its vertex's data store. Plugin group
|
||||
dependencies are also resolved and added as group-derived plugins.
|
||||
Once the plugins have been loaded, they are sorted into their current load
|
||||
order:
|
||||
|
||||
* If both plugins have positions in the current load order, the function
|
||||
preserves their existing relative order.
|
||||
* If one plugin has a position and the other does not, the plugin with a
|
||||
position goes before the plugin without a position.
|
||||
* If neither plugin has a load order position, a case-insensitive
|
||||
lexicographical comparison of their filenames without file extensions is used
|
||||
to decide their order. If they are equal, a case-insensitive lexicographical
|
||||
comparison of their file extensions is used.
|
||||
|
||||
After that, a directed graph is created and the plugins are added to it as
|
||||
vertices in their sorted order.
|
||||
|
||||
Any metadata a plugin has in the masterlist and userlist are then merged into
|
||||
its vertex's data store. Plugin group dependencies are also resolved and added
|
||||
as group-derived plugins.
|
||||
|
||||
Create plugin graph edges
|
||||
==============================
|
||||
@@ -47,6 +61,9 @@ For each plugin:
|
||||
4. Add edges coming from all the plugin's load after files that are installed
|
||||
plugins.
|
||||
|
||||
Group edges
|
||||
-----------
|
||||
|
||||
Group-derived interdependencies are then evaluated. Each plugin's group-derived
|
||||
plugins are iterated over and individually checked to see if adding an edge from
|
||||
the group-derived plugin to the plugin would cause a cycle, and if not the edge
|
||||
@@ -57,6 +74,9 @@ At this point the plugin graph is checked for cycles, and an error is thrown if
|
||||
any are encountered, so that metadata (or indeed plugin data) that cause them
|
||||
can be corrected.
|
||||
|
||||
Overlap edges
|
||||
-------------
|
||||
|
||||
Plugin overlap edges are then added. Two plugins overlap if they contain the
|
||||
same record, i.e. if they both edit the same record or if one edits a record the
|
||||
other plugin adds. Plugins also overlap if they both load one or more BSAs (BA2s
|
||||
@@ -79,21 +99,60 @@ plugin's masters to be installed, so if a plugin has missing masters, its total
|
||||
record count is used in place of its override record count. Morrowind plugins
|
||||
also can't load BSAs, so they can't have overlapping assets.
|
||||
|
||||
Finally, tie-break edges are added to ensure that sorting is consistent. For
|
||||
each plugin, iterate over all other plugins and add an edge between each pair of
|
||||
plugins in the direction given by the tie-break comparison function, unless that
|
||||
edge would cause a cycle.
|
||||
Tie-break edges
|
||||
---------------
|
||||
|
||||
The tie-break comparison function compares current plugin load order positions,
|
||||
falling back to plugin names.
|
||||
Finally, tie-break edges are added to ensure that sorting is consistent. The
|
||||
graph's vertices are iterated over in their insertion order (i.e. the current
|
||||
load order). Each loop looks at the current vertex and the next one following it
|
||||
(e.g. the first iteration is for vertices 0 and 1, the second is for 1 and 2,
|
||||
etc.).
|
||||
|
||||
* If both plugins have positions in the current load order, the function
|
||||
preserves their existing relative order.
|
||||
* If one plugin has a position and the other does not, the edge added goes from
|
||||
the plugin with a position to the plugin without a position.
|
||||
* If neither plugin has a load order position, a case-insensitive
|
||||
lexicographical comparison of their filenames without file extensions is used
|
||||
to decide their order.
|
||||
For each (``current``, ``next``) pair of vertices, try to find a path from
|
||||
``next`` to ``current``.
|
||||
|
||||
If sorting makes no changes, then there won't be any paths found and it'll
|
||||
therefore be possible to add an edge from ``current`` to ``next`` without
|
||||
causing a cycle, producing the old load order.
|
||||
|
||||
If no path is found then that means the old load order can be used for those two
|
||||
plugins. If the ``current`` vertex has not already been processed (which will be
|
||||
the case unless it appeared in a path found earlier and had its position pinned,
|
||||
see below), append it to a list representing the new load order and record the
|
||||
vertex as having been processed.
|
||||
|
||||
If no path is found but the ``current`` vertex has been processed and is not the
|
||||
last vertex in the new load order list, pin the position of the ``next`` vertex
|
||||
(see below).
|
||||
|
||||
If a path is found then that means the old load order for those two plugins
|
||||
(which is ``current`` before ``next``) can't be used. If ``current`` is the
|
||||
first vertex in the iteration order, then ``next`` is simply treated as the
|
||||
start of the new load order. If ``current`` is not the first vertex,
|
||||
iterate over the vertices in the path found, going from ``next`` to ``current``,
|
||||
and pin each vertex's position.
|
||||
|
||||
Pinning vertex positions
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
A vertex's position needs to be pinned when it must go somewhere before the last
|
||||
plugin in the new load order list, because although it has a fixed position
|
||||
relative to that last plugin, it doesn't necessarily have a fixed position
|
||||
relative to the plugins that come before the last plugin. I.e. it needs to load
|
||||
earlier, but how much earlier?
|
||||
|
||||
To pin a vertex's position, iterate over the new load order list in reverse
|
||||
order, going from the last vertex towards the first, and stop at the first
|
||||
load order vertex for which there is no path going from the unpinned vertex to
|
||||
the load order vertex. This is equivalent to finding the last plugin that the
|
||||
unpinned vertex's plugin can load after (which is not necessarily the same as
|
||||
the last plugin it *must* load after).
|
||||
|
||||
If such a load order vertex is found, add an edge going from it to the unpinned
|
||||
vertex. If the found vertex is not the last vertex in the load order list, also
|
||||
add an edge going from the unpinned vertex to the vertex after the found vertex.
|
||||
Then record the unpinned vertex's new position in the new load order list: the
|
||||
vertex is now pinned.
|
||||
|
||||
Topologically sort the plugin graph
|
||||
===================================
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -82,6 +82,9 @@ public:
|
||||
bool EdgeExists(const vertex_t& fromVertex, const vertex_t& toVertex);
|
||||
bool PathExists(const vertex_t& fromVertex, const vertex_t& toVertex);
|
||||
|
||||
std::optional<std::vector<vertex_t>> FindPath(const vertex_t& fromVertex,
|
||||
const vertex_t& toVertex);
|
||||
|
||||
void AddEdge(const vertex_t& fromVertex,
|
||||
const vertex_t& toVertex,
|
||||
EdgeType edgeType);
|
||||
|
||||
@@ -381,6 +381,129 @@ TEST_F(PluginGraphTest,
|
||||
EXPECT_TRUE(graph.EdgeExists(v1, v2));
|
||||
EXPECT_FALSE(graph.EdgeExists(v2, v1));
|
||||
}
|
||||
|
||||
TEST_F(PluginGraphTest, addTieBreakEdgesShouldNotErrorOnAGraphWithOneVertex) {
|
||||
const auto plugin = CreatePluginSortingData("A.esp");
|
||||
|
||||
PluginGraph graph;
|
||||
graph.AddVertex(plugin);
|
||||
|
||||
graph.AddTieBreakEdges();
|
||||
}
|
||||
|
||||
TEST_F(
|
||||
PluginGraphTest,
|
||||
addTieBreakEdgesShouldResultInASortOrderEqualToVertexCreationOrderIfThereAreNoOtherEdges) {
|
||||
PluginGraph graph;
|
||||
|
||||
for (size_t i = 0; i < 10; ++i) {
|
||||
const auto plugin = CreatePluginSortingData(std::to_string(i) + ".esp");
|
||||
graph.AddVertex(plugin);
|
||||
}
|
||||
|
||||
graph.AddTieBreakEdges();
|
||||
const auto sorted = graph.TopologicalSort();
|
||||
const auto names = graph.ToPluginNames(sorted);
|
||||
|
||||
std::vector<std::string> expected({"0.esp",
|
||||
"1.esp",
|
||||
"2.esp",
|
||||
"3.esp",
|
||||
"4.esp",
|
||||
"5.esp",
|
||||
"6.esp",
|
||||
"7.esp",
|
||||
"8.esp",
|
||||
"9.esp"});
|
||||
|
||||
EXPECT_FALSE(graph.IsHamiltonianPath(sorted).has_value());
|
||||
EXPECT_EQ(expected, names);
|
||||
}
|
||||
|
||||
TEST_F(
|
||||
PluginGraphTest,
|
||||
addTieBreakEdgesShouldPinPathsThatPreventTheVertexCreationOrderBeingUsed) {
|
||||
PluginGraph graph;
|
||||
|
||||
for (size_t i = 0; i < 10; ++i) {
|
||||
const auto plugin = CreatePluginSortingData(std::to_string(i) + ".esp");
|
||||
graph.AddVertex(plugin);
|
||||
}
|
||||
|
||||
// Add a path 6 -> 7 -> 8 -> 5.
|
||||
vertex_t five = graph.GetVertexByName("5.esp").value();
|
||||
vertex_t six = graph.GetVertexByName("6.esp").value();
|
||||
vertex_t seven = graph.GetVertexByName("7.esp").value();
|
||||
vertex_t eight = graph.GetVertexByName("8.esp").value();
|
||||
|
||||
graph.AddEdge(six, seven, EdgeType::overlap);
|
||||
graph.AddEdge(seven, eight, EdgeType::overlap);
|
||||
graph.AddEdge(eight, five, EdgeType::overlap);
|
||||
|
||||
// Also add a path going from 6 to 3 and another from 8 to 4.
|
||||
vertex_t three = graph.GetVertexByName("3.esp").value();
|
||||
vertex_t four = graph.GetVertexByName("4.esp").value();
|
||||
|
||||
graph.AddEdge(six, three, EdgeType::overlap);
|
||||
graph.AddEdge(eight, four, EdgeType::overlap);
|
||||
|
||||
graph.AddTieBreakEdges();
|
||||
const auto sorted = graph.TopologicalSort();
|
||||
const auto names = graph.ToPluginNames(sorted);
|
||||
|
||||
std::vector<std::string> expected({"0.esp",
|
||||
"1.esp",
|
||||
"2.esp",
|
||||
"6.esp",
|
||||
"3.esp",
|
||||
"7.esp",
|
||||
"8.esp",
|
||||
"4.esp",
|
||||
"5.esp",
|
||||
"9.esp"});
|
||||
|
||||
EXPECT_FALSE(graph.IsHamiltonianPath(sorted).has_value());
|
||||
EXPECT_EQ(expected, names);
|
||||
}
|
||||
|
||||
TEST_F(
|
||||
PluginGraphTest,
|
||||
addTieBreakEdgesShouldPrefixPathToNewLoadOrderIfTheFirstPairOfVerticesCannotBeUsedInCreationOrder) {
|
||||
PluginGraph graph;
|
||||
|
||||
for (size_t i = 0; i < 10; ++i) {
|
||||
const auto plugin = CreatePluginSortingData(std::to_string(i) + ".esp");
|
||||
graph.AddVertex(plugin);
|
||||
}
|
||||
|
||||
// Add a path 1 -> 2 -> 3 -> 0.
|
||||
vertex_t zero = graph.GetVertexByName("0.esp").value();
|
||||
vertex_t one = graph.GetVertexByName("1.esp").value();
|
||||
vertex_t two = graph.GetVertexByName("2.esp").value();
|
||||
vertex_t three = graph.GetVertexByName("3.esp").value();
|
||||
|
||||
graph.AddEdge(one, two, EdgeType::overlap);
|
||||
graph.AddEdge(two, three, EdgeType::overlap);
|
||||
graph.AddEdge(three, zero, EdgeType::overlap);
|
||||
|
||||
graph.AddTieBreakEdges();
|
||||
const auto sorted = graph.TopologicalSort();
|
||||
const auto names = graph.ToPluginNames(sorted);
|
||||
|
||||
std::vector<std::string> expected({"1.esp",
|
||||
"2.esp",
|
||||
"3.esp",
|
||||
"0.esp",
|
||||
"4.esp",
|
||||
"5.esp",
|
||||
"6.esp",
|
||||
"7.esp",
|
||||
"8.esp",
|
||||
"9.esp"});
|
||||
|
||||
EXPECT_FALSE(graph.IsHamiltonianPath(sorted).has_value());
|
||||
EXPECT_EQ(expected, names);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -336,14 +336,26 @@ TEST_P(
|
||||
SortPlugins(game_, game_.GetLoadOrder());
|
||||
FAIL();
|
||||
} catch (CyclicInteractionError &e) {
|
||||
ASSERT_EQ(3, e.GetCycle().size());
|
||||
EXPECT_EQ("Blank - Different Master Dependent.esm",
|
||||
e.GetCycle()[0].GetName());
|
||||
EXPECT_EQ(EdgeType::group, e.GetCycle()[0].GetTypeOfEdgeToNextVertex());
|
||||
EXPECT_EQ("Blank.esm", e.GetCycle()[1].GetName());
|
||||
EXPECT_EQ(EdgeType::master, e.GetCycle()[1].GetTypeOfEdgeToNextVertex());
|
||||
EXPECT_EQ("Blank - Master Dependent.esm", e.GetCycle()[2].GetName());
|
||||
EXPECT_EQ(EdgeType::group, e.GetCycle()[2].GetTypeOfEdgeToNextVertex());
|
||||
if (GetParam() == GameType::fo4) {
|
||||
ASSERT_EQ(4, e.GetCycle().size());
|
||||
EXPECT_EQ("Blank.esm", e.GetCycle()[0].GetName());
|
||||
EXPECT_EQ(EdgeType::master, e.GetCycle()[0].GetTypeOfEdgeToNextVertex());
|
||||
EXPECT_EQ("Blank - Master Dependent.esm", e.GetCycle()[1].GetName());
|
||||
EXPECT_EQ(EdgeType::group, e.GetCycle()[1].GetTypeOfEdgeToNextVertex());
|
||||
EXPECT_EQ("Blank - Different.esm", e.GetCycle()[2].GetName());
|
||||
EXPECT_EQ(EdgeType::master, e.GetCycle()[2].GetTypeOfEdgeToNextVertex());
|
||||
EXPECT_EQ("Blank - Different Master Dependent.esm",
|
||||
e.GetCycle()[3].GetName());
|
||||
EXPECT_EQ(EdgeType::group, e.GetCycle()[3].GetTypeOfEdgeToNextVertex());
|
||||
} else {
|
||||
ASSERT_EQ(3, e.GetCycle().size());
|
||||
EXPECT_EQ(masterFile, e.GetCycle()[0].GetName());
|
||||
EXPECT_EQ(EdgeType::group, e.GetCycle()[0].GetTypeOfEdgeToNextVertex());
|
||||
EXPECT_EQ("Blank.esm", e.GetCycle()[1].GetName());
|
||||
EXPECT_EQ(EdgeType::master, e.GetCycle()[1].GetTypeOfEdgeToNextVertex());
|
||||
EXPECT_EQ("Blank - Master Dependent.esm", e.GetCycle()[2].GetName());
|
||||
EXPECT_EQ(EdgeType::group, e.GetCycle()[2].GetTypeOfEdgeToNextVertex());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -442,10 +454,10 @@ TEST_P(PluginSortTest,
|
||||
blankMasterDependentEsm,
|
||||
blankDifferentMasterDependentEsm,
|
||||
blankDifferentEsp,
|
||||
blankMasterDependentEsp,
|
||||
blankDifferentMasterDependentEsp,
|
||||
blankDifferentPluginDependentEsp,
|
||||
blankEsp,
|
||||
blankMasterDependentEsp,
|
||||
blankDifferentMasterDependentEsp,
|
||||
blankPluginDependentEsp,
|
||||
});
|
||||
|
||||
@@ -474,10 +486,10 @@ TEST_P(PluginSortTest,
|
||||
blankMasterDependentEsm,
|
||||
blankDifferentMasterDependentEsm,
|
||||
blankDifferentEsp,
|
||||
blankMasterDependentEsp,
|
||||
blankDifferentMasterDependentEsp,
|
||||
blankDifferentPluginDependentEsp,
|
||||
blankEsp,
|
||||
blankMasterDependentEsp,
|
||||
blankDifferentMasterDependentEsp,
|
||||
blankPluginDependentEsp,
|
||||
});
|
||||
|
||||
|
||||
Reference in New Issue
Block a user