Update PluginDescriptor to have parity with FPluginDescriptor in terms of handling custom JSON fields. The order in which the original fields are written to disk is preserved.
When saving the PluginDescriptor to disk, the same order is followed and any new entries are added to the end of the plugin file.
Note: PluginDescriptor.Save and PluginDescriptor.Save2 will exist concurrently for now. There are still parts of UBT that use PluginDescriptor.Write and the refactor to create a unified Save function will come later as a lot of testing still needs to be done.
JsonObject:
- Introduced AddOrSetFieldValue() methods overloaded on value types to add/set fields in JsonObject.
- Changed the underlying data structure of JsonObject from Dictionary<string,Object?> to OrderedDictionary to guarantee that order of all .uplugin fields (default and custom) are preserved
- Made the JsonObject constructor that takes an OrderedDictionary private to ensure type safety of values passed to JsonObject
- Introduced ToJsonString() for JsonObject that serializes the JsonObject to match the formatting of .uplugin
IMPORTANT: The ToJsonString() method uses the JavaScriptEncoder.UnsafeRelaxedJsonEscaping encoder for Utf8JsonWriter.
This allows +,<,>,&,` etc characters to be written properly for .uplugins. As per MS docs, with this encoding, none of the contents should be written to HTML or a script.
- Introduced JsonObjectTests that unit tested JsonObject to ensure previous behavior was maintained.
- Ensured that keys for the JsonObject are processed in a case insensitive and culture independent way.
PluginDescriptor
- Added a JsonObject field that acts as the cache of all fields that were read from a file or parsed from a json string.
- Cached JsonObject records the order of all fields that were read and captures custom fields in the .uplugin
- Introduced Save2(), the new save algorithm that writes all fields (default and custom) to disk.
- Deleted the private default constructor because it didn't seem to be used anywhere.
- Introduced PluginDescriptorTests to ensure previous behavior is maintained. In particular when dealing with a .uplugin with only default fields, results of Save() and Save2() are the same.
Descriptor Files:
- Introduced ToJsonObject() methods to replace the previous JsonWriter patterns of writing the descriptor data.
- Introduced the UpdateJsonObject pattern which mimics the old WriteArray pattern of writing data to a JsonWriter.
#rb: Mark.Winter, Joe.Kirchoff
#jira: none
#test: Developed this in a TDD style with unit tests for all the major functions. Also reviewed output of created .uplugins and diffed them.
[CL 26920712 by leon huang in ue5-main branch]
This addresses issues with CPU resources on Mac going under utilized on systems where total RAM is only 2-3x the number of cores (e.g 8c/16GB, 10c/32GB) and the user has applications running (UE, XCode, browsers) that are occupying RAM.
Historically this check limited actions based on free memory to ensure action processes weren't forced to use swap, but there's strong evidence that limiting based on total system memory and relying on the OS to swap out other processes to make room is fine. That said only Mac has been extensively tested here in this scenario (and tends to have faster disk performance than other platforms). So for now we'll change Mac and leave Windows/Linux using the old behavior
Tests -
* Ran on an M1 8c/16GB Mac with Xcode & UE open and compared core utilization before (5) to after this change (8)
* Loaded up a bucket-load of apps, browser tabs, and a running game and verified engine built successfully 2x with no compiler OOMs
#jira UE-192238
#rb swarm
#swarm [at]Joe.Kirchoff [at]Zack.Neyland
[CL 26906094 by andrew grant in ue5-main branch]
The fix is to restore the <= 5.1 behavior of UBT where the MaxParallelActions argument specifies the maximum number of actions to execute in parallel, not a cap on UBTs heuristic-based selection.
This change also modifies BenchmarkBuild so it will default to the available processor count if no -cores argument is specified. This ensures the behavior of repeated runs will be deterministic as UBT may select different values if the system free memory changes
#jira UE-192236
#rb swarm
[REVIEW] [at]Joe.Kirchoff
#tests Ran UBT / BenchmarkBuild with arguments that forces the number of actions higher than the locally available core count
[CL 26889927 by andrew grant in ue5-main branch]
When password is set for a service account, a Perforce login needs to be made. But in case of a ticket that's not necessary (and will not be accepted by login).
[CL 26883041 by carl bystrom in ue5-main branch]
* Added actions first produced item directory path as part of sorting for scheduling. This groups modules together nicely which will benefit when running things remotely where remotes takes batches of actions at the time meaning there is a higher chance each remote helper will need to download fewer variations of pch files
#rb joe.kirchoff
[CL 26856192 by henrik karlsson in ue5-main branch]
IUnrealRoleConfigurations are a type that let you modularize functionality for any role in a gauntlet test. It's primarily designed to generate command line arguments, but can do anything that's desired to the target Role.
IUnrealRoleConfigurations supply two interface functions
-ApplyConfigToRole - Core implementation of the configuration, this is the "doing"
-VerifyRoleConfig - Verification implementation. This gives you a chance to verify the "Apply" step had the desired result.
#rb Clayton.Langford
[CL 26855812 by brendan lienau in ue5-main branch]
