VerseCLR can now compile verse code with maps, and iterations over maps, to the new VM.
Fix unrelated bug triggered by nested for-iterations over arrays, returning an array of arrays.
#rb Saam.Barati
#rnx
[CL 27582449 by niklas rojemo in ue5-main branch]
#rb markus.breyer
This patch makes it so calls with multiple arguments don't require a tuple allocation. We just allow functions to have multiple arguments and be called normally. However, when such functions are invoked with a tuple, we'll unbox the tuple into the argument slots.
We also allow functions with single-param-as-tuple to be invoked with multiple arguments. And such functions will box the arguments into a tuple when being called.
A function with N arguments now take up the first N register slots in the frame.
This patch also fills out some more of the bytecode generator to make some tests I wrote work.
[CL 27510491 by saam barati in ue5-main branch]
[FYI] Joe.Kirchoff
Original CL Desc
-----------------------------------------------------------------
UnrealBuildTool: Always enable ValidateFormatStrings for engine modules, and create a pch variant for modules that opt out
#rnx
[CL 27331965 by joe kirchoff in ue5-main branch]
When we have hybrid projects, we add temporary target and build files in Intermediate/Source. We need to regenerate the RulesFileCache for that temporary folder.
If we don't, our 'initial' attempt to BuildCookRun a project will fail because we won't discover any target files because of an invalid cache entry.
#jira UE-190114 UE-193398
#rb Kriss.Gossart Jeff.Fisher
[CL 27246945 by robert srinivasiah in ue5-main branch]
-The logic in NativeProjects.cs IsHybridContentOnlyProject which decides whether we need to build an executable for a project was always using the buildhost platform (usually win64) to decide which plugins were active. This caused it to miss enabled code plugins that build for the actual target platform but not the host platform and potentially not do an executible build when one is required.
-Now we pass the TargetPlatforms for the current build into ConditionalMakeTempTargetForHybridProject so that we can test against them. If any project in this list needs code built we will make a temp target.
#jira UE-189611 UE-192833
#rb Josh.Adams
[CL 27101769 by jeff fisher in ue5-main branch]
Update PluginDescriptor to have parity with FPluginDescriptor in terms of handling custom JSON fields. The order in which the original fields are written to disk is preserved.
When saving the PluginDescriptor to disk, the same order is followed and any new entries are added to the end of the plugin file.
Note: PluginDescriptor.Save and PluginDescriptor.Save2 will exist concurrently for now. There are still parts of UBT that use PluginDescriptor.Write and the refactor to create a unified Save function will come later as a lot of testing still needs to be done.
JsonObject:
- Introduced AddOrSetFieldValue() methods overloaded on value types to add/set fields in JsonObject.
- Changed the underlying data structure of JsonObject from Dictionary<string,Object?> to OrderedDictionary to guarantee that order of all .uplugin fields (default and custom) are preserved
- Made the JsonObject constructor that takes an OrderedDictionary private to ensure type safety of values passed to JsonObject
- Introduced ToJsonString() for JsonObject that serializes the JsonObject to match the formatting of .uplugin
IMPORTANT: The ToJsonString() method uses the JavaScriptEncoder.UnsafeRelaxedJsonEscaping encoder for Utf8JsonWriter.
This allows +,<,>,&,` etc characters to be written properly for .uplugins. As per MS docs, with this encoding, none of the contents should be written to HTML or a script.
- Introduced JsonObjectTests that unit tested JsonObject to ensure previous behavior was maintained.
- Ensured that keys for the JsonObject are processed in a case insensitive and culture independent way.
PluginDescriptor
- Added a JsonObject field that acts as the cache of all fields that were read from a file or parsed from a json string.
- Cached JsonObject records the order of all fields that were read and captures custom fields in the .uplugin
- Introduced Save2(), the new save algorithm that writes all fields (default and custom) to disk.
- Deleted the private default constructor because it didn't seem to be used anywhere.
- Introduced PluginDescriptorTests to ensure previous behavior is maintained. In particular when dealing with a .uplugin with only default fields, results of Save() and Save2() are the same.
Descriptor Files:
- Introduced ToJsonObject() methods to replace the previous JsonWriter patterns of writing the descriptor data.
- Introduced the UpdateJsonObject pattern which mimics the old WriteArray pattern of writing data to a JsonWriter.
#rb: Mark.Winter, Joe.Kirchoff
#jira: none
#test: Developed this in a TDD style with unit tests for all the major functions. Also reviewed output of created .uplugins and diffed them.
[CL 26920712 by leon huang in ue5-main branch]
This addresses issues with CPU resources on Mac going under utilized on systems where total RAM is only 2-3x the number of cores (e.g 8c/16GB, 10c/32GB) and the user has applications running (UE, XCode, browsers) that are occupying RAM.
Historically this check limited actions based on free memory to ensure action processes weren't forced to use swap, but there's strong evidence that limiting based on total system memory and relying on the OS to swap out other processes to make room is fine. That said only Mac has been extensively tested here in this scenario (and tends to have faster disk performance than other platforms). So for now we'll change Mac and leave Windows/Linux using the old behavior
Tests -
* Ran on an M1 8c/16GB Mac with Xcode & UE open and compared core utilization before (5) to after this change (8)
* Loaded up a bucket-load of apps, browser tabs, and a running game and verified engine built successfully 2x with no compiler OOMs
#jira UE-192238
#rb swarm
#swarm [at]Joe.Kirchoff [at]Zack.Neyland
[CL 26906094 by andrew grant in ue5-main branch]
The fix is to restore the <= 5.1 behavior of UBT where the MaxParallelActions argument specifies the maximum number of actions to execute in parallel, not a cap on UBTs heuristic-based selection.
This change also modifies BenchmarkBuild so it will default to the available processor count if no -cores argument is specified. This ensures the behavior of repeated runs will be deterministic as UBT may select different values if the system free memory changes
#jira UE-192236
#rb swarm
[REVIEW] [at]Joe.Kirchoff
#tests Ran UBT / BenchmarkBuild with arguments that forces the number of actions higher than the locally available core count
[CL 26889927 by andrew grant in ue5-main branch]
* Added actions first produced item directory path as part of sorting for scheduling. This groups modules together nicely which will benefit when running things remotely where remotes takes batches of actions at the time meaning there is a higher chance each remote helper will need to download fewer variations of pch files
#rb joe.kirchoff
[CL 26856192 by henrik karlsson in ue5-main branch]
