Commit Graph

825 Commits

Author SHA1 Message Date
Devin Doucette
01274b17c8 Logging: Added a dedicated master thread for logging
#preflight 622fb0eca709bd110fea0da5
#rb Zousar.Shaker

[CL 19387540 by Devin Doucette in ue5-main branch]
2022-03-15 12:31:55 -04:00
steve robb
0e1e8c13fb Removal of unused CopyFormatted*() and *_TextOnly() functions.
#rb graeme.thornton
#preflight 6229fae77341dd1ecf70f0a0

#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19335163 via CL 19339363 via CL 19345711 via CL 19350978 via CL 19351029
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)

[CL 19351730 by steve robb in ue5-main branch]
2022-03-11 04:01:36 -05:00
marc audy
51b9541255 Moving IsRoutingPostLoad to a tighter scope to prevent crashes when loading packages with non-native objects
#rb trivial
#preflight none


#ushell-cherrypick of 19339131 by robert.manuszewski

#ROBOMERGE-AUTHOR: robert.manuszewski
#ROBOMERGE-SOURCE: CL 19317027 via CL 19345332 via CL 19350794 via CL 19350872
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)

[CL 19351728 by marc audy in ue5-main branch]
2022-03-11 04:01:16 -05:00
pj kack
0988dfc4fb AsyncLoading2: Fix crash in seamless server travel.
When reloading an existing UPackage make sure to always set all serialized package flags again since they may have been cleared since the last load.
One example is UWorld::FinishDestroy that may clear PKG_ContainsMap on its world UPackage.

#jira UE-135502
#rb carlmagnus.nordin,per.larsson
#rnx
#preflight 62275410e83598518f2a720f

#ROBOMERGE-AUTHOR: pj.kack
#ROBOMERGE-SOURCE: CL 19302798 in //UE5/Release-5.0/... via CL 19303924
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)

[CL 19346856 by pj kack in ue5-main branch]
2022-03-10 20:49:12 -05:00
andrew rodham
355efbd7ef Fix for mismatched weakptr serialization between FJsonArchiveOutputFormatter and FJsonArchiveInputFormatter.
#rb Steve.Robb
#preflight 621da3fdca28c556340cc56d

#ROBOMERGE-AUTHOR: andrew.rodham
#ROBOMERGE-SOURCE: CL 19203318 via CL 19207261 via CL 19213316 via CL 19213622 via CL 19213693
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v921-19075845)

[CL 19225224 by andrew rodham in ue5-main branch]
2022-03-02 13:57:41 -05:00
danny couture
5b1b53c0b3 Avoid stalling for long periods of time during map load and PIE
- Remove slowtask throttling during load as it could prevent meaningful top level slowtask to be created
  - Fix delayed slowtask that was never showing when children were added before the delay
  - Add missing slowtask handling on long gamethread functions to improve UI responsiveness
  - Add PIE preparation slowtask scopes
  - Test to make sure the total amount of time spent in all slowtask functions is reasonable (1s on a 2 minute map load)

#rb Jason.Nadro
#preflight 621e1ccf3e14f0c7e53fe337

[CL 19197334 by danny couture in ue5-main branch]
2022-03-01 09:02:24 -05:00
robert millar
5811293e2a Add TargetRules.bFNameOutlineNumber as an option to reduce the size of FName by 4 bytes by storing the number in the name table.
Adds a define UE_FNAME_OUTLINE_NUMBER.
Removes of FName, FMinimalName from memory image support. Adds of FMemoryImageName.
Removal of FMinimalName operator<<, all fields made private, size made variable.
All fields of FScriptName made private.
Added console commands for dumping numbered/unnumbered names and stats.

#rb johan.torp

#ROBOMERGE-OWNER: robert.millar
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19058026 via CL 19058611 via CL 19058656 via CL 19061727 via CL 19061740 via CL 19064047
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)

[CL 19066701 by robert millar in ue5-main branch]
2022-02-21 02:35:48 -05:00
robert manuszewski
c36e03d5e3 do clause support
#rb Andrew.Scheidecker
#preflight 620f3a108863746a104d6c36

#ROBOMERGE-AUTHOR: robert.manuszewski
#ROBOMERGE-SOURCE: CL 19047989 via CL 19048186 via CL 19048197 via CL 19048203 via CL 19049414
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)

[CL 19065767 by robert manuszewski in ue5-main branch]
2022-02-21 00:33:13 -05:00
Matt Peters
7645a7dd38 OnEndLoadPackage: Change the arguments to a context object to avoid future need to change the function prototype. Add RecursiveDepth and bSynchronous arguments so that callers can take action only on the top-level EndLoadPackage if necessary.
BulkDataRegistry: Wait until the top-level EndLoadPackage, and wait until after OnEnginePostInit, before deleting bulkdatasize records. Deleting them sooner can make them be deleted before they're used by the EditorDomain if the EditorDomain's TrySavePackage triggers package loading and a recursive OnEndLoadPackage.
#rb Zousar.Shaker
#rnx
#preflight 620f00e7e8554a6f64f74d5a

[CL 19046732 by Matt Peters in ue5-main branch]
2022-02-17 22:02:59 -05:00
Matt Peters
6dcd01dd70 EditorBulkData: Add an Owner argument for UpdatePayload, and supply it from FTextureSource, so we can know ahead of the save which package an in-memory bulk data will be saved under.
#rb Paul.Chipchase
#rnx
#preflight 620efcb4c602491fb81538f8

[CL 19046506 by Matt Peters in ue5-main branch]
2022-02-17 21:12:20 -05:00
jeanfrancois dube
0d527b196f Added support for override package in package text serialization:
- Text format now includes import and export tables instead of discovering import while serializating exports. This is required to have enough information about objects with an override package that have their Outer in another package.
- UObject Json formatters now export import/export indices instead of relying on the objects full paths, which index into the exported import/export tables.
- Merged code to serialize UObject* and FObjectPtr in Json formatters to avoid code duplication.

#rb francis.hurteau, steve.robb
#preflight 61fd34030a43b689e1750d28
#rnx

#ROBOMERGE-AUTHOR: jeanfrancois.dube
#ROBOMERGE-SOURCE: CL 18978952 via CL 18980785 via CL 18981531 via CL 18981661 via CL 18992713 via CL 18993052
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589)

[CL 18997360 by jeanfrancois dube in ue5-main branch]
2022-02-15 11:35:28 -05:00
carlmagnus nordin
3e2f2637c9 AsyncLoading2: Fixed Async and AsyncLoading flags sometimes not being cleared
#rnx
#rb pj.kack

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18933303 via CL 18933306 via CL 18933307 via CL 18933723 via CL 18933770
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v916-18915374)

[CL 18933772 by carlmagnus nordin in ue5-main branch]
2022-02-10 05:06:30 -05:00
jamie dale
70f2cb952f Consider all properties within an editor-only class to be editor-only when gathering localization
#jira UE-141785
#preflight 61fc66f2dc0b3ecbecb76bb3
#rb Nick.Darnell
#lockdown jeanmichel.dignard

#ROBOMERGE-AUTHOR: jamie.dale
#ROBOMERGE-SOURCE: CL 18903352 in //UE5/Release-5.0/... via CL 18903564 via CL 18903986
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v912-18901109)

[CL 18904017 by jamie dale in ue5-main branch]
2022-02-08 12:12:30 -05:00
julien stjean
1f961a0b33 Fixed a crash caused by a implicit conversion of a FSharedBuffer to a FIoHash.
#jira UE-141087
#preflight 61fb26515e35b9215b060899

#ROBOMERGE-AUTHOR: julien.stjean
#ROBOMERGE-SOURCE: CL 18838290 in //UE5/Release-5.0/... via CL 18838292 via CL 18838386
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v910-18824042)

[CL 18838404 by julien stjean in ue5-main branch]
2022-02-02 20:40:58 -05:00
paul chipchase
05d81ee7f3 EditorBulkData and the virtualization system now use FIoHash directly and FPayloadId is removed
#rb Per.Larsson, Devin.Doucette
#jira UE-133497
#rnx
#preflight 61f835491c5ac5523462810a

- A lot of files have been touched but most of this is changing FPayload::IsValid to !FIoHash::IsZero, the main changes are in EditorBulkData/EditorBulkDataTests
- The only difference between FPayloadId and FIoHash is that the former considered the hash of an invalid or empty buffer to be 'invalid' too where as FIoHash would return a valid hash
-- To keep behaviour the same, we only hash payloads in EditorBulkData using a utility method ::HashBuffer which will not hash empty or invalid payloads and return a default FIoHash instead.
-- Unit tests have been extended to prove that the behaviour has not changed.

#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18806362 in //UE5/Release-5.0/... via CL 18808527 via CL 18821790
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)

[CL 18822154 by paul chipchase in ue5-main branch]
2022-02-02 02:21:24 -05:00
robert manuszewski
97b5e82c0b Deprecating EInternalObjectFlags::PendingKill. Making sure iterators use the appropriate flags based on the current state of PendingKill being enabled or not.
#preflight 61f8f33d537702981c352c7a
#rb Steve.Robb

#ROBOMERGE-AUTHOR: robert.manuszewski
#ROBOMERGE-SOURCE: CL 18806353 in //UE5/Release-5.0/... via CL 18808526 via CL 18821789
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)

[CL 18822151 by robert manuszewski in ue5-main branch]
2022-02-02 02:21:12 -05:00
paul chipchase
2af0d61698 Fix NSLOCTEXT being split over multiple lines
#rb trivial
#rnx
#preflight 61f43acf801201ab38924266

#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18776018 in //UE5/Release-5.0/... via CL 18777341 via CL 18777587
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18777599 by paul chipchase in ue5-main branch]
2022-01-28 15:26:27 -05:00
carlmagnus nordin
73a67c95d9 Renamed EPackageLocationFilter enums from Cooked/Uncooked to IoDispatcher/FileSystem
Cleaned up duplicated logic between DoesPackageExist and DoesPackageExistEx
#rnx
#preflight 61f3b9f2801201ab387e6e50
#rb per.larsson

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18770321 in //UE5/Release-5.0/... via CL 18770330 via CL 18770438
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18770443 by carlmagnus nordin in ue5-main branch]
2022-01-28 08:48:13 -05:00
carlmagnus nordin
a201cd8e12 Fixed up uses of FIoDispatcher::IsInitialized() so that they work correctly also when there's a mix of cooked packages in iostore and uncooked packages on disk
FIoDispatcher::IsInitialized() will now return true if it exists (which is almost always)
Renamed FArchive::IsUsingEventDrivenLoader() to IsLoadingFromCookedPackage and based it on the PKG_Cooked flag
#rnx
#rb pj.kack
#preflight 61f3ab6f74510448a6740570

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18769198 in //UE5/Release-5.0/... via CL 18769204 via CL 18769272
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18769277 by carlmagnus nordin in ue5-main branch]
2022-01-28 04:29:58 -05:00
paul chipchase
0d4d469bbd Add LexToString to FPayloadId and change existing use of ToString to use it.
#rb Per.Larsson
#rnx
#jira UE-133497
#preflight 61f259a3706ac5ea0cf2ee3e

#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18751491 in //UE5/Release-5.0/... via CL 18751500 via CL 18751554
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18751559 by paul chipchase in ue5-main branch]
2022-01-27 04:20:46 -05:00
francis hurteau
5714cca54f Reduced runtime UPackage size from 176b to 144b or 120b when stripping deprecated properties
This should saves around ~1.5 Mb to ~3 Mb at runtime with ~50000 packages

Added a new core define UE_STRIP_DEPRECATED_PROPERTIES that could be used to wrap deprecated properties and strip them to regain memory when a projects becomes compliant. this can be set in the project target file
Deprecated most public properties from UPackage and created accessors for them

#rb PJ.Kack
#jira UE-138957
#preflight 61f17a6f7266f4e79bd62601

#ROBOMERGE-AUTHOR: francis.hurteau
#ROBOMERGE-SOURCE: CL 18738937 in //UE5/Release-5.0/... via CL 18739524 via CL 18741373
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18742135 by francis hurteau in ue5-main branch]
2022-01-26 15:00:04 -05:00
carlmagnus nordin
b80a5b3461 AsyncLoading2: Disable check that all RF_Public objects have the LoadImport flag set, RF_Public can be added later to objects without export hashes that can never be imported
#rnx
#rb pk.kack
#jira UE-140062
#preflight none

#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 18720743 via CL 18721623 via CL 18721633 via CL 18721637 via CL 18724689 via CL 18725155
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18725167 by carlmagnus nordin in ue5-main branch]
2022-01-25 13:06:14 -05:00
paul chipchase
4faa718790 Saving a package containing virtualized payloads will no longer force the package to 'rehydrate' and download the payloads to store locally.
#rb PJ.Kack
#jira UE-136131
#rnx
#preflight 61eacb3d445cebac10c3d046

- A cvar named "Serialization.RehydrateOnSave" now controls if we rehydrate a package on save, defaulting to false.
-- At the very least we will need to be able to turn this to true to run any "rehydrate the project" UAT command in the future.
- The package trailer no longer stores a valid compressed size for a payload. In the best case scenario we only know the compressed size of the payload when it was still stored on locally on disk. Each virtualization backend is free to take the payload and recompress it anyway it sees fit etc.
- When duplicate payloads entries are added to a FPackageTrailerBuilder we now keep the original payload entry and discard the duplicate rather than replacing the existing entry with the duplicate, which was the original intent.

#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18721599 in //UE5/Release-5.0/... via CL 18721602 via CL 18721661
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18721674 by paul chipchase in ue5-main branch]
2022-01-25 08:05:16 -05:00
paul chipchase
de8990dd1b Fix an assert for projects with virtualization and editor domain both enabled
#rb PJ.Kack
#jira UE-140005
#rnx
[FYI] Mark.Lintott
#preflight 61ee5ec4e6107b23a020d0f5

- EditorBulkData payloads that are updated in PostLoad were being added to the package trailer builder for local access when saving the package for the editor domain.
- The package trailer system is not able to support this (although this will change soon) so when this occurs we should force those payloads to the legacy serialization path.
- Cleaned up ::ShouldUseLegacySerialization by factoring out the code that is actually affected by UE_ENABLE_VIRTUALIZATION_TOGGLE is wrapped in it, making things easier to read.

#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18706628 in //UE5/Release-5.0/... via CL 18706631 via CL 18706657
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18706667 by paul chipchase in ue5-main branch]
2022-01-24 05:03:42 -05:00
paul chipchase
77c12cf814 Virtualized bulkdata is now know as EditorBulkData
#rb PJ.Kack
#rnx
#jira UE-139708
#preflight 61eaaedfc92021e535b6d1e0

- The goal is to show a clear separation between the bulkdata object and the virtualization system.
- NOTE that all of these classes were added during 5.0 development so we are just renaming and moving them, no deprecation paths.
- Renamed FVirtualizedUntypedBulkData to FEditorBulkData
-- Also removed the derived templated types, they were never actually used
- Renamed FVirtualizedBulkDataReader to FEditorBulkDataReader
- Renamed FVirtualizedBulkDataWriter to FEditorBulkDataWriter
- Moved the renamed classes to the UE::Serialization namespace from UE::Virtualization
- Renamed the files of the renamed classes where required.
- Replaced use of LogVirtualization with LogSerialization.
- Renamed defines prefixed with VBD_* to UE_ and make sure they are undefed by the end of the cpp
- Edited unit tests to show up under "System.CoreUObject.Serialization.EditorBulkData.*"

#ROBOMERGE-AUTHOR: paul.chipchase
#ROBOMERGE-SOURCE: CL 18688778 in //UE5/Release-5.0/... via CL 18688787 via CL 18688810
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)

[CL 18688823 by paul chipchase in ue5-main branch]
2022-01-21 09:18:53 -05:00