Commit Graph

726 Commits

Author SHA1 Message Date
matt peters
a1e506797a Generalize IPackageResourceManager::OnEndLoad into FCoreUObjectDelegates::OnEndLoadPackage for use by the BulkDataRegistry, and add UPackage::bHasBeenEndLoaded for BulkDataRegistry to know when a package has finished its initial load serialization.
Add OnEndLoad calls for Async loaded packages, since the editor sometimes uses async loads.
#rb Francis.Hurteau
#rnx

#ROBOMERGE-SOURCE: CL 16861817 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v836-16769935)

[CL 16861829 by matt peters in ue5-release-engine-test branch]
2021-07-15 09:49:49 -04:00
matt peters
0724105f6c BulkDataRegistry
Updating FVirtualizedUntypedBulkData and textures to use the BulkDataRegistry.
BulkDataRegistry: Add get/put accessors for the cached BulkDataList of packages.
EditorDomain: Move ClassDigests into a global variable that can be shared with BulkDataRegistry.
EditorDomain: Improve performance of GetFileSize by fetching metadata only.
Tickable Cook Objects, for systems used by the cooker that need to be ticked.
Implementation of the the BulkDataRegistry that uses the DDC cache for persistent storage of the BulkDataList.

#rb Devin.Doucette, Paul.Chipchase, Zousar.Shaker

#ROBOMERGE-SOURCE: CL 16768772 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)

[CL 16768778 by matt peters in ue5-release-engine-test branch]
2021-06-24 00:51:58 -04:00
aurel cordonnier
02c0f425e8 Copy up from Release-Engine-Staging @ 16738359
This represents UE4/Main @ 16738161 and Dev-PerfTest @ 16737719

[CL 16738582 by aurel cordonnier in ue5-release-engine-test branch]
2021-06-22 00:27:54 -04:00
per larsson
78582ba328 Fix non unity Crash reporter compile error
#rb none
#rnx

#ROBOMERGE-SOURCE: CL 16725383 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)

[CL 16725388 by per larsson in ue5-release-engine-test branch]
2021-06-21 03:40:46 -04:00
per larsson
38d3910b5c Remove dependency to CookOnTheFly module from CoreUObject and Launch when compiling programs
#rb Paul.Chipchase

#ROBOMERGE-SOURCE: CL 16725292 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)

[CL 16725294 by per larsson in ue5-release-engine-test branch]
2021-06-21 02:46:31 -04:00
zousar shaker
c27625b056 Integrating //UE5/Dev-Cooker [at] 16678003 to //UE5/Main (Zousar.Shaker-YEG-0943-Quaternary)
Non-DevIteration_ShooterGame


#ROBOMERGE-SOURCE: CL 16678907 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v835-16672529)

[CL 16678928 by zousar shaker in ue5-release-engine-test branch]
2021-06-15 16:38:03 -04:00
paul chipchase
d3e8c0243c Fix a bug where bulkdata that was compressed during the content cook was returning the compressed data rather than uncompressed data when the IoDispatcher was active.
- This has gone so long without being noticed as it is not expected that bulkdata will be compressed during a cook, as the pakfiles themselves will be compressed.
- InternalLoadFromIoStore now only loads via the IoDispatcher directly to the destination buffer if the data is in uncompressed format.
- If the data is going to be compressed we let the IoDispatcher allocate and return the compressed buffer and bulkdata will decompress that to the destination buffer instead.

#rb Per.Larsson, CM.Nordin
[FYI] Charles.deRousiers
#preflight 60c83e2143ec380001e2ad1b

#ROBOMERGE-SOURCE: CL 16670240 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v834-16658389)

[CL 16670246 by paul chipchase in ue5-release-engine-test branch]
2021-06-15 03:16:08 -04:00
francis hurteau
381867fdd0 Fixed crash in BulkData when detaching an already detached bulkdata
#rb Danny.Couture
#issue 313

#ROBOMERGE-SOURCE: CL 16641407 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v831-16623017)

[CL 16641417 by francis hurteau in ue5-release-engine-test branch]
2021-06-11 10:45:50 -04:00
carlmagnus nordin
0293546296 Added support for storing shaders in iostore.
Opt-In with core setting AllowShadersInIoStore
#review-16450322

#ROBOMERGE-OWNER: carlmagnus.nordin
#ROBOMERGE-AUTHOR: carlmagnus.nordin
#ROBOMERGE-SOURCE: CL 16622428 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v830-16605563)
#ROBOMERGE-CONFLICT from-shelf

[CL 16622474 by carlmagnus nordin in ue5-release-engine-test branch]
2021-06-10 09:36:47 -04:00
paul chipchase
0c2529d248 Reloading inline bulkdata in the old loader will now raise an ensure rather than log warnings as it should make it easier for people to spot the problem before it hits the new loader and fails to work entirely.
- Since reloading inline data does not work in the old loader we need to make it more obvious to developers when they are doing it in the old loader so that it doesn't get submitted.
- An ensure will cause the process top break in the debugger and is more likely to get attention. It will also provide the callstack for any QA bug report which will make it easier for us to triage to the correct team to fix their calling code than a vague warning about bulkdata.
- Also provided a new define, UE_KEEP_INLINE_RELOADING_CONSISTENT, which when enabled will cause reloading of inline bulkdata to fail (and error) in the old loader as well.
- The ensure can be supressed by setting [Core.System]IgnoreInlineBulkDataReloadEnsures=True in an engine config file, this might be useful for projects that do not care about running from staged data for what ever reason (but supressing the ensure will not prevent the data from failing to load when run with the new loader so this is strictly a 'use at your own risk' option)

#rb CarlMagnus.Nordin
#preflight 60bf45f97bc5f2000173195a

[CL 16586071 by paul chipchase in ue5-main branch]
2021-06-08 08:44:17 -04:00
robert millar
0cc7bdcb60 Don't use SetAndHold in csv profiler which will be backed out, register game thread ticks to push data to the csv profiler each frame.
#rb dan.phillips

#ROBOMERGE-OWNER: robert.millar
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 16554441 via CL 16554457 via CL 16554465 via CL 16554507 via CL 16554868
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Test -> Main) (v828-16531559)

[CL 16554877 by robert millar in ue5-main branch]
2021-06-03 18:48:51 -04:00
robert millar
51aa7f0d82 Expose IoDispatcher and AsyncLoading stats to csv profiler.
#ROBOMERGE-OWNER: robert.millar
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 16517115 via CL 16517137 via CL 16517179 via CL 16517335 via CL 16517350
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Test -> Main) (v826-16501804)

[CL 16517376 by robert millar in ue5-main branch]
2021-06-01 13:31:25 -04:00
robert millar
4af3923ac4 Expose IoDispatcher and AsyncLoading stats to csv profiler.
#ROBOMERGE-OWNER: robert.millar
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 16510694 via CL 16510697 via CL 16510701 via CL 16510734 via CL 16510747
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Test -> Main) (v826-16501804)

[CL 16510779 by robert millar in ue5-main branch]
2021-05-31 16:35:03 -04:00
aurel cordonnier
43fa62fcd8 Merge from Release-Engine-Test @ 16487383 to UE5/Main
This represents UE4/Main @ 16445039 and Dev-PerfTest @ 16444526

[CL 16488106 by aurel cordonnier in ue5-main branch]
2021-05-27 13:40:37 -04:00
Devin Doucette
fd7825f16c Removed unnecessary include of PathViews.h in a header that is included nearly everywhere
#rb Matt.Peters
#rnx
#preflight 60a7ef3f6905a600011d463b

[CL 16423171 by Devin Doucette in ue5-main branch]
2021-05-21 15:24:13 -04:00
CarlMagnus Nordin
602e857cfc AsyncLoading2:
Fixed pending io requests trace counter
Added total loaded trace counter
#rnx
#rb pj.kack

[CL 16402446 by CarlMagnus Nordin in ue5-main branch]
2021-05-20 10:13:18 -04:00
paul chipchase
bc5aaa90f5 Fix data conversion warnings in the bulkdata code
- FUntypedBulkData::SetBulkDataAlignment now takes a uint16 as that is the storage type used anyway.
- FBulkDataBuffer now uses TArrayView64 as the wrapper around it's data. This has been requested by another team anyway.
-- SoundWave.cpp will pass on the return type of FBulkDataBuffer to other systems. So I moved the 'is the data too big for TArrayView checks there and did not change the SoundWave api.

#rb Per.Larsson, Max.Hayes
#rnx
#preflight 60a606414c320300018014f4

[CL 16399515 by paul chipchase in ue5-main branch]
2021-05-20 03:33:32 -04:00
brandon dawson
7a6494a236 Allow for time-sliced unhashing of uobjects when in the editor. All unhashing will still occur before streaming is continued, but async loading will provide additional time to the unhashing, allowing it to finish faster, but over a few frames.
[REVIEW] [at]Robert.Manuszewski [at]Nick.Whiting [at]Graham.Wihlidal
#jira UETOP-1620

#ROBOMERGE-SOURCE: CL 16329986 in //UE5/Release-5.0-EarlyAccess/...
#ROBOMERGE-BOT: STARSHIP (Release-5.0-EarlyAccess -> Main) (v804-16311228)

[CL 16330069 by brandon dawson in ue5-main branch]
2021-05-14 11:15:55 -04:00
ionut matasaru
b9ed7c5b33 Fixed errors re format string / parameter list for various UE_LOGs:
- number of supplied parameters less or more than the number of specifiers used in format string
  - invalid format string specifiers used

#rb Matt.Peters

[CL 16315056 by ionut matasaru in ue5-main branch]
2021-05-13 13:21:24 -04:00
JeanFrancois Dube
ed2bc29a82 Added profile tag in ProcessAsyncLoading.
#rb none

[CL 16266901 by JeanFrancois Dube in ue5-main branch]
2021-05-11 07:51:23 -04:00
Matt Peters
988f40e081 FUnversionedStructSchema: fix Address Sanitzer's warning about calling delete on a pointer that was not allocated with new.
#rb Johan.Torp
#rnx

[CL 16206906 by Matt Peters in ue5-main branch]
2021-05-05 10:03:10 -04:00
Matt Peters
99b03f82ae GetSchemaHash: Fix buildbreak with programs that have WITH_EDITOR_DATA but not CACHE_UNVERSIONED_PROPERTY_SCHEMA.
#rb None, trivial
#fyi Johan.Torp
#rnx

[CL 16192085 by Matt Peters in ue5-main branch]
2021-05-04 08:00:09 -04:00
Matt Peters
6935c29d01 Create function UStruct::GetSchemaHash. This function provides a hash that can be used to check whether tagged property serialization stored in a saved package is a match for the current binary's UClasses. This hash can be used to test whether versioned tagged property serialization will produce unchanged results, and whether unversioned property serialization will succeed.
#rb Johan.Torp
#rnx

[CL 16186852 by Matt Peters in ue5-main branch]
2021-05-03 16:54:05 -04:00
aurel cordonnier
50944fd712 Merge UE5/RES @ 16162155 to UE5/Main
This represents UE4/Main @ 16130047 and Dev-PerfTest @ 16126156

[CL 16163576 by aurel cordonnier in ue5-main branch]
2021-04-29 19:32:06 -04:00
Jamie Dale
4afb7ce6bb Made EName an enum class to prevent implicit conversion to int
The following code would call the int overload rather than the name overload when passed an EName constant, which could easily lead to confusion and bugs:
    using FMyId = uint8;
    void DoThing(FName);
    void DoThing(FMyId);
    DoThing(NAME_Actor);

The change to use enum class requires that any code that actually deals with EName as an int (mostly internal name code and serialisation) be updated to explicitly cast, but prevents the implicit conversion that causes the issue above.

In order to preserve the NAME_X aliases that the old-style enum added to the global scope, new aliases have been added that point to the EName scoped versions. Unfortunately these can cause shadowing warnings if NAME_X is defined in the local scope, as the old-style enum used to allow that without shadowing, however there is no way to prevent this so we'll just need to fix any warnings that occur.

#rb Johan.Torp
#preflight 6087e06349a9840001414708

[CL 16126708 by Jamie Dale in ue5-main branch]
2021-04-27 10:11:18 -04:00