Commit Graph

3441 Commits

Author SHA1 Message Date
paul chipchase
ec34aa82a8 Rework how the virtualization backends are defined.
#rb Per.Larsson
#rnx
#preflight 60f93e100fc49a0001c95bea

- When a package is saved we now push it's payloads to local storage backends (in memory/local file cache etc)
- In a future piece of work, when a package is submitted we will push it's payloads to the persistent storage backends (jupiter/source control etc)
- When pulling a payload we will first search the local storage backends for it and then go to the slower persistent storage backends if it was not found.
-- Once a payload has been found when being pulled we will immediately cache it to the local storage backends so that subsequent pulls will be faster.
-- At some point a ZenStorage backend will be added which will most likely be the primary local storage backend.

[CL 16922588 by paul chipchase in ue5-main branch]
2021-07-22 09:17:11 -04:00
Matt Peters
304411e847 IPackageStoreWriter: Update SavePackage to allow saving non-cooked packages into a PackageStoreWriter.
#rb Per.Larsson, Paul.Chipchase
#rnx

[CL 16922549 by Matt Peters in ue5-main branch]
2021-07-22 09:07:37 -04:00
Jamie Dale
2eb793f77f Improved consistency of how FScriptArrayHelper handles zero counts
The following functions would gracefully handle a count of zero:
 - EmptyAndAddValues
 - EmptyAndAddUninitializedValues

The following functions would assert if given a count of zero:
 - AddUninitializedValues
 - AddValues
 - InsertValues
 - RemoveValues
 - ClearValues

This change makes all of the above functions gracefully handle a count of zero, which addresses some editor crashes that could easily occur in user scripts when using unreal.Array in Python.

#fyi Malachi.Duncan
#rb Matt.Peters
#preflight 60f74556e885f20001b56003

[CL 16909007 by Jamie Dale in ue5-main branch]
2021-07-21 10:04:22 -04:00
paul chipchase
eb106bd9a4 Some small improvements to SVirtualizationStatusIndicator
#rb Per.Larsson
#rnx
#preflight 60f7b910a6959a000164fb78

* SVirtualizationStatusIndicator
- Added the total time taken for each activity to the widget's tooltip and reformatted the output to be clearer.
- The initial tool tip should just give a brief highlevel overview of the activity taking place so groups all backends together across all threads.

* VirtualizationManager
- Profiling data for Mirage is now stored as a struct per activity type, making it easier to edit in the future.
- We now store the profiling data in the base format and let the different display methods format it instead.

[CL 16906627 by paul chipchase in ue5-main branch]
2021-07-21 03:59:22 -04:00
Matt Peters
6a632d6c70 BulkDataRegistry:
Change texture and meshdescription BulkData guids to be unique.
#rb Paul.Chipchase
#rnx

[CL 16903425 by Matt Peters in ue5-main branch]
2021-07-20 18:18:32 -04:00
Tim Smith
bb9a97a14d 1) Removed the need to create a UScriptStruct to test to see if it has a NoOp constructor.
2) Added a quick method to query the deferred CppStructOps until we decide on a long term solution
3) Added method to test to see if a source file is the NoExports include file.

#rb none
#rnx
#preflight 60f6c42f25c166000152cbdc

[CL 16895867 by Tim Smith in ue5-main branch]
2021-07-20 09:30:11 -04:00
ben zeigler
d02926553a #jira UE-119853 Fix issue where Untracked metadata would not work on arrays of soft pointers
Fix similar issue with IncludeAssetBundlesName
Original fix from Roubert.Rouhani via UDN
#rb none

[CL 16887122 by ben zeigler in ue5-main branch]
2021-07-19 14:50:26 -04:00
Zousar Shaker
47761e865d Changes to ifdef some TObjectPtr and FObjectHandle code to ensure we're not absorbing overhead in non-late-resolve and non-access-tracking builds.
#rb devin.doucette
#preflight 60f5a15beab3510001d5b691

[CL 16885563 by Zousar Shaker in ue5-main branch]
2021-07-19 13:35:01 -04:00
Jamie Dale
26ecf0e7b9 Fixed single-property config updates failing to remove entries that were no longer needed as they matched the CDO
The UObject would write the correct config state, but UpdateSinglePropertyInSection would fail to remove the data for the updated key as it (correctly) wasn't present in the config being applied. This change has it remove the key from the config section if it exists.

#jira UE-119534
#rb Brooke.Hubert, Matt.Peters

[CL 16884096 by Jamie Dale in ue5-main branch]
2021-07-19 11:29:39 -04:00
yangke li
0a48229025 add RGB555A1 texture format support(not supported on Mac)
useful for RVT on mobile platforms, to save the cost of runtime compression
#rb Dmitriy.Dyomin mihnea.balta

[CL 16872220 by yangke li in ue5-main branch]
2021-07-16 11:38:24 -04:00
Zousar Shaker
6305185e3d First pass header changes in support of Engine+ShooterGame TObjectPtr refresh.
#rb devin.doucette

[CL 16865811 by Zousar Shaker in ue5-main branch]
2021-07-15 15:16:50 -04:00
Matt Peters
686e35b862 Generalize IPackageResourceManager::OnEndLoad into FCoreUObjectDelegates::OnEndLoadPackage for use by the BulkDataRegistry, and add UPackage::bHasBeenEndLoaded for BulkDataRegistry to know when a package has finished its initial load serialization.
Add OnEndLoad calls for Async loaded packages, since the editor sometimes uses async loads.
#rb Francis.Hurteau
#rnx

[CL 16861817 by Matt Peters in ue5-main branch]
2021-07-15 09:48:56 -04:00
Steve Robb
0559bde41d Fix for optimized map and set properties serialization when there are no defaults and the container is already populated.
#jira UE-118822
#rb none
#fyi johan.torp

[CL 16843259 by Steve Robb in ue5-main branch]
2021-07-13 17:45:15 -04:00
jordan hoffmann
d14371e681 Fix: The 'const' checkbox should not be visible in the Details tab for Blueprint Library functions.
#jira UE-39711
#rb mark.audy

[CL 16838997 by jordan hoffmann in ue5-main branch]
2021-07-13 13:03:03 -04:00
Michael Noland
12c4b10e37 Core: Made the verbosity of the unintialized reflected property check configurable via ini
- Issues with UObject* properties are upgraded from Warning to Error
- Other types remain the same for both engine and project modules (Display), but engine will soon change to Error as well

Settings can be overridden for either project or engine modules in DefaultEngine.ini (engine defaults are in BaseEngine.ini), e.g.,

[CoreUObject.UninitializedScriptStructMembersCheck]
EngineModuleReflectedUninitializedPropertyVerbosity=Error
ProjectModuleReflectedUninitializedPropertyVerbosity=Warning

#rb marc.audy

[CL 16826159 by Michael Noland in ue5-main branch]
2021-07-12 11:49:03 -04:00
ionut matasaru
338b2dc613 [Insights]
- Fixed the trace of scope cycle counters for UObject to be enabled everywhere where FScopeCycleCounterUObject is used switced by Stats::IsThreadCollectingData().
  - Fixed UObject::CallFunction to trace UObject name for Insights only when GVerboseScriptStats is on.

#rb Catalin.Dragoiu
#fyi Johan.Berg, Jeff.Farris

[CL 16824659 by ionut matasaru in ue5-main branch]
2021-07-12 09:22:58 -04:00
JeanFrancois Dube
320eada9b3 Backout CL16728888 as it doesn't properly handle the ECoreRedirectFlags::Option_MatchSubstring flag, found by running the CoreRedirects unit tests.
#jira UE-118831
#rb none

[CL 16819857 by JeanFrancois Dube in ue5-main branch]
2021-07-10 16:57:55 -04:00
Ben Woodhouse
2e0be09d2f Back out 16780416 for now since it's causing cook failures
#fyi Michael.Noland

[CL 16783107 by Ben Woodhouse in ue5-main branch]
2021-06-25 10:06:21 -04:00
Michael Noland
9366100a2f Core: Changed the verbosity of the unintialized reflected property check to an error for types in engine modules or UObject* properties
#codereview nick.darnell
#preflight 60d50948925f140001249b76

[CL 16780416 by Michael Noland in ue5-main branch]
2021-06-24 20:03:03 -04:00
francis hurteau
c3bd55cb3d Remove the usage of modal message dialog in package saving
Always try to find the likely culprit in case of illegal references

#rb Matt.Peters
#jira UE-115971


#ROBOMERGE-SOURCE: CL 16774967
#ROBOMERGE-BOT: (v836-16769935)

[CL 16774971 by francis hurteau in ue5-main branch]
2021-06-24 12:05:25 -04:00
Matt Peters
07a0af0f5b BulkDataRegistry
Updating FVirtualizedUntypedBulkData and textures to use the BulkDataRegistry.
BulkDataRegistry: Add get/put accessors for the cached BulkDataList of packages.
EditorDomain: Move ClassDigests into a global variable that can be shared with BulkDataRegistry.
EditorDomain: Improve performance of GetFileSize by fetching metadata only.
Tickable Cook Objects, for systems used by the cooker that need to be ticked.
Implementation of the the BulkDataRegistry that uses the DDC cache for persistent storage of the BulkDataList.

#rb Devin.Doucette, Paul.Chipchase, Zousar.Shaker

[CL 16768772 by Matt Peters in ue5-main branch]
2021-06-24 00:50:53 -04:00
aurel cordonnier
d17d20ca36 Merge from Release-Engine-Test @ 16758890 to UE5/Main
This represents UE4/Main @ 16738161 and Dev-PerfTest @ 16737719 (and Release-17.00 @ 16658211)

[CL 16763350 by aurel cordonnier in ue5-main branch]
2021-06-23 17:51:32 -04:00
PJ Kack
93ab2dcb43 GarbageCollection: Improve GC reachability analysis wall time by letting the GT do work using a ParallelFor instead of waiting.
#jira UE-116567
#rb danny.couture
#rnx

[CL 16755950 by PJ Kack in ue5-main branch]
2021-06-23 08:53:57 -04:00
Matt Peters
c70c4d00f1 VirtualizedBulkData: fix incorrect serialized PayloadIds for editordomain versions of non-virtual payloads that virtualize on save; the incorrect paylodIds were noticeable by the "Failed to pull virtual data with guid" error when loading the saved package.
#rb Paul.Chipchase
#rnx

[CL 16754551 by Matt Peters in ue5-main branch]
2021-06-23 04:07:31 -04:00
danny couture
0c5673e3f3 Optimize the ddc fill job
- Avoid redundant iterations and processing of UPackages by implementing a listener instead
  - Process many packages asynchronously (not just one) by keeping them in memory longer and finishing them later
  - Make the ddc fill job wait for async tasks properly
  - Fix texture always being single threaded by allowing async at all time for ddc fill jobs
  - Avoid calling IsCachedCookedPlatformDataLoaded too often on objects still being compiled
  - Use a more sensible max memory usage in WorldPartitionHelpers to avoid calling the GC too often

  - 30m16s to 7m44s when running -run=DerivedDataCache -fill -map=P_Construct_WP -mapsonly -projectonly with hot cache
  - 2h30m to 1h11s when running -run=DerivedDataCache -fill -map=P_Construct_WP -mapsonly -projectonly with cold cache

#rb Zousar.Shaker, Devin.Doucette
#preflight 60d1fd5571002f0001b8880e

[CL 16745009 by danny couture in ue5-main branch]
2021-06-22 12:50:20 -04:00