Commit Graph

3162 Commits

Author SHA1 Message Date
Zousar Shaker
fe196e0254 Prototype Wrapped Object Pointers (ObjectHandle/ObjectPtr)
Ensure that the Save1 codepath doesn't attempt to lazy load/resolve wrapped object pointers mid-save (which is not permitted).

#rb matt.peters
#rb francis.hurteau
#fyi devin.doucette

[CL 14826094 by Zousar Shaker in ue5-main branch]
2020-11-30 22:45:11 -04:00
Josh Adams
613180ea74 - Added ability for UObject to override the section name it uses for config properties. This is useful for moving a config class to another module, but still allowing back compatibility. The new name cannot be used in the .ini, only the old names can be used.
#rb brandon.schaefer

[CL 14824206 by Josh Adams in ue5-main branch]
2020-11-30 18:18:29 -04:00
Francis Hurteau
eb46965923 Move BeginCachePlatformCookedData to before package harvesting for Save2 using the same approach as PreSave
This prevent missing property name changed from default when using tagged property serialization for example

Fixed -DiffOnly not calling presave properly, since it shouldn't be called only on the DiffCallstack pass

#rb Matt.Peters

[CL 14822073 by Francis Hurteau in ue5-main branch]
2020-11-30 14:15:36 -04:00
Matt Peters
3a0e45b31f PackageResourceManager SearchForPackageOnDisk bugfix: IFileManager::FindFilesRecursive requires a wildcard that matches the full basename+extension, and I incorrectly left off the .* portion of the wildcard.
#rnx
#rb Rune.Stubbe

[CL 14817786 by Matt Peters in ue5-main branch]
2020-11-27 22:03:51 -04:00
Matt Peters
aae0fe62eb PackageResourceManager and PackagePaths
Create a new interface IPackageResourceManager that handles all attempts to load packages from storage. The default implementation passes through to loading from IFileManager. Future implementations will load from other services such as the EditorDomain and TargetDomain.
Add FPackagePath class that holds a LocalFilePath or LongPackageName, or a relative path to a package under a mounted content root (which can be converted into either the LocalFilePath or LongPackageName).
Modify LinkerLoad, BulkData, and AsyncLoader to use FPackagePath and IResourceManager.
Modify UPackage and FLinkerLoad to have a FPackagePath instead of a Filename.
#rb Paul.Chipchase, CarlMagnus.Nordin, Francis.Hurteau, Devin.Doucette
#rn Major Core

[CL 14814912 by Matt Peters in ue5-main branch]
2020-11-25 22:11:10 -04:00
Francis Hurteau
53b874ae28 Fixed bug where setting an external package to a package wasn't a noop as it should have resulting in GetObjectsInPackage to return the package itself after undo/redo operation.
#rb JeanFrancois.Dube

[CL 14814007 by Francis Hurteau in ue5-main branch]
2020-11-25 17:12:55 -04:00
paul chipchase
ab3002c705 CIS Fix: Change the ::Unlock method for BulkData2 to be const to match the original API.
- This stops code compiling fine in the editor then breaking when compiled for a different config.

#rb Martin.Ridgers
#fyi ben.ingram
#rnx

[CL 14812609 by paul chipchase in ue5-main branch]
2020-11-25 08:14:11 -04:00
Marc Audy
a7f9391231 Merge UE5/Release-Engine-Staging @ 14811410 to UE5/Main
This represents UE4/Main @ 14768117

For ReleaseObjectVersion.h
#lockdown Marcus.Wassmer

[CL 14811440 by Marc Audy in ue5-main branch]
2020-11-24 18:42:39 -04:00
Francis Hurteau
a2dc080863 Fixed Save2 when using unversioned serialization where the archive were properly set up but the necessary associated package flag was missing
Changed Save2 to use the old import sorting algo to keep binary consistent between the new and old save algo

Encapsulated "UseUnversionedPropertySerialization" in LinkerSave to set both the archive mode and the necessary package flag
Moved FObjectImportSortHelper to SavePackageUtilities
Cache "s.ProcessPrestreamingRequests" for Save2

#rb Matt.Peters

[CL 14790090 by Francis Hurteau in ue5-main branch]
2020-11-19 16:03:52 -04:00
Jamie Dale
f9db154c26 Allow a Reset TStrongObjectPtr to be destroyed during static de-init on Mac
On Mac the Main thread isn't the Game thread, so this trips the assert in ~FInternalReferenceCollector(), even if the TStrongObjectPtr has been manually Reset prior.

#jira UE-101794
#fyi Alexis.Matte
#rb Steve.Robb

[CL 14787366 by Jamie Dale in ue5-main branch]
2020-11-19 12:23:16 -04:00
sebastian nordgren
e1c8c29475 Added check for ArrayIndex >= 0 in Container*PtrToValuePtrInternal, because the pointer math can access invalid memory if negative.
This isn't related to any particular bug, but while debugging I found myself confirming that this was not the case - especially since INDEX_NONE may appear to be a valid value for ArrayIndex.

#rb paul.chipchase

[CL 14774958 by sebastian nordgren in ue5-main branch]
2020-11-18 10:08:43 -04:00
Martin Ridgers
be972be826 Moved the Trace namespace into the UE namespace.
#rb jb
#rnx

[CL 14762673 by Martin Ridgers in ue5-main branch]
2020-11-17 06:54:28 -04:00
Francis Hurteau
1bb81bb56b Disable Save2 again until new issue that cropped up on console is resolved
#rb trivial
#fyi Matt.Peters, Zousar.Shaker

[CL 14755790 by Francis Hurteau in ue5-main branch]
2020-11-16 12:51:33 -04:00
Zousar Shaker
3cdeeb8075 Prototype Wrapped Object Pointers (ObjectHandle/ObjectPtr) Change ternary operators that are ambiguous when TObjectPtr can be implicitly constructed from a raw pointer. Also allow TObjectPtr to be implicitly constructed from an appropriate raw pointer.
#rb devin.doucette

[CL 14739879 by Zousar Shaker in ue5-main branch]
2020-11-12 18:18:02 -04:00
Francis Hurteau
4f0fd5bf15 Re-enable Save2 for cooking
#rb trivial
#fyi Zousar.Shaker

[CL 14719822 by Francis Hurteau in ue5-main branch]
2020-11-11 11:30:21 -04:00
Christina TempelaarL
a9dd09fc9a Fixed blueprint nativization for SplineComponents/FInterpCurvePoints.
#jira UE-67542
#rb phillip.kavan

[CL 14713419 by Christina TempelaarL in ue5-main branch]
2020-11-10 17:16:52 -04:00
danny couture
3e4affd3e0 Allow async compilation to take place in Editor when starting in -game mode
- Allow multi-threaded bulkdata reading even when EDL is turned on

#rb Paul.Chipchase

[CL 14700046 by danny couture in ue5-main branch]
2020-11-10 11:31:12 -04:00
Francis Hurteau
6ca2cb8e82 Move setting GIsSavingPackage in Save2 to after reference gathering
#rb Zousar.Shaker

[CL 14699340 by Francis Hurteau in ue5-main branch]
2020-11-10 10:03:12 -04:00
danny couture
c23f6884fe Async Static Mesh Compilation / Loading
- Add protection against async property accessed during async build and postload.
   - Split PostLoad into 3 steps with the most part now being made async.
   - Make Build/BatchBuild API async when the feature is activated.
   - Add a new StaticMesh compilation manager. (This will be refactory later to reduce duplicated code with texture compiler).
   - Skip RenderState creation for any static mesh component still being built so they only show up when ready.
   - Fixed DistanceField and MeshCard computation that might trigger too soon if fixups are required to RenderData during PostLoad.
   - Dynamic priorisation for building and updating static meshes nearest to the viewport.
   - Implement different PIE modes, current default is to stall only for mesh that might affect nagivation/collision for players/bots.
   - Add a new generic interface for assets being built (still a WIP).
   - Add ability in FrontEnd filters to skip serialization of asset being async compiled and refresh when compilation finishes.
   - Prevent auto-save while textures and static meshes are being built (same as shaders)
   - Logic has been reordered in some Fortnite Building Component to avoid unnecessarily touching static mesh properties.

General Optimizations
   - Use cached thumbnails until shaders/textures/static meshes are ready to improve performance and avoid rendering incomplete thumbnails.

DEBUGGING
 - Can be forcibly enabled/disabled through command-line via -asyncstaticmeshcompilation=[off, on, paused]
 - Can pause staticmesh compilation using Editor.AsyncStaticMeshCompilation = 2 or -asyncstaticmeshcompilation=paused
 - Can manually resume a specified amount of paused compilation using Editor.AsyncStaticMeshCompilationResume [Num]
 - Can forcibly wait on all compilation using Editor.AsyncStaticMeshCompilationFlushAll

BENCHMARKS
 - 2m19s to 17.9s for Loading FortGPUTestbed's LumenTest with an empty local only DDC
 - 2h45m to 5mxxs for Loading Reverb's P_World with an empty local only DDC
 - 17m29 to 10m27s for Loading Apollo_Terrain_Edit with an empty local only DDC

TESTS
 - Loading maps from Reverb, Fortnite, ShooterGame, QAGame, FortGPUTestbed
 - Content browser interactions while still under heavy compilation
 - Importing with Datasmith / Dataprep
 - Working with the new Mesh Modeling Tool
 - Shutting down the editor while loading
 - Most of EngineTest Test Suite

REFERENCES
 - Design https://docs.google.com/document/d/1O4GI1G9AtQN6l0SaGFfXw0DyNhlLunLoEqne5INvxIQ
 - Documentation https://docs.google.com/document/d/1KCdFEMhhcsGwfgOisTVwlOwtsmEd7qbB0V6Tc39Gb10

#rb Francis.Hurteau

[CL 14688146 by danny couture in ue5-main branch]
2020-11-09 07:50:34 -04:00
Devin Doucette
c9f320e334 SharedBuffer: Extended shared buffer with an optional outer buffer and optional buffer owner
MakeView now has overloads that accept an outer shared buffer that it will hold a reference to and inherit the read-only state from. The view must be contained by the outer shared buffer.

TakeOwnership now has overloads that accept an instance of a type derived from FSharedBuffer::FBufferOwner that will be moved and stored to manage the lifetime of the buffer when a delete function is insufficient.

Functions that create buffers now return pointers instead of references, for more optimal reference counting. ToSharedRef can be called on the return value to convert it to a reference in cases where that is useful.

#jira none
#rb Zousar.Shaker
#rnx

[CL 14680479 by Devin Doucette in ue5-main branch]
2020-11-06 16:01:46 -04:00
Francis Hurteau
83febb3b86 Make sure the package localization cache is up to date before going in the GIsSavingPackage block in Save2
#rb Jamie.Dale
#jira UE-101268

[CL 14680029 by Francis Hurteau in ue5-main branch]
2020-11-06 15:51:51 -04:00
Francis Hurteau
e2d71cc438 Turning Save2 back off until some fixes are adressed.
#rb trivial
#fyi Zousar.Shaker

[CL 14677743 by Francis Hurteau in ue5-main branch]
2020-11-06 12:50:11 -04:00
Francis Hurteau
ff77076dfb Activate Save2 on cook by default
Activate Save2 everywhere on QAGame

#rb Matt.Peters, Zousar.Shaker

[CL 14664664 by Francis Hurteau in ue5-main branch]
2020-11-05 10:21:40 -04:00
Matt Peters
fb17a69e3d SavePackage private object errors: Identify private archetypes as the MostLikelyCulprit. Always print out the MostLikelyCulprit when running a commandlet rather than prompting.
#rb Zousar.Shaker
#rnx

[CL 14654440 by Matt Peters in ue5-main branch]
2020-11-04 13:35:45 -04:00
Francis Hurteau
5d34df4e57 Small Misc changes for Save2:
* Modified EnableNewSave cvar to use multiple values: 0: disabled, 1: enabled for cook, 2: enabled for uncooked, 3: enabled everywhere
* Added proper Prestream Packages support to Save2 since it isn't something that will be deprecated in the end
* Fixed new package not being saved properly when passing in a Base Object because they aren't properly marked as fully loaded in Save1
* Added a way for SaveConcurrent to use serial save with Save1 for now when not being passed in the SAVE_Concurrent flag
* Switched Save2 traces to use the SaveTimeChannel like Save1

#rb Matt.Peters

[CL 14652312 by Francis Hurteau in ue5-main branch]
2020-11-04 10:47:45 -04:00