Commit Graph

340 Commits

Author SHA1 Message Date
Devin Doucette
605998206f CompactBinary: Replaced local ReadUnaligned function with FPlatformMemory::ReadUnaligned
#preflight 6294ced34c7bcac173d19fb1
#rb Zousar.Shaker
#rnx

[CL 20431253 by Devin Doucette in ue5-main branch]
2022-05-30 11:19:23 -04:00
Per Larsson
c30e677d1b Merge BulkData 1&2
#rb Paul.Chipchase, PJ.Kack, CarlMagnus.Nordin
#jira none
#preflight 629085d14f63120d8e170977

###
This change attempts to merge the editor and the runtime version of bulk data, i.e. bulk data one and two.
The major difference between the two is the fact that the editor version supported element vice
serialization via inheritance. This has been complelety removed in the runtime version. In order
to mitigate this divergance a non-virtual base class has been extracted that forms the base
for the untyped and typed versions of bulk data.
The untyped version is only used in editor and will be deprecated. However this is a breaking change,
previously it was possible to inherit from FByteBulkData in editor builds.
Other differences between the two includes:
* IsBulkDataLoaded - returns true for bulk data 1 if an allocation has been made even if the size is zero
* StartStreaming - was removed in bulk data 2 and has been removed
* FlushAsyncLoading - bulk data 2 was busy waiting, now an event is used

All I/O related functionality has been extracted into BulkDataStreaming.cpp and is using
a  FBulkDataChunkId that is either a package path or a package ID.This could potentially
be extended to contain a cache key in development builds to support loading from DDC, i.e derived
data references.

[CL 20391050 by Per Larsson in ue5-main branch]
2022-05-27 04:31:58 -04:00
Steve Robb
7778bc5c05 GitHub #8600 : Rename Ar to StructuredArchive
#rb steve.robb
#jira UE-134432
#preflight 628e287d2c7cb2d10d65322d

[CL 20363343 by Steve Robb in ue5-main branch]
2022-05-25 09:20:14 -04:00
Matt Peters
02ba0b5b70 ZenStoreWriter: Fix assertion "Trying to add package that is already pending" if the cooker commits a package a second time while the first commit of the package is queued for asynchronous completion aand has not yet reached CommitPackageInternal.
This is necessary because as of CL 20151503, the cooker can commit the same package multiple times if the first save failed due to a timeout.

#fyi CarlMagnus.Nordin
#rnx
#rb Per.Larsson
#preflight 62853ba90f0d5076dee74ced

[CL 20265738 by Matt Peters in ue5-main branch]
2022-05-18 14:49:11 -04:00
robert millar
949ed7ba3a Insert dummy values into memory images for names like vtable pointers.
#rb ben.ingram

#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 20115901 via CL 20115948 via CL 20115961 via CL 20115990
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 20117747 by robert millar in ue5-main branch]
2022-05-09 23:27:24 -04:00
ilya loshchinin
33b775b08a FLargeMemoryData buildfix: missing include
#ROBOMERGE-AUTHOR: ilya.loshchinin
#ROBOMERGE-SOURCE: CL 19971151 via CL 19973220 via CL 19973386
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 19974313 by ilya loshchinin in ue5-main branch]
2022-04-28 21:19:04 -04:00
ilya loshchinin
6bd4945ede Optimized StaticDuplicateObject by pooling its temporary memory block allocation.
Added FPooledLargeMemoryData that allows lockless access to temporary memory blocks.

#tests Squads games on WSL, using FramePro to validate performance gains
#rb danny.couture,jamie.dale

#ROBOMERGE-AUTHOR: ilya.loshchinin
#ROBOMERGE-SOURCE: CL 19970339 via CL 19971380 via CL 19972286
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 19974304 by ilya loshchinin in ue5-main branch]
2022-04-28 21:18:27 -04:00
steve robb
76eddd083e Workaround for IncludeTool error.
#rb robert.manuszewski
#jira UE-148948
#preflight 6267d7ad5c29321b2dc74f60

#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19918554 via CL 19919564 via CL 19922223 via CL 19922670
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v943-19904690)

[CL 19925628 by steve robb in ue5-main branch]
2022-04-26 14:38:27 -04:00
Devin Doucette
92a888c1f6 Replaced uses of _ASV with ANSITEXTVIEW
#preflight 6266acf50634d0904cdfe91c
#rb Steve.Robb
#rnx

[CL 19903251 by Devin Doucette in ue5-main branch]
2022-04-25 11:31:36 -04:00
CarlMagnus Nordin
63b8cbc9dd AsyncLoading2: Implemented loading of optional package segments
#rnx
#preflight 62627548ac29fcf6a213fe33
#rb francis.hurteau

[CL 19900368 by CarlMagnus Nordin in ue5-main branch]
2022-04-25 07:37:07 -04:00
Johan Torp
ef8e0c4660 Optimize WITH_CASE_PRESERVING_NAME FName loading
LoadDisplayNames() 1.06s -> 0.24s when async loading the name batch in CachedAssetRegistry.bin during editor startup of an internal project

* Introduce FDisplayNameEntryId to avoid cache misses in FName::GetComparisonIdFromDisplayId() lookups
* Force name batch serialization to use display entry ids to help enforce cooking determinism
* Fix determinism problem when loading then saving via FPackageStoreNameMapBuilder
* Inline FNameEntryId::FromUnstableInt()
* Remove slow IsNumbered() check in CreateFromDisplayId when not using 4B FName
* Remove expensive heap sort by sorting up front instead

#rb rob.millar,matt.peters,pj.kack
#preflight 62664bd06968f18d5137d419

[CL 19899104 by Johan Torp in ue5-main branch]
2022-04-25 03:48:33 -04:00
steve robb
c4817fc296 Removal of SA_FIELD_NAME.
#rb graeme.thornton
#preflight 624f47086637e291d3d49b2d
[FYI] francis.hurteau

#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19680202 via CL 19681535 via CL 19681539 via CL 19681542
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19682750 by steve robb in ue5-main branch]
2022-04-07 21:06:54 -04:00
steve robb
a74e2ef185 Missing include added.
#rb none
#preflight none

#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19647781 via CL 19647829 via CL 19647844
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19651214 by steve robb in ue5-main branch]
2022-04-06 13:46:40 -04:00
steve robb
0d3a8d60fa Missing includes added.
#rb none
#preflight none

#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19646815 via CL 19646849 via CL 19646914
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19651131 by steve robb in ue5-main branch]
2022-04-06 13:44:19 -04:00
steve robb
0316418f30 Split StructuredArchive code into individual files.
#rb none
[FYI] graeme.thornton, francis.hurteau
#preflight 624ce75c0b1b11b96bb0d52e

#ROBOMERGE-OWNER: steve.robb
#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19644869 via CL 19645450 via CL 19645856 via CL 19645875
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v938-19570697)

[CL 19651014 by steve robb in ue5-main branch]
2022-04-06 13:39:54 -04:00
Dan Thompson
cd73707429 Update package writers to gather package hashes internally, then have CookByTheBookFinished copy them to asset package datas.
Add iostore chunk hashes to the asset registry during cook.
Remove old hashing code, which was extensive. This moves the async writes in to UE::Task from the old AsyncWorkSequence().
#rb Matt.Peters
#rb Jeff.Roberts
#preflight 624b24a5f73c316f68303946

[CL 19611812 by Dan Thompson in ue5-main branch]
2022-04-04 13:26:27 -04:00
brian bekich
fda9b2af93 Add ResetData to FBitReader that won't reallocate the internal buffer unless it is necessary
Reducing allocations during replay scrub

#rb john.barrett

#ROBOMERGE-AUTHOR: brian.bekich
#ROBOMERGE-SOURCE: CL 19482343 via CL 19483947 via CL 19486339 via CL 19496184 via CL 19496470
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v936-19480137)

[CL 19498463 by brian bekich in ue5-main branch]
2022-03-24 13:29:02 -04:00
fabian giesen
8a5308dc23 FArchiveState::CustomVer doc comment fix
The implementation returns -1 if the version tag is not found, not 0.

#rb danny.couture
#preflight 623bf802437e1bac26c112a2

[CL 19492200 by fabian giesen in ue5-main branch]
2022-03-24 01:03:37 -04:00
zach bethel
50274e3050 Converted FIntPoint / FIntVector / FIntRect to a template type and added unsigned variants.
#rb luke.thatcher, andrew.davidson, steve.robb
#preflight 6239e9d204769ab49354a30f

[CL 19468937 by zach bethel in ue5-main branch]
2022-03-22 13:40:58 -04:00
robert millar
9b14ca87aa When building name patches, compare names case sensitively. (Sorting was already done case sensitively.)
Removing ensure from FName about unfreezing names that were created for a case-insensitive target. We rely on the archive used for serialization to save names with their case intact.

#rb ben.ingram, johan.torp

#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19422180 via CL 19424113 via CL 19430180 via CL 19430244
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v930-19419903)

[CL 19432581 by robert millar in ue5-main branch]
2022-03-18 02:34:50 -04:00
Per Larsson
b7980e9e38 Add support for reserving memory when using LargeMemoryWriter.
#rb paul.chipchase
#rnx
#preflight 19417934

[CL 19418052 by Per Larsson in ue5-main branch]
2022-03-17 04:05:15 -04:00
steve robb
0e1e8c13fb Removal of unused CopyFormatted*() and *_TextOnly() functions.
#rb graeme.thornton
#preflight 6229fae77341dd1ecf70f0a0

#ROBOMERGE-OWNER: marc.audy
#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19335163 via CL 19339363 via CL 19345711 via CL 19350978 via CL 19351029
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)

[CL 19351730 by steve robb in ue5-main branch]
2022-03-11 04:01:36 -05:00
marc audy
abffec6e31 StructuredArchive namespace fixes.
#rb none
#preflight 6228bfd231133a23da83376d


#ushell-cherrypick of 19339219 by steve.robb

#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19320363 via CL 19345602 via CL 19350976 via CL 19351027
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)

[CL 19351729 by marc audy in ue5-main branch]
2022-03-11 04:01:29 -05:00
steve robb
d2b1edbe26 Structured archive config defines moved into their own header.
#rb none
#preflight 6228a767d773d6497bfc599d

#ROBOMERGE-AUTHOR: steve.robb
#ROBOMERGE-SOURCE: CL 19319301 via CL 19339201 via CL 19344374 via CL 19350327 via CL 19350522
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v926-19321884)

[CL 19351704 by steve robb in ue5-main branch]
2022-03-11 03:58:24 -05:00
arciel rekman
0872f56977 Fix overflows reported by static analysis (UE-141612)
#rb Ben.Ingram, Jason.Nadro
[REVIEW] [at]Ben.Ingram, [at]Jason.Nadro
#jira UE-141612
#preflight 6220e0846a33f9b416aa4681
#lockdown Mihnea.Balta
#rnx

#ROBOMERGE-AUTHOR: arciel.rekman
#ROBOMERGE-SOURCE: CL 19244385 in //UE5/Release-5.0/... via CL 19248971
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v924-19243027)

[CL 19262903 by arciel rekman in ue5-main branch]
2022-03-04 03:03:38 -05:00