Ben Marsh
0b926700de
Horde: Flush nodes to the backing store as they are generated.
...
#preflight none
[CL 22230462 by Ben Marsh in ue5-main branch]
2022-09-28 16:37:39 -04:00
Ben Marsh
f1b55f3565
Horde: Implement ranged reads for storage backends. Remove code to emulate ranged reads from StorageClientBase.
...
#preflight none
[CL 22227699 by Ben Marsh in ue5-main branch]
2022-09-28 14:37:09 -04:00
Ben Marsh
71e0f535e2
Horde: Add a read queue to StorageClientBase, and add support for concurrent reads.
...
* Background task creates up to 4 concurrent read threads.
* When reading a bundle header, we read a fixed 1mb chunk of data from the start of the bundle and add any encoded packets within that range to the cache.
* When reading a bundle packet, we also read more data that specifically requested, and add that to the cache.
* Packet decode requests are also performed asynchronously and shared between requests.
#preflight none
[CL 22226744 by Ben Marsh in ue5-main branch]
2022-09-28 13:58:49 -04:00
Zousar Shaker
6540112e2f
Allow an Epic-only configuration on UnrealHeaderTool that enforces TObjectPtr usage in Engine Plugins specifically. Cannot be used externally due to the occurrance of third party plugins being placed in the engine plugins directory. Tested against C++ UHT and C# UHT both with and without the Epic-only configuration file present.
...
#rb tim.smith
#preflight 6334666b0169596cdf0cc710
[CL 22225475 by Zousar Shaker in ue5-main branch]
2022-09-28 12:58:24 -04:00
Ben Marsh
5a3173bbe3
EpicGames.Perforce: Add support for "p4 sizes" command. #9584
...
#jira UE-164680
#preflight none
[CL 22222365 by Ben Marsh in ue5-main branch]
2022-09-28 10:47:54 -04:00
Joe Kirchoff
ebea927bd0
UnrealBuildTool: Fix analysis warnings
...
#rnx
#rb trivial
#preflight skipped
[CL 22214533 by Joe Kirchoff in ue5-main branch]
2022-09-27 19:15:34 -04:00
Joe Kirchoff
1e0e05a324
UnrealBuildTool: File IoHasher with caching based on last write time and file size, with simple saving and loading of the hash database
...
Add tests for Epicgames.IoHash
#rnx
#rb Ben.Marsh
#preflight 63334d76665f6b8f7f45c2e3
[CL 22209062 by Joe Kirchoff in ue5-main branch]
2022-09-27 16:42:06 -04:00
Ben Marsh
0dfed3f566
Horde: Multi-threaded processing of files being added to bundles.
...
#preflight none
[CL 22208034 by Ben Marsh in ue5-main branch]
2022-09-27 14:42:23 -04:00
tim smith
0c077846eb
Fixed issue where already built UBT plugins were not being listed properly.
...
#rb self
#rnx
#preflight 6332fb5ab20e73a0989d5f7d
[CL 22205909 by tim smith in ue5-main branch]
2022-09-27 10:35:15 -04:00
Ben Marsh
9f4e2c95e1
EpicGames.Core: Add utility method for generating SourceFile log values with custom display text.
...
#preflight none
[CL 22205245 by Ben Marsh in ue5-main branch]
2022-09-27 10:10:39 -04:00
Ben Marsh
80d884159f
Fix missing comments.
...
#preflight none
[CL 22204156 by Ben Marsh in ue5-main branch]
2022-09-27 08:48:37 -04:00
Ben Marsh
0d70ed83e6
Horde: Refactor IStorageClient.
...
* Now supports partial reads of blobs, and caches them inside StorageClientBase.
* IStorageClient includes methods for reading/writing TreeNode-derived classes, and caches bundle information and encoded/decoded packets internally.
* TreeWriter class can be used for packing data together into bundles. Supports creating async tasks to fence nodes being flushed rather than an explicit concept of forking child writers.
#preflight none
[CL 22192856 by Ben Marsh in ue5-main branch]
2022-09-26 16:09:47 -04:00
Ben Marsh
83180e089d
EpicGames.Perforce: Fix incorrect path copying native PDB to output directory.
...
#preflight none
[CL 22180965 by Ben Marsh in ue5-main branch]
2022-09-26 09:18:01 -04:00
Ben Marsh
8118b1a3e0
Horde: Move cache management and partial reads of bundles into the implementation of IStorageClient.
...
Implementation of TreeStore will likely be the same for all implementations, while there are many different strategies we might use for bulk data.
#preflight none
[CL 22158930 by Ben Marsh in ue5-main branch]
2022-09-23 14:52:11 -04:00
Ben Marsh
03bd4507be
Horde: Store bundle packets in separate memory blocks, and move encode/decode logic out of TreeStore.
...
#preflight none
[CL 22157363 by Ben Marsh in ue5-main branch]
2022-09-23 13:12:10 -04:00
Ben Marsh
59a39161b0
Horde: Add support for ranged reads of blobs.
...
#preflight none
[CL 22155634 by Ben Marsh in ue5-main branch]
2022-09-23 11:26:56 -04:00
tim smith
c974960bd6
Modified C# UHT to generate errors when extra ';' are detected. C++ UHT will generate an error.
...
#rb self
#rnx
#preflight 632c69f2c7791417aa85136b
[CL 22145828 by tim smith in ue5-main branch]
2022-09-22 17:37:26 -04:00
tim smith
075437d87a
Fixed C# UHT issue where unterminated strings were not detected properly and could result in an infinite loop.
...
#rb self
[FYI] matt.peters
#preflight 632c5744b4515b7e22c50162
[CL 22145735 by tim smith in ue5-main branch]
2022-09-22 17:34:34 -04:00
jeff newquist
8e7244f6b7
Fix turnkey valid SDK filtering
...
#rb josh.adams
#jira UE-164423
#preflight none
[CL 22141936 by jeff newquist in ue5-main branch]
2022-09-22 14:59:04 -04:00
Ben Marsh
c7960bdfae
Horde: Add support for using Oodle to compress bundle packets.
...
#preflight none
[CL 22140891 by Ben Marsh in ue5-main branch]
2022-09-22 14:19:16 -04:00
tim smith
d97581da03
Fix wrong API macro being used in C# UHT
...
Cleaned up formatting in both C++ and C# UHT
#rb self
#preflight 632b37b21003050806ec9f3a
[CL 22140411 by tim smith in ue5-main branch]
2022-09-22 14:05:55 -04:00
Ben Marsh
654134a1a4
Horde: Add a C# wrapper library for Oodle data.
...
#preflight none
[CL 22140325 by Ben Marsh in ue5-main branch]
2022-09-22 14:03:34 -04:00
Ben Marsh
6edb2f147b
Horde: Add support for Gzip compression in bundles.
...
#preflight none
[CL 22138345 by Ben Marsh in ue5-main branch]
2022-09-22 12:35:58 -04:00
Ben Marsh
a0ace7ac33
Horde: Encode data into bundles asynchonously while prefetching the next block of data.
...
#preflight none
[CL 22138005 by Ben Marsh in ue5-main branch]
2022-09-22 12:19:15 -04:00
Ben Marsh
d06078c7f2
Horde: Add Stream wrappers around IMemoryReader and IMemoryWriter, and fix ByteArrayBuilder.GetMemory() always returning the minimum buffer size.
...
#preflight none
[CL 22136931 by Ben Marsh in ue5-main branch]
2022-09-22 11:23:43 -04:00