Ben Marsh 71e0f535e2 Horde: Add a read queue to StorageClientBase, and add support for concurrent reads. ...

* Background task creates up to 4 concurrent read threads.
* When reading a bundle header, we read a fixed 1mb chunk of data from the start of the bundle and add any encoded packets within that range to the cache.
* When reading a bundle packet, we also read more data that specifically requested, and add that to the cache.
* Packet decode requests are also performed asynchronously and shared between requests.

#preflight none

[CL 22226744 by Ben Marsh in ue5-main branch]

2022-09-28 13:58:49 -04:00

..

AssemblyUtils.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

AsyncEvent.cs

…

AsyncUtils.cs

Horde: Add a read queue to StorageClientBase, and add support for concurrent reads.

2022-09-28 13:58:49 -04:00

BinaryArchive.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

BinaryArchiveAttributes.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

BinaryArchiveReader.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

BinaryArchiveWriter.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

BinaryConverter.cs

…

BinaryFormatterUtils.cs

…

BinaryReaderExtensions.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

BinaryWriterExtensions.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

Boxed.cs

…

BuzHash.cs

Fix missing documentation comment.

2022-08-17 09:29:32 -04:00

ByteArrayBuilder.cs

Horde: Refactor IStorageClient.

2022-09-26 16:09:47 -04:00

CancellationTask.cs

…

CertificateUtils.cs

…

CityHash.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

CityHash.tps

…

CommandHost.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

CommandLineArguments.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

CommandLineAttribute.cs

…

CommandLineExtensions.cs

…

ConsoleUtils.cs

…

ContentHash.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

CsProjBuildHook.cs

Fixed issue with UAT/UBT script module rebuild tracking. If MSBuild doesn't update the target DLL due to there being no real changes, then the simple check against target time stamps being out of order would fail. Changed to save the reference time stamp and then compare them to test to see if they don't match.

2022-07-15 08:42:45 -04:00

CsProjBuildRecord.cs

Fixing missing XML documentation

2022-07-15 11:41:05 -04:00

CsProjectInfo.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

Digest.cs

Add a common interface for reading/writing to memory (IMemoryReader/IMemoryWriter), which is implemented by the legacy MemoryReader/MemoryWriter classes as well as the newer ByteArrayBuilder.

2022-06-28 16:14:51 -04:00

DirectoryReference.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

DirectoryUtils.cs

…

EnumerableExtensions.cs

…

EpicGames.Core.csproj

Horde: Upgrade all NuGet packages to latest.

2022-08-29 13:34:15 -04:00

ExceptionUtils.cs

Branch snapshot for CL 21319338

2022-08-10 16:03:37 +00:00

FatalErrorException.cs

…

FileFilter.cs

BuildGraph: Fix path matching with UNC paths.

2022-06-27 10:24:44 -04:00

FileFilterNode.cs

…

FilePattern.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

FileReference.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

FileSet.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

FileSystemReference.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

FileTransaction.cs

Fix missing XML documentation

2022-07-06 11:11:48 -04:00

FileUtils.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

ForwardingLogger.cs

Horde: Upgrade all NuGet packages to latest.

2022-08-29 13:34:15 -04:00

HelpUtils.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

HttpClientExtensions.cs

…

ILogCursor.cs

BuildGraph: Reformat output from Docker to allow regular pattern matchers to work correctly, and add an MSTest event matcher.

2022-06-09 13:18:32 -04:00

ILoggerIndent.cs

Horde: Upgrade all NuGet packages to latest.

2022-08-29 13:34:15 -04:00

ILoggerProgress.cs

…

JsonExtensions.cs

…

JsonKnownTypes.cs

…

JsonLogEvent.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

JsonObject.cs

…

JsonSchema.cs

Horde: Add tests for schema generation, and fix current serialization issues.

2022-09-09 10:28:19 -04:00

JsonSchemaAttributes.cs

…

JsonSchemaType.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

JsonTracer.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

JsonWriter.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

KnownLogEvents.cs

Unshelved from pending changelist '22074449':

2022-09-19 21:54:31 -04:00

ListExtensions.cs

…

ListSegment.cs

…

Log.cs

Horde: Upgrade all NuGet packages to latest.

2022-08-29 13:34:15 -04:00

LogBuffer.cs

Horde: Use long-lived regex instances for matching patterns.

2022-05-31 12:02:25 -04:00

LogEvent.cs

EpicGames.Core: Add utility method for generating SourceFile log values with custom display text.

2022-09-27 10:10:39 -04:00

LogEventBuilder.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

LogEventMarkup.cs

EpicGames.Core: Add utility method for generating SourceFile log values with custom display text.

2022-09-27 10:10:39 -04:00

LogEventParser.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

LogIndent.cs

…

LogIndentScope.cs

…

LogStatusScope.cs

…

LogValue.cs

EpicGames.Core: Add utility method for generating SourceFile log values with custom display text.

2022-09-27 10:10:39 -04:00

LogValueFormatter.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

ManagedProcess.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

Md5Hash.cs

…

MemoryReader.cs

Horde: Refactor IStorageClient.

2022-09-26 16:09:47 -04:00

MemoryReaderStream.cs

Horde: Add Stream wrappers around IMemoryReader and IMemoryWriter, and fix ByteArrayBuilder.GetMemory() always returning the minimum buffer size.

2022-09-22 11:23:43 -04:00

MemoryWriter.cs

Fix missing comments.

2022-09-27 08:48:37 -04:00

MemoryWriterStream.cs

Horde: Add Stream wrappers around IMemoryReader and IMemoryWriter, and fix ByteArrayBuilder.GetMemory() always returning the minimum buffer size.

2022-09-22 11:23:43 -04:00

MessageTemplate.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

ParallelTask.cs

…

ProcessUtils.cs

…

ReadOnlyHashSet.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

ReadOnlyListExtensions.cs

…

ReadOnlyMemoryStream.cs

…

ReadOnlySequenceBuilder.cs

Horde: API for manipulating tree structures in storage. Designed to abstract the logical tree structure away from storage/retrieval when allow working on larger-than-memory data sets.

2022-06-29 21:14:05 -04:00

ReadOnlySequenceStream.cs

Horde: Write memory sequences to storage without copying to a separate buffer first.

2022-08-30 09:34:19 -04:00

RuntimePlatform.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

ScopeProfiler.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

SemVer.cs

…

Sha1Hash.cs

Add a common interface for reading/writing to memory (IMemoryReader/IMemoryWriter), which is implemented by the legacy MemoryReader/MemoryWriter classes as well as the newer ByteArrayBuilder.

2022-06-28 16:14:51 -04:00

StartupTraceListener.cs

…

StreamUtils.cs

Horde: Store bundle packets in separate memory blocks, and move encode/decode logic out of TreeStore.

2022-09-23 13:12:10 -04:00

StringComparerUE.cs

…

StringUtils.cs

Horde: Updated blob store API.

2022-08-27 16:18:15 -04:00

StringView.cs

…

ThreadPoolWorkQueue.cs

…

TraceSpan.cs

…

UHTTypes.cs

Fix Xmldoc warnings in EpicGames.Core prior to enabling tests in CI.

2022-06-22 13:23:01 -04:00

UnrealEngineTypes.cs

Updated documentation on PackageFlag for C#

2022-08-23 08:06:25 -04:00

Utf8String.cs

Add a common interface for reading/writing to memory (IMemoryReader/IMemoryWriter), which is implemented by the legacy MemoryReader/MemoryWriter classes as well as the newer ByteArrayBuilder.

2022-06-28 16:14:51 -04:00

VarInt.cs

Add a common interface for reading/writing to memory (IMemoryReader/IMemoryWriter), which is implemented by the legacy MemoryReader/MemoryWriter classes as well as the newer ByteArrayBuilder.

2022-06-28 16:14:51 -04:00

Win32ExceptionWithCode.cs

…

ZipArchiveExtensions.cs

Unzip optimization for archives with lots of small files

2022-07-01 19:42:38 -04:00