Fixing backends to respect the masked flags includes a few fixes to handling of ~Query by Put and PutValue.
#rb Matt.Peters
#rnx
#preflight 61f7f7ede55232619f887977
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18790481 in //UE5/Release-5.0/... via CL 18790769 via CL 18791304
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545)
[CL 18791515 by devin doucette in ue5-main branch]
The name more accurately reflects the purpose of the flag, which is to allow a partial record to be fetched or stored in the absence of data for otherwise-required values.
#rb Zousar.Shaker
#rnx
#preflight 61e5ae87a2616066f68abc35
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18634974 in //UE5/Release-5.0/... via CL 18634979 via CL 18634988
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)
[CL 18637552 by devin doucette in ue5-main branch]
This hides implementation details and reduces the number of files that need to change when refactoring backends.
Backends moved in this change: AsyncPut, Http, Memory, S3, Zen.
#rb Zousar.Shaker
#rnx
#preflight 61dd0d758d72a407aab89074
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18573340 in //UE5/Release-5.0/... via CL 18573347
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)
[CL 18573359 by devin doucette in ue5-release-engine-test branch]
The structure with a value and attachments is an artifact of an early design where the record *was* a compact binary package. That design changed months ago and it now simplifies the cache record significantly to make it a collection of values without specific "value" and "attachment" types.
#rb Zousar.Shaker
#rnx
#preflight 61d8b94ed17842e547def37a
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18553971 in //UE5/Release-5.0/... via CL 18553976
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)
[CL 18553977 by devin doucette in ue5-release-engine-test branch]
A payload was conceptually a value with an ID. That has been formalized by removing the ID from the payload and having separate FValue and FValueId types. This separation cleans up the API in a few areas, and provides a more natural path to providing a basic key/value API.
#rb Zousar.Shaker
#rnx
#preflight 61d704c04c252480ca284d61
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18531844 in //UE5/Release-5.0/... via CL 18531856
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)
[CL 18531864 by devin doucette in ue5-release-engine-test branch]
The interface was in the public folder before 5.0, but was not usable from there in a meaningful way.
#rb Zousar.Shaker
#rnx
#preflight 61bbac388a62de8385c132c7
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18479761 in //UE5/Release-5.0/... via CL 18481541
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)
[CL 18481817 by devin doucette in ue5-release-engine-test branch]
Requests to Put and Get now specify the policy independently for each record or key, but the implementations have not yet been updated to take this into account.
The existing Put and Get API has been maintained temporarily to allow the conversion to happen separately, and will be removed after callers have been converted to use the new API.
#rb Zousar.Shaker
#rnx
#preflight 61b2e550ee0de9822e2e84dc
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18443052 in //UE5/Release-5.0/... via CL 18445037
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)
[CL 18445201 by devin doucette in ue5-release-engine-test branch]
Requiring the use of a separate reader type makes it more likely that readers will be reused, and makes it easier to audit reader usage going forward. Reusing readers is desirable to reduce the number of large temporary allocations made during partial decompression of a buffer.
- Added FCompressedBuffer::Save(FArchive&) and renamed FromCompressed(FArchive&) to Load(FArchive&).
- Added FCompressedBufferReaderSourceScope to set a buffer source within a scope.
- Added proper bounds checks to FNoneDecoder.
- Store the header checksum on the decoder context to allow raw blocks to be reused across sources.
- Decode the header on the fly to avoid a temporary header allocation when the header is in contiguous memory.
#rb Zousar.Shaker
#rnx
#preflight 61a98d53800738dbfbc84c73
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 18382211 in //UE5/Release-5.0/... via CL 18382310
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v896-18170469)
[CL 18382377 by devin doucette in ue5-release-engine-test branch]
This represents UE4/Main @17774255, Release-5.0 @17791557 and Dev-PerfTest @17789485
[CL 17794212 by aurel cordonnier in ue5-release-engine-test branch]
- ICacheStore::Put() has updated documentation to reflect the requirements of partial records.
- ICacheStore::Get() now takes a FCacheRecordPolicy, which is implicitly constructible from ECachePolicy, and allows setting the policy by payload.
- ICacheStore::GetPayload() is replaced by ICacheStore::GetChunks(), which allows loading parts of payloads.
- ICacheStore::CancelAll() is moved to ICache::CancelAll() because the cache can track requests at the top level and cancel them without exposing cancellation on individual cache stores.
- ECachePolicy::SkipLocalCopy has been removed because it is difficult to reason about.
- ECachePolicy::SkipData flags now have a documented meaning for put requests, to hint that record existence implies payload existence.
- The filesystem and memory cache stores have been updated to support partial records, filtering of payloads, and loading parts of payloads.
- Requesting part of a payload will decompress the entire payload for now, until compressed buffers expose a way to decompress only part.
- Fixed a bug in FTexturePlatformData::AreDerivedMipsAvailable() that caused it to return false for structured cache keys.
#rb Zousar.Shaker
#rnx
#preflight 615e03241ed62f0001b95454
#ROBOMERGE-OWNER: Devin.Doucette
#ROBOMERGE-AUTHOR: devin.doucette
#ROBOMERGE-SOURCE: CL 17748550 in //UE5/Release-5.0/... via CL 17748555
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v879-17706426)
#ROBOMERGE-CONFLICT from-shelf
#ROBOMERGE[STARSHIP]: UE5-Main
[CL 17748602 by Devin Doucette in ue5-release-engine-test branch]
FCacheBucket is now always exposed as FAnsiStringView.
#rb Zousar.Shaker
#rnx
#ROBOMERGE-SOURCE: CL 17084521 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v853-17066230)
[CL 17084549 by devin doucette in ue5-release-engine-test branch]
Any function that may create a request now has a IRequestOwner& parameter, and uses the Begin and End functions on the owner to manage the lifetime of any requests that it creates, as well as using End to invoke the completion callback for any request which has one.
The new FRequestBarrier may be used to block a group from being considered complete in a scope where more requests may be added to it.
#rb Matt.Peters
#rnx
#preflight 6109b5c403d303000144cce5
#preflight 610acf7103d30300016fda94
#ROBOMERGE-SOURCE: CL 17060470 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v850-17047176)
[CL 17060649 by devin doucette in ue5-release-engine-test branch]
- Optional versions are now returned from operations that may fail.
- Fixed a bug in the cache where a null record could be dereferenced.
- Improved validation when loading build types.
#rb Zousar.Shaker
#rnx
#ROBOMERGE-SOURCE: CL 16332909 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v804-16311228)
[CL 16332914 by devin doucette in ue5-release-engine-test branch]
- Optional versions are now returned from operations that may fail.
- Fixed a bug in the cache where a null record could be dereferenced.
- Improved validation when loading build types.
#rb Zousar.Shaker
#rnx
[CL 16332909 by Devin Doucette in ue5-main branch]
