izzy/UnrealEngineUWP
0
0
Fork 0
mirror of https://github.com/izzy2lost/UnrealEngineUWP.git synced 2026-03-26 18:15:20 -07:00
Code Issues Packages Projects Releases Wiki Activity
Files
5.5
UnrealEngineUWP/Engine/Source/Developer/DerivedDataCache/Private/ZenCacheStore.cpp

2275 lines
79 KiB
C++
Raw Permalink Normal View History

Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
// Copyright Epic Games, Inc. All Rights Reserved.
DDC: Temporary compile fix for platforms that should not include DerivedDataCache #preflight skip #rb none #rnx [CL 20445864 by Devin Doucette in ue5-main branch]
2022-05-31 23:25:37 -04:00
#include "DerivedDataLegacyCacheStore.h"
Move Zen module public headers into an "Experimental" subfolder to have them reflect the feature status. Fix locations that include the headers to use the "Experimental" subfolder and re-sort includes. No functional change, purely a move, search+replace, and sort of includes. #rb matt.peters #preflight 63939eebc415e8dc78d35d63 [CL 23468934 by Zousar Shaker in ue5-main branch]
2022-12-09 16:11:41 -05:00
#include "Experimental/ZenServerInterface.h"
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
#include "Experimental/ZenStatistics.h"
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
#if UE_WITH_ZEN
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
#include "Async/ManualResetEvent.h"
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
#include "Async/UniqueLock.h"
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
#include "BatchView.h"
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
#include "DerivedDataBackendInterface.h"
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
#include "DerivedDataCachePrivate.h"
#include "DerivedDataCacheRecord.h"
#include "DerivedDataCacheUsageStats.h"
#include "DerivedDataChunk.h"
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
#include "DerivedDataHttpRequestQueue.h"
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
#include "DerivedDataRequest.h"
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
#include "DerivedDataRequestOwner.h"
Move Zen module public headers into an "Experimental" subfolder to have them reflect the feature status. Fix locations that include the headers to use the "Experimental" subfolder and re-sort includes. No functional change, purely a move, search+replace, and sort of includes. #rb matt.peters #preflight 63939eebc415e8dc78d35d63 [CL 23468934 by Zousar Shaker in ue5-main branch]
2022-12-09 16:11:41 -05:00
#include "Experimental/ZenStatistics.h"
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
#include "HAL/FileManager.h"
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
#include "HAL/Thread.h"
2nd submit: Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 #preflight 629642a1be0ae0b33a46d012 [CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
#include "Http/HttpClient.h"
Select maximum HTTP connections for ZenCacheStore based on logical core count (clamped between 8 and 64). Speculative solution for farm cooks using Zen experiencing "couldn't connect errors". #rb trivial #fyi devin.doucette #rnx #preflight 62b00bd92405dc6102315616 [CL 20734103 by Zousar Shaker in ue5-main branch]
2022-06-20 02:08:49 -04:00
#include "Math/UnrealMathUtility.h"
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
#include "Misc/App.h"
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
#include "Misc/ConfigCacheIni.h"
#include "Misc/Optional.h"
#include "ProfilingDebugging/CountersTrace.h"
#include "ProfilingDebugging/CpuProfilerTrace.h"
#include "Serialization/CompactBinary.h"
#include "Serialization/CompactBinaryPackage.h"
#include "Serialization/CompactBinaryWriter.h"
2nd submit: Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 #preflight 629642a1be0ae0b33a46d012 [CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
#include "Serialization/LargeMemoryWriter.h"
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
#include "Serialization/MemoryReader.h"
2nd submit: Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 #preflight 629642a1be0ae0b33a46d012 [CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
#include "Templates/Function.h"
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
#include "ZenBackendUtils.h"
#include "ZenSerialization.h"
[ZenCacheStore] - Fix race condition on counters #rb dan.engelbrecht [CL 31255886 by danny couture in ue5-main branch]
2024-02-07 09:25:38 -05:00
TRACE_DECLARE_ATOMIC_INT_COUNTER(ZenDDC_Get, TEXT("ZenDDC Get"));
TRACE_DECLARE_ATOMIC_INT_COUNTER(ZenDDC_GetHit, TEXT("ZenDDC Get Hit"));
TRACE_DECLARE_ATOMIC_INT_COUNTER(ZenDDC_Put, TEXT("ZenDDC Put"));
TRACE_DECLARE_ATOMIC_INT_COUNTER(ZenDDC_PutHit, TEXT("ZenDDC Put Hit"));
TRACE_DECLARE_ATOMIC_INT_COUNTER(ZenDDC_BytesReceived, TEXT("ZenDDC Bytes Received"));
TRACE_DECLARE_ATOMIC_INT_COUNTER(ZenDDC_BytesSent, TEXT("ZenDDC Bytes Sent"));
TRACE_DECLARE_ATOMIC_INT_COUNTER(ZenDDC_CacheRecordRequestCountInFlight, TEXT("ZenDDC CacheRecord Request Count"));
TRACE_DECLARE_ATOMIC_INT_COUNTER(ZenDDC_ChunkRequestCountInFlight, TEXT("ZenDDC Chunk Request Count"));
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
namespace UE::DerivedData
{
template<typename T>
void ForEachBatch(const int32 BatchSize, const int32 TotalCount, T&& Fn)
{
check(BatchSize > 0);
if (TotalCount > 0)
{
const int32 BatchCount = FMath::DivideAndRoundUp(TotalCount, BatchSize);
const int32 Last = TotalCount - 1;
for (int32 BatchIndex = 0; BatchIndex < BatchCount; BatchIndex++)
{
const int32 BatchFirstIndex = BatchIndex * BatchSize;
const int32 BatchLastIndex = FMath::Min(BatchFirstIndex + BatchSize - 1, Last);
Fn(BatchFirstIndex, BatchLastIndex);
}
}
}
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
struct FZenCacheStoreParams
{
FString Name;
FString Host;
FString Namespace;
FString Sandbox;
int32 MaxBatchPutKB = 1024;
int32 RecordBatchSize = 8;
int32 ChunksBatchSize = 8;
float DeactivateAtMs = -1.0f;
TOptional<bool> bLocal;
TOptional<bool> bRemote;
bool bFlush = false;
bool bReadOnly = false;
Add option to bypass proxies to the derived data http client (default is no bypass). Zen cache store client uses it and defaults to bypassing proxies (but can be overridden in INI). UE Cloud DDC client uses it and defaults to NOT bypassing proxies (but can be overridden in INI). #rb Devin.Doucette [FYI] dan.engelbrecht [FYI] joakim.lindqvist #jira UE-210567 [CL 32490735 by zousar shaker in ue5-main branch]
2024-03-25 18:59:10 -04:00
bool bBypassProxy = true;
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
void Parse(const TCHAR* NodeName, const TCHAR* Config);
};
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
/**
* Backend for a HTTP based caching service (Zen)
*/
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
class FZenCacheStore final : public ILegacyCacheStore
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
{
public:
/**
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
* Creates the cache store client, checks health status and attempts to acquire an access token.
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
*
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
* @param ServiceUrl Base url to the service including scheme.
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
* @param Namespace Namespace to use.
*/
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
FZenCacheStore(
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
const FZenCacheStoreParams& InParams,
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
ICacheStoreOwner* Owner);
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
FZenCacheStore(
UE::Zen::FServiceSettings&& Settings,
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
const FZenCacheStoreParams& InParams,
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
ICacheStoreOwner* Owner);
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
DDC: Fixed leaking cache store stats in the Zen cache #jira UE-133395 #rb none #rnx [CL 25977512 by devin doucette in ue5-main branch]
2023-06-14 10:12:47 -04:00
~FZenCacheStore() final;
Adjusting logs related to zen usage to: - Remove repeated log related to fetching cache stats - Ensure that the Zen cache usage has log lines explaining the status of the caches it attempts to connect to (success or failure) #rb Matt.Peters [CL 30710842 by zousar shaker in ue5-main branch]
2024-01-18 22:43:41 -05:00
inline const FString& GetName() const { return NodeName; }
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
/**
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
* Checks if cache service is usable (reachable and accessible).
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
* @return true if usable
*/
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
inline bool IsUsable() const { return bIsUsable; }
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
Downgrade warning severity messages to display severity when we are unable to reach a remote zenserver (as opposed to a local one). #rb Devin.Doucette [CL 29761186 by zousar shaker in ue5-main branch]
2023-11-15 17:17:29 -05:00
/**
* Checks if cache service is on the local machine.
* @return true if it is local
*/
inline bool IsLocalConnection() const { return bIsLocalConnection; }
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
// ICacheStore
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
void Put(
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
TConstArrayView<FCachePutRequest> Requests,
IRequestOwner& Owner,
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
FOnCachePutComplete&& OnComplete = FOnCachePutComplete()) final;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
void Get(
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
TConstArrayView<FCacheGetRequest> Requests,
IRequestOwner& Owner,
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
FOnCacheGetComplete&& OnComplete) final;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
void PutValue(
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
TConstArrayView<FCachePutValueRequest> Requests,
IRequestOwner& Owner,
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
FOnCachePutValueComplete&& OnComplete = FOnCachePutValueComplete()) final;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
void GetValue(
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
TConstArrayView<FCacheGetValueRequest> Requests,
IRequestOwner& Owner,
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
FOnCacheGetValueComplete&& OnComplete = FOnCacheGetValueComplete()) final;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
void GetChunks(
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
TConstArrayView<FCacheGetChunkRequest> Requests,
IRequestOwner& Owner,
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
FOnCacheGetChunkComplete&& OnComplete) final;
// ILegacyCacheStore
void LegacyStats(FDerivedDataCacheStatsNode& OutNode) final;
bool LegacyDebugOptions(FBackendDebugOptions& Options) final;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
Change Zen DDC cache store to allow: - Local storage flag to be inferred by whether the service is run locally or set explicitly in config. - Remote storage flag to be inferred by whether the service is NOT run locally or set explicitly in config. - ReadOnly storage flag to be set explicitly in config (defaults to true). #rb devin.doucette [CL 24822351 by zousar shaker in ue5-main branch]
2023-03-28 13:43:50 -04:00
const Zen::FZenServiceInstance& GetServiceInstance() const { return ZenService.GetInstance(); }
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
private:
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
void Initialize(const FZenCacheStoreParams& Params);
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
bool IsServiceReady();
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
static FCompositeBuffer SaveRpcPackage(const FCbPackage& Package);
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
void CreateRpcRequest(IRequestOwner& Owner, FHttpRequestQueue::FOnRequest&& OnRequest);
using FOnRpcComplete = TUniqueFunction<void(const THttpUniquePtr<IHttpResponse>& HttpResponse, const FCbPackage& Response)>;
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
void EnqueueAsyncRpc(IRequestOwner& Owner, FCbObject RequestObject, FOnRpcComplete&& OnComplete);
void EnqueueAsyncRpc(IRequestOwner& Owner, const FCbPackage& RequestPackage, FOnRpcComplete&& OnComplete);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
void ActivatePerformanceEvaluationThread();
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
void ConditionalEvaluatePerformance();
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
void ConditionalUpdateStorageSize();
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
void UpdateStatus();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
template <typename T, typename... ArgTypes>
static TRefCountPtr<T> MakeAsyncOp(ArgTypes&&... Args)
{
// TODO: This should in-place construct from a pre-allocated memory pool
return TRefCountPtr<T>(new T(Forward<ArgTypes>(Args)...));
}
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
private:
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
template <typename RequestType>
struct TRequestWithStats;
template <typename OutContainerType, typename InContainerType, typename BucketAccessorType, typename RequestTypeAccessorType>
static void StartRequests(
OutContainerType& Out,
const InContainerType& In,
BucketAccessorType BucketAccessor,
RequestTypeAccessorType TypeAccessor,
const ERequestOp Op);
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
class FPutOp;
class FGetOp;
class FPutValueOp;
class FGetValueOp;
class FGetChunksOp;
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
enum class EHealth
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
{
Unknown,
Ok,
Error,
};
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
using FOnHealthComplete = TUniqueFunction<void(const THttpUniquePtr<IHttpResponse>& HttpResponse, EHealth Health)>;
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
class FHealthReceiver;
class FAsyncHealthReceiver;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
class FCbPackageReceiver;
class FAsyncCbPackageReceiver;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
Adjusting logs related to zen usage to: - Remove repeated log related to fetching cache stats - Ensure that the Zen cache usage has log lines explaining the status of the caches it attempts to connect to (success or failure) #rb Matt.Peters [CL 30710842 by zousar shaker in ue5-main branch]
2024-01-18 22:43:41 -05:00
FString NodeName;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
FString Namespace;
UE::Zen::FScopeZenService ZenService;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
ICacheStoreOwner* StoreOwner = nullptr;
ICacheStoreStats* StoreStats = nullptr;
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
THttpUniquePtr<IHttpConnectionPool> ConnectionPool;
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
FHttpRequestQueue RequestQueue;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
bool bIsUsable = false;
Allow support for receiving local file references for cache items Companion PR in zen: https://github.com/EpicGames/zen/pull/184 #rb stefan.boberg #preflight 6368ce57d0174259cc1e7b7a [CL 23009037 by dan engelbrecht in ue5-main branch]
2022-11-07 05:33:21 -05:00
bool bIsLocalConnection = false;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
bool bTryEvaluatePerformance = false;
std::atomic<bool> bDeactivatedForPerformance = false;
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
int32 MaxBatchPutKB = 1024;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
int32 CacheRecordBatchSize = 8;
int32 CacheChunksBatchSize = 8;
FBackendDebugOptions DebugOptions;
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
TAnsiStringBuilder<256> RpcUri;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
std::atomic<int64> LastPerformanceEvaluationTicks;
TOptional<FThread> PerformanceEvaluationThread;
FManualResetEvent PerformanceEvaluationThreadShutdownEvent;
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
std::atomic<int64> LastStorageSizeUpdateTicks;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
float DeactivateAtMs = -1.0f;
ECacheStoreFlags OperationalFlags;
FRequestOwner PerformanceEvaluationRequestOwner;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
};
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
template <typename RequestType>
struct FZenCacheStore::TRequestWithStats
{
RequestType Request;
mutable FRequestStats Stats;
explicit TRequestWithStats(const RequestType& InRequest)
: Request(InRequest)
{
}
void EndRequest(FZenCacheStore& Outer, const EStatus Status) const
{
{
TUniqueLock Lock(Stats.Mutex);
Stats.EndTime = FMonotonicTimePoint::Now();
Stats.Status = Status;
}
if (Outer.StoreStats)
{
Outer.StoreStats->AddRequest(Stats);
}
}
};
template <typename OutContainerType, typename InContainerType, typename BucketAccessorType, typename RequestTypeAccessorType>
void FZenCacheStore::StartRequests(
OutContainerType& Out,
const InContainerType& In,
BucketAccessorType BucketAccessor,
RequestTypeAccessorType TypeAccessor,
const ERequestOp Op)
{
const FMonotonicTimePoint Now = FMonotonicTimePoint::Now();
Out.Reserve(In.Num());
for (const auto& Request : In)
{
auto& RequestWithStats = Out[Out.Emplace(Request)];
DDC: Added request name to FCacheStoreRequestStats #jira UE-133395 #rb Zousar.Shaker #rnx [CL 26210120 by devin doucette in ue5-main branch]
2023-06-23 10:51:52 -04:00
RequestWithStats.Stats.Name = Request.Name;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
RequestWithStats.Stats.Bucket = BucketAccessor(Request);
RequestWithStats.Stats.Type = TypeAccessor(Request);
RequestWithStats.Stats.Op = Op;
RequestWithStats.Stats.StartTime = Now;
}
}
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
class FZenCacheStore::FPutOp final : public FThreadSafeRefCountedObject
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
public:
FPutOp(FZenCacheStore& InCacheStore,
IRequestOwner& InOwner,
const TConstArrayView<FCachePutRequest> InRequests,
FOnCachePutComplete&& InOnComplete)
: CacheStore(InCacheStore)
, Owner(InOwner)
, OnComplete(MoveTemp(InOnComplete))
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
StartRequests(Requests, InRequests, [](const FCachePutRequest& Request) { return Request.Record.GetKey().Bucket; },
[](auto&) { return ERequestType::Record; }, ERequestOp::Put);
Batches = TBatchView<const TRequestWithStats<FCachePutRequest>>(Requests,
[this](const TRequestWithStats<FCachePutRequest>& NextRequest) { return BatchGroupingFilter(NextRequest.Request); });
TRACE_COUNTER_ADD(ZenDDC_Put, int64(Requests.Num()));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
virtual ~FPutOp()
{
FMonotonicTimeSpan AverageMainThreadTime = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.MainThreadTime.ToSeconds() / Requests.Num());
FMonotonicTimeSpan AverageOtherThreadTime = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.OtherThreadTime.ToSeconds() / Requests.Num());
FMonotonicTimeSpan AverageLatency = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.Latency.ToSeconds() / Requests.Num());
for (TRequestWithStats<FCachePutRequest>& Request : Requests)
{
Request.Stats.MainThreadTime = AverageMainThreadTime;
Request.Stats.OtherThreadTime = AverageOtherThreadTime;
Request.Stats.Latency = AverageLatency;
}
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
CacheStore.ConditionalEvaluatePerformance();
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
CacheStore.ConditionalUpdateStorageSize();
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
void IssueRequests()
{
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FRequestTimer RequestTimer(Requests[0].Stats);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FRequestBarrier Barrier(Owner);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (TArrayView<const TRequestWithStats<FCachePutRequest>> Batch : Batches)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
FCbPackage BatchPackage;
FCbWriter BatchWriter;
BatchWriter.BeginObject();
{
BatchWriter << ANSITEXTVIEW("Method") << "PutCacheRecords";
Add "Accept" field in Zen RPC request to enable new style RPC responses. See https://github.com/EpicGames/zen/pull/180 for Zen side change. #rb stefan.boberg zousar.shaker #preflight 6347c2568a0a7b2adcbe4d71 [CL 22505182 by dan engelbrecht in ue5-main branch]
2022-10-13 11:17:53 -04:00
BatchWriter.AddInteger(ANSITEXTVIEW("Accept"), Zen::Http::kCbPkgMagic);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchWriter.BeginObject(ANSITEXTVIEW("Params"));
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
ECachePolicy BatchDefaultPolicy = Batch[0].Request.Policy.GetRecordPolicy();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchWriter << ANSITEXTVIEW("DefaultPolicy") << *WriteToString<128>(BatchDefaultPolicy);
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
BatchWriter.AddString(ANSITEXTVIEW("Namespace"), CacheStore.Namespace);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchWriter.BeginArray(ANSITEXTVIEW("Requests"));
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (const TRequestWithStats<FCachePutRequest>& RequestWithStats : Batch)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const FCachePutRequest& Request = RequestWithStats.Request;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
const FCacheRecord& Record = Request.Record;
BatchWriter.BeginObject();
{
BatchWriter.SetName(ANSITEXTVIEW("Record"));
Record.Save(BatchPackage, BatchWriter);
Make sure record policy is set up correctly in batch requests to Zen #rb devin.doucette [CL 25982752 by dan engelbrecht in ue5-main branch]
2023-06-14 12:31:13 -04:00
if ((!Request.Policy.IsUniform()) || (Request.Policy.GetRecordPolicy() != BatchDefaultPolicy))
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Fixed deprecation warnings in the Zen cache store #jira UE-157180 #rb none #rnx #ROBOMERGE-AUTHOR: devin.doucette #ROBOMERGE-SOURCE: CL 20745126 via CL 20745729 #ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v970-20704180) [CL 20751392 by devin doucette in ue5-main branch]
2022-06-20 23:40:25 -04:00
BatchWriter << ANSITEXTVIEW("Policy") << Request.Policy;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
}
BatchWriter.EndObject();
}
BatchWriter.EndArray();
}
BatchWriter.EndObject();
}
BatchWriter.EndObject();
BatchPackage.SetObject(BatchWriter.Save().AsObject());
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
auto OnRpcComplete = [this, OpRef = TRefCountPtr<FPutOp>(this), Batch](const THttpUniquePtr<IHttpResponse>& HttpResponse, const FCbPackage& Response)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
const bool bCanceled = !HttpResponse || HttpResponse->GetErrorCode() == EHttpErrorCode::Canceled;
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FRequestTimer RequestTimer(Requests[0].Stats);
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
// Latency can't be measured for Put operations because it is intertwined with upload time.
Requests[0].Stats.Latency = FMonotonicTimeSpan::Infinity();
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
int32 RequestIndex = 0;
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (!bCanceled && HttpResponse->GetErrorCode() == EHttpErrorCode::None && HttpResponse->GetStatusCode() >= 200 && HttpResponse->GetStatusCode() <= 299)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
const FCbObject& ResponseObj = Response.GetObject();
for (FCbField ResponseField : ResponseObj[ANSITEXTVIEW("Result")])
{
if (RequestIndex >= Batch.Num())
{
++RequestIndex;
continue;
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const TRequestWithStats<FCachePutRequest>& RequestWithStats = Batch[RequestIndex++];
const FCacheKey& Key = RequestWithStats.Request.Record.GetKey();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
bool bPutSucceeded = ResponseField.AsBool();
if (CacheStore.DebugOptions.ShouldSimulatePutMiss(Key))
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Simulated miss for put of %s from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(Key), *RequestWithStats.Request.Name);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
bPutSucceeded = false;
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
bPutSucceeded ? OnHit(RequestWithStats) : OnMiss(RequestWithStats);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
if (RequestIndex != Batch.Num())
{
Downgrade messages from ZenCacheStore from Warning to Display as they can be expected to come up due to network conditions or temporary outages and do not break functionality. #rb devin.doucette [CL 28357691 by zousar shaker in ue5-main branch]
2023-09-29 17:36:33 -04:00
UE_LOG(LogDerivedDataCache, Display,
DDC: Added more detail to warnings from the Zen cache client #preflight 62d9ced6d76ea4b503d3f487 #rb Zousar.Shaker #rnx [CL 21214276 by Devin Doucette in ue5-main branch]
2022-07-21 23:01:51 -04:00
TEXT("%s: Invalid response received from PutCacheRecords RPC: %d results expected, received %d, from %s"),
*CacheStore.GetName(), Batch.Num(), RequestIndex, *WriteToString<256>(*HttpResponse));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
}
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
else if (!bCanceled && (HttpResponse->GetStatusCode() != 404))
Check status code for responses received from zenserver DDC requests. #rb stefan.boberg #preflight 645a3339013f3751584a39ed [CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
{
Downgrade messages from ZenCacheStore from Warning to Display as they can be expected to come up due to network conditions or temporary outages and do not break functionality. #rb devin.doucette [CL 28357691 by zousar shaker in ue5-main branch]
2023-09-29 17:36:33 -04:00
UE_LOG(LogDerivedDataCache, Display,
Check status code for responses received from zenserver DDC requests. #rb stefan.boberg #preflight 645a3339013f3751584a39ed [CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
TEXT("%s: Error response received from PutCacheRecords RPC: from %s"),
*CacheStore.GetName(), *WriteToString<256>(*HttpResponse));
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (const TRequestWithStats<FCachePutRequest>& RequestWithStats : Batch.RightChop(RequestIndex))
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (bCanceled)
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
{
OnCanceled(RequestWithStats);
}
else
{
OnMiss(RequestWithStats);
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
};
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
CacheStore.EnqueueAsyncRpc(Owner, BatchPackage, MoveTemp(OnRpcComplete));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
}
private:
EBatchView BatchGroupingFilter(const FCachePutRequest& NextRequest)
{
const FCacheRecord& Record = NextRequest.Record;
uint64 RecordSize = sizeof(FCacheKey) + Record.GetMeta().GetSize();
for (const FValueWithId& Value : Record.GetValues())
{
RecordSize += Value.GetData().GetCompressedSize();
}
BatchSize += RecordSize;
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
if (BatchSize > CacheStore.MaxBatchPutKB*1024)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
BatchSize = RecordSize;
return EBatchView::NewBatch;
}
return EBatchView::Continue;
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
void OnHit(const TRequestWithStats<FCachePutRequest>& RequestWithStats)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache put complete for %s from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Record.GetKey()), *RequestWithStats.Request.Name);
if (const FCbObject& Meta = RequestWithStats.Request.Record.GetMeta())
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
RequestWithStats.Stats.PhysicalWriteSize += Meta.GetSize();
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (const FValueWithId& Value : RequestWithStats.Request.Record.GetValues())
{
RequestWithStats.Stats.AddLogicalWrite(Value);
RequestWithStats.Stats.PhysicalWriteSize += Value.GetData().GetCompressedSize();
}
RequestWithStats.EndRequest(CacheStore, EStatus::Ok);
TRACE_COUNTER_ADD(ZenDDC_BytesSent, int64(RequestWithStats.Stats.PhysicalWriteSize));
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
TRACE_COUNTER_INCREMENT(ZenDDC_PutHit);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
OnComplete(RequestWithStats.Request.MakeResponse(EStatus::Ok));
}
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
void OnMiss(const TRequestWithStats<FCachePutRequest>& RequestWithStats)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache put failed for '%s' from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Record.GetKey()), *RequestWithStats.Request.Name);
RequestWithStats.EndRequest(CacheStore, EStatus::Error);
OnComplete(RequestWithStats.Request.MakeResponse(EStatus::Error));
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
void OnCanceled(const TRequestWithStats<FCachePutRequest>& RequestWithStats)
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache put failed with canceled request for '%s' from '%s'"),
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Record.GetKey()), *RequestWithStats.Request.Name);
RequestWithStats.EndRequest(CacheStore, EStatus::Canceled);
OnComplete(RequestWithStats.Request.MakeResponse(EStatus::Canceled));
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FZenCacheStore& CacheStore;
IRequestOwner& Owner;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TArray<TRequestWithStats<FCachePutRequest>, TInlineAllocator<1>> Requests;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
uint64 BatchSize = 0;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TBatchView<const TRequestWithStats<FCachePutRequest>> Batches;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FOnCachePutComplete OnComplete;
};
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
class FZenCacheStore::FGetOp final : public FThreadSafeRefCountedObject
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
public:
FGetOp(FZenCacheStore& InCacheStore,
IRequestOwner& InOwner,
const TConstArrayView<FCacheGetRequest> InRequests,
FOnCacheGetComplete&& InOnComplete)
: CacheStore(InCacheStore)
, Owner(InOwner)
, OnComplete(MoveTemp(InOnComplete))
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
StartRequests(Requests, InRequests, [](const FCacheGetRequest& Request) { return Request.Key.Bucket; },
[](auto&) { return ERequestType::Record; }, ERequestOp::Get);
TRACE_COUNTER_ADD(ZenDDC_Get, int64(Requests.Num()));
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
TRACE_COUNTER_ADD(ZenDDC_CacheRecordRequestCountInFlight, int64(Requests.Num()));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
virtual ~FGetOp()
{
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
TRACE_COUNTER_SUBTRACT(ZenDDC_CacheRecordRequestCountInFlight, int64(Requests.Num()));
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FMonotonicTimeSpan AverageMainThreadTime = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.MainThreadTime.ToSeconds() / Requests.Num());
FMonotonicTimeSpan AverageOtherThreadTime = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.OtherThreadTime.ToSeconds() / Requests.Num());
FMonotonicTimeSpan AverageLatency = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.Latency.ToSeconds() / Requests.Num());
for (TRequestWithStats<FCacheGetRequest>& Request : Requests)
{
Request.Stats.MainThreadTime = AverageMainThreadTime;
Request.Stats.OtherThreadTime = AverageOtherThreadTime;
Request.Stats.Latency = AverageLatency;
}
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
CacheStore.ConditionalEvaluatePerformance();
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
CacheStore.ConditionalUpdateStorageSize();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
void IssueRequests()
{
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FRequestTimer RequestTimer(Requests[0].Stats);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FRequestBarrier Barrier(Owner);
ForEachBatch(CacheStore.CacheRecordBatchSize, Requests.Num(),
[this](int32 BatchFirst, int32 BatchLast)
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
{
TConstArrayView<TRequestWithStats<FCacheGetRequest>> Batch(Requests.GetData() + BatchFirst, BatchLast - BatchFirst + 1);
FCbWriter BatchRequest;
BatchRequest.BeginObject();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchRequest << ANSITEXTVIEW("Method") << ANSITEXTVIEW("GetCacheRecords");
BatchRequest.AddInteger(ANSITEXTVIEW("Accept"), Zen::Http::kCbPkgMagic);
if (CacheStore.bIsLocalConnection)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchRequest.AddInteger(ANSITEXTVIEW("AcceptFlags"), static_cast<uint32_t>(Zen::Http::RpcAcceptOptions::kAllowLocalReferences));
BatchRequest.AddInteger(ANSITEXTVIEW("Pid"), FPlatformProcess::GetCurrentProcessId());
}
Add "Accept" field in Zen RPC request to enable new style RPC responses. See https://github.com/EpicGames/zen/pull/180 for Zen side change. #rb stefan.boberg zousar.shaker #preflight 6347c2568a0a7b2adcbe4d71 [CL 22505182 by dan engelbrecht in ue5-main branch]
2022-10-13 11:17:53 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchRequest.BeginObject(ANSITEXTVIEW("Params"));
{
ECachePolicy BatchDefaultPolicy = Batch[0].Request.Policy.GetRecordPolicy();
BatchRequest << ANSITEXTVIEW("DefaultPolicy") << *WriteToString<128>(BatchDefaultPolicy);
BatchRequest.AddString(ANSITEXTVIEW("Namespace"), CacheStore.Namespace);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchRequest.BeginArray(ANSITEXTVIEW("Requests"));
for (const TRequestWithStats<FCacheGetRequest>& RequestWithStats : Batch)
{
BatchRequest.BeginObject();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchRequest << ANSITEXTVIEW("Key") << RequestWithStats.Request.Key;
Make sure record policy is set up correctly in batch requests to Zen #rb devin.doucette [CL 25982752 by dan engelbrecht in ue5-main branch]
2023-06-14 12:31:13 -04:00
if ((!RequestWithStats.Request.Policy.IsUniform()) || (RequestWithStats.Request.Policy.GetRecordPolicy() != BatchDefaultPolicy))
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchRequest << ANSITEXTVIEW("Policy") << RequestWithStats.Request.Policy;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchRequest.EndObject();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchRequest.EndArray();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
BatchRequest.EndObject();
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
}
BatchRequest.EndObject();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
FGetOp* OriginalOp = this;
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
auto OnRpcComplete = [this, OpRef = TRefCountPtr<FGetOp>(OriginalOp), Batch](const THttpUniquePtr<IHttpResponse>& HttpResponse, const FCbPackage& Response)
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
const bool bCanceled = !HttpResponse || HttpResponse->GetErrorCode() == EHttpErrorCode::Canceled;
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FRequestTimer RequestTimer(Requests[0].Stats);
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (HttpResponse)
{
const FHttpResponseStats& ResponseStats = HttpResponse->GetStats();
Requests[0].Stats.Latency = FMonotonicTimeSpan::FromSeconds(ResponseStats.GetLatency());
}
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
int32 RequestIndex = 0;
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (!bCanceled && HttpResponse->GetErrorCode() == EHttpErrorCode::None && HttpResponse->GetStatusCode() >= 200 && HttpResponse->GetStatusCode() <= 299)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const FCbObject& ResponseObj = Response.GetObject();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (FCbField RecordField : ResponseObj[ANSITEXTVIEW("Result")])
{
if (RequestIndex >= Batch.Num())
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
++RequestIndex;
continue;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const TRequestWithStats<FCacheGetRequest>& RequestWithStats = Batch[RequestIndex++];
const FCacheKey& Key = RequestWithStats.Request.Key;
FOptionalCacheRecord Record;
if (CacheStore.DebugOptions.ShouldSimulateGetMiss(Key))
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Simulated miss for put of '%s' from '%s'"),
*CacheStore.GetName(), *WriteToString<96>(Key), *RequestWithStats.Request.Name);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
else if (!RecordField.IsNull())
{
Record = FCacheRecord::Load(Response, RecordField.AsObject());
}
Record ? OnHit(RequestWithStats, MoveTemp(Record).Get()) : OnMiss(RequestWithStats);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
if (RequestIndex != Batch.Num())
Check status code for responses received from zenserver DDC requests. #rb stefan.boberg #preflight 645a3339013f3751584a39ed [CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
{
Downgrade messages from ZenCacheStore from Warning to Display as they can be expected to come up due to network conditions or temporary outages and do not break functionality. #rb devin.doucette [CL 28357691 by zousar shaker in ue5-main branch]
2023-09-29 17:36:33 -04:00
UE_LOG(LogDerivedDataCache, Display,
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TEXT("%s: Invalid response received from GetCacheRecords RPC: %d results expected, received %d, from %s"),
*CacheStore.GetName(), Batch.Num(), RequestIndex, *WriteToString<256>(*HttpResponse));
Check status code for responses received from zenserver DDC requests. #rb stefan.boberg #preflight 645a3339013f3751584a39ed [CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
}
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
else if (!bCanceled && (HttpResponse->GetStatusCode() != 404))
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
{
Downgrade messages from ZenCacheStore from Warning to Display as they can be expected to come up due to network conditions or temporary outages and do not break functionality. #rb devin.doucette [CL 28357691 by zousar shaker in ue5-main branch]
2023-09-29 17:36:33 -04:00
UE_LOG(LogDerivedDataCache, Display,
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TEXT("%s: Error response received from GetCacheRecords RPC: from %s"),
*CacheStore.GetName(), *WriteToString<256>(*HttpResponse));
}
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (const TRequestWithStats<FCacheGetRequest>& RequestWithStats : Batch.RightChop(RequestIndex))
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (bCanceled)
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
{
OnCanceled(RequestWithStats);
}
else
{
OnMiss(RequestWithStats);
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
}
};
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
CacheStore.EnqueueAsyncRpc(Owner, BatchRequest.Save().AsObject(), MoveTemp(OnRpcComplete));
});
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
private:
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
void OnHit(const TRequestWithStats<FCacheGetRequest>& RequestWithStats, FCacheRecord&& Record)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache hit for '%s' from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
Return error if we receive a partial record from zenserver #rb devin.doucette #prefligth 648a3d2f88385dd88cfdda4a [CL 26006460 by dan engelbrecht in ue5-main branch]
2023-06-15 04:54:40 -04:00
bool bInComplete = false;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
if (const FCbObject& Meta = Record.GetMeta())
{
RequestWithStats.Stats.PhysicalReadSize += Meta.GetSize();
}
for (const FValueWithId& Value : Record.GetValues())
{
RequestWithStats.Stats.AddLogicalRead(Value);
RequestWithStats.Stats.PhysicalReadSize += Value.GetData().GetCompressedSize();
Return error if we receive a partial record from zenserver #rb devin.doucette #prefligth 648a3d2f88385dd88cfdda4a [CL 26006460 by dan engelbrecht in ue5-main branch]
2023-06-15 04:54:40 -04:00
ECachePolicy ValuePolicy = RequestWithStats.Request.Policy.GetValuePolicy(Value.GetId());
if (EnumHasAnyFlags(ValuePolicy, ECachePolicy::SkipData))
{
continue;
}
if (Value.HasData())
{
continue;
}
if (EnumHasAnyFlags(ValuePolicy, ECachePolicy::Query))
{
bInComplete = true;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
}
Return error if we receive a partial record from zenserver #rb devin.doucette #prefligth 648a3d2f88385dd88cfdda4a [CL 26006460 by dan engelbrecht in ue5-main branch]
2023-06-15 04:54:40 -04:00
}
EStatus Status = bInComplete ? EStatus::Error : EStatus::Ok;
RequestWithStats.EndRequest(CacheStore, Status);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
TRACE_COUNTER_INCREMENT(ZenDDC_GetHit);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TRACE_COUNTER_ADD(ZenDDC_BytesReceived, int64(RequestWithStats.Stats.PhysicalReadSize));
Return error if we receive a partial record from zenserver #rb devin.doucette #prefligth 648a3d2f88385dd88cfdda4a [CL 26006460 by dan engelbrecht in ue5-main branch]
2023-06-15 04:54:40 -04:00
OnComplete({RequestWithStats.Request.Name, MoveTemp(Record), RequestWithStats.Request.UserData, Status});
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
void OnMiss(const TRequestWithStats<FCacheGetRequest>& RequestWithStats)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache miss for '%s' from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
RequestWithStats.EndRequest(CacheStore, EStatus::Error);
OnComplete(RequestWithStats.Request.MakeResponse(EStatus::Error));
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
void OnCanceled(const TRequestWithStats<FCacheGetRequest>& RequestWithStats)
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache miss with canceled request for '%s' from '%s'"),
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
RequestWithStats.EndRequest(CacheStore, EStatus::Canceled);
OnComplete(RequestWithStats.Request.MakeResponse(EStatus::Canceled));
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FZenCacheStore& CacheStore;
IRequestOwner& Owner;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TArray<TRequestWithStats<FCacheGetRequest>, TInlineAllocator<1>> Requests;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FOnCacheGetComplete OnComplete;
};
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
class FZenCacheStore::FPutValueOp final : public FThreadSafeRefCountedObject
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
public:
FPutValueOp(FZenCacheStore& InCacheStore,
IRequestOwner& InOwner,
const TConstArrayView<FCachePutValueRequest> InRequests,
FOnCachePutValueComplete&& InOnComplete)
: CacheStore(InCacheStore)
, Owner(InOwner)
, OnComplete(MoveTemp(InOnComplete))
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
StartRequests(Requests, InRequests, [](const FCachePutValueRequest& Request) { return Request.Key.Bucket; },
[](auto&) { return ERequestType::Value; }, ERequestOp::Put);
TRACE_COUNTER_ADD(ZenDDC_Put, int64(Requests.Num()));
Batches = TBatchView<const TRequestWithStats<FCachePutValueRequest>>(Requests,
[this](const TRequestWithStats<FCachePutValueRequest>& NextRequest) { return BatchGroupingFilter(NextRequest.Request); });
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
virtual ~FPutValueOp()
{
FMonotonicTimeSpan AverageMainThreadTime = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.MainThreadTime.ToSeconds() / Requests.Num());
FMonotonicTimeSpan AverageOtherThreadTime = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.OtherThreadTime.ToSeconds() / Requests.Num());
FMonotonicTimeSpan AverageLatency = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.Latency.ToSeconds() / Requests.Num());
for (TRequestWithStats<FCachePutValueRequest>& Request : Requests)
{
Request.Stats.MainThreadTime = AverageMainThreadTime;
Request.Stats.OtherThreadTime = AverageOtherThreadTime;
Request.Stats.Latency = AverageLatency;
}
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
CacheStore.ConditionalEvaluatePerformance();
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
CacheStore.ConditionalUpdateStorageSize();
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
void IssueRequests()
{
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FRequestTimer RequestTimer(Requests[0].Stats);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FRequestBarrier Barrier(Owner);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (TArrayView<const TRequestWithStats<FCachePutValueRequest>> Batch : Batches)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
FCbPackage BatchPackage;
FCbWriter BatchWriter;
BatchWriter.BeginObject();
{
BatchWriter << ANSITEXTVIEW("Method") << ANSITEXTVIEW("PutCacheValues");
Add "Accept" field in Zen RPC request to enable new style RPC responses. See https://github.com/EpicGames/zen/pull/180 for Zen side change. #rb stefan.boberg zousar.shaker #preflight 6347c2568a0a7b2adcbe4d71 [CL 22505182 by dan engelbrecht in ue5-main branch]
2022-10-13 11:17:53 -04:00
BatchWriter.AddInteger(ANSITEXTVIEW("Accept"), Zen::Http::kCbPkgMagic);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchWriter.BeginObject(ANSITEXTVIEW("Params"));
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
ECachePolicy BatchDefaultPolicy = Batch[0].Request.Policy;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchWriter << ANSITEXTVIEW("DefaultPolicy") << *WriteToString<128>(BatchDefaultPolicy);
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
BatchWriter.AddString(ANSITEXTVIEW("Namespace"), CacheStore.Namespace);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchWriter.BeginArray("Requests");
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (const TRequestWithStats<FCachePutValueRequest>& RequestWithStats : Batch)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
BatchWriter.BeginObject();
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchWriter << ANSITEXTVIEW("Key") << RequestWithStats.Request.Key;
const FValue& Value = RequestWithStats.Request.Value;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchWriter.AddBinaryAttachment("RawHash", Value.GetRawHash());
if (Value.HasData())
{
BatchPackage.AddAttachment(FCbAttachment(Value.GetData()));
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
if (RequestWithStats.Request.Policy != BatchDefaultPolicy)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchWriter << ANSITEXTVIEW("Policy") << WriteToString<128>(RequestWithStats.Request.Policy);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
}
BatchWriter.EndObject();
}
BatchWriter.EndArray();
}
BatchWriter.EndObject();
}
BatchWriter.EndObject();
BatchPackage.SetObject(BatchWriter.Save().AsObject());
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
auto OnRpcComplete = [this, OpRef = TRefCountPtr<FPutValueOp>(this), Batch](const THttpUniquePtr<IHttpResponse>& HttpResponse, const FCbPackage& Response)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
const bool bCanceled = !HttpResponse || HttpResponse->GetErrorCode() == EHttpErrorCode::Canceled;
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FRequestTimer RequestTimer(Requests[0].Stats);
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
// Latency can't be measured for Put operations because it is intertwined with upload time.
Requests[0].Stats.Latency = FMonotonicTimeSpan::Infinity();
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
int32 RequestIndex = 0;
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (!bCanceled && HttpResponse->GetErrorCode() == EHttpErrorCode::None && HttpResponse->GetStatusCode() >= 200 && HttpResponse->GetStatusCode() <= 299)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
const FCbObject& ResponseObj = Response.GetObject();
for (FCbField ResponseField : ResponseObj[ANSITEXTVIEW("Result")])
{
if (RequestIndex >= Batch.Num())
{
++RequestIndex;
continue;
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const TRequestWithStats<FCachePutValueRequest>& RequestWithStats = Batch[RequestIndex++];
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
bool bPutSucceeded = ResponseField.AsBool();
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
if (CacheStore.DebugOptions.ShouldSimulatePutMiss(RequestWithStats.Request.Key))
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Simulated miss for PutValue of %s from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
bPutSucceeded = false;
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
bPutSucceeded ? OnHit(RequestWithStats) : OnMiss(RequestWithStats);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
if (RequestIndex != Batch.Num())
{
Downgrade messages from ZenCacheStore from Warning to Display as they can be expected to come up due to network conditions or temporary outages and do not break functionality. #rb devin.doucette [CL 28357691 by zousar shaker in ue5-main branch]
2023-09-29 17:36:33 -04:00
UE_LOG(LogDerivedDataCache, Display,
DDC: Added more detail to warnings from the Zen cache client #preflight 62d9ced6d76ea4b503d3f487 #rb Zousar.Shaker #rnx [CL 21214276 by Devin Doucette in ue5-main branch]
2022-07-21 23:01:51 -04:00
TEXT("%s: Invalid response received from PutCacheValues RPC: %d results expected, received %d, from %s"),
*CacheStore.GetName(), Batch.Num(), RequestIndex, *WriteToString<256>(*HttpResponse));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
}
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
else if (!bCanceled && (HttpResponse->GetStatusCode() != 404))
Check status code for responses received from zenserver DDC requests. #rb stefan.boberg #preflight 645a3339013f3751584a39ed [CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
{
Downgrade messages from ZenCacheStore from Warning to Display as they can be expected to come up due to network conditions or temporary outages and do not break functionality. #rb devin.doucette [CL 28357691 by zousar shaker in ue5-main branch]
2023-09-29 17:36:33 -04:00
UE_LOG(LogDerivedDataCache, Display,
Check status code for responses received from zenserver DDC requests. #rb stefan.boberg #preflight 645a3339013f3751584a39ed [CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
TEXT("%s: Error response received from PutCacheValues RPC: from %s"),
*CacheStore.GetName(), *WriteToString<256>(*HttpResponse));
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (const TRequestWithStats<FCachePutValueRequest>& RequestWithStats : Batch.RightChop(RequestIndex))
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (bCanceled)
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
{
OnCanceled(RequestWithStats);
}
else
{
OnMiss(RequestWithStats);
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
};
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
CacheStore.EnqueueAsyncRpc(Owner, BatchPackage, MoveTemp(OnRpcComplete));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
}
private:
EBatchView BatchGroupingFilter(const FCachePutValueRequest& NextRequest)
{
uint64 ValueSize = sizeof(FCacheKey) + NextRequest.Value.GetData().GetCompressedSize();
BatchSize += ValueSize;
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
if (BatchSize > CacheStore.MaxBatchPutKB*1024)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
BatchSize = ValueSize;
return EBatchView::NewBatch;
}
return EBatchView::Continue;
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
void OnHit(const TRequestWithStats<FCachePutValueRequest>& RequestWithStats)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache put complete for %s from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
RequestWithStats.Stats.AddLogicalWrite(RequestWithStats.Request.Value);
RequestWithStats.Stats.PhysicalWriteSize += RequestWithStats.Request.Value.GetData().GetCompressedSize();
RequestWithStats.EndRequest(CacheStore, EStatus::Ok);
TRACE_COUNTER_INCREMENT(ZenDDC_PutHit);
TRACE_COUNTER_ADD(ZenDDC_BytesSent, RequestWithStats.Stats.PhysicalWriteSize);
OnComplete(RequestWithStats.Request.MakeResponse(EStatus::Ok));
}
void OnMiss(const TRequestWithStats<FCachePutValueRequest>& RequestWithStats)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache put failed for '%s' from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
RequestWithStats.EndRequest(CacheStore, EStatus::Error);
OnComplete(RequestWithStats.Request.MakeResponse(EStatus::Error));
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
void OnCanceled(const TRequestWithStats<FCachePutValueRequest>& RequestWithStats)
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache put failed with canceled request for '%s' from '%s'"),
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
RequestWithStats.EndRequest(CacheStore, EStatus::Canceled);
OnComplete(RequestWithStats.Request.MakeResponse(EStatus::Canceled));
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FZenCacheStore& CacheStore;
IRequestOwner& Owner;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TArray<TRequestWithStats<FCachePutValueRequest>, TInlineAllocator<1>> Requests;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
uint64 BatchSize = 0;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TBatchView<const TRequestWithStats<FCachePutValueRequest>> Batches;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FOnCachePutValueComplete OnComplete;
};
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
class FZenCacheStore::FGetValueOp final : public FThreadSafeRefCountedObject
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
public:
FGetValueOp(FZenCacheStore& InCacheStore,
IRequestOwner& InOwner,
const TConstArrayView<FCacheGetValueRequest> InRequests,
FOnCacheGetValueComplete&& InOnComplete)
: CacheStore(InCacheStore)
, Owner(InOwner)
, OnComplete(MoveTemp(InOnComplete))
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
StartRequests(Requests, InRequests, [](const FCacheGetValueRequest& Request) { return Request.Key.Bucket; },
[](auto&) { return ERequestType::Value; }, ERequestOp::Get);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
TRACE_COUNTER_ADD(ZenDDC_Get, (int64)Requests.Num());
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
TRACE_COUNTER_ADD(ZenDDC_CacheRecordRequestCountInFlight, int64(Requests.Num()));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
virtual ~FGetValueOp()
{
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
TRACE_COUNTER_SUBTRACT(ZenDDC_CacheRecordRequestCountInFlight, int64(Requests.Num()));
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FMonotonicTimeSpan AverageMainThreadTime = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.MainThreadTime.ToSeconds() / Requests.Num());
FMonotonicTimeSpan AverageOtherThreadTime = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.OtherThreadTime.ToSeconds() / Requests.Num());
FMonotonicTimeSpan AverageLatency = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.Latency.ToSeconds() / Requests.Num());
for (TRequestWithStats<FCacheGetValueRequest>& Request : Requests)
{
Request.Stats.MainThreadTime = AverageMainThreadTime;
Request.Stats.OtherThreadTime = AverageOtherThreadTime;
Request.Stats.Latency = AverageLatency;
}
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
CacheStore.ConditionalEvaluatePerformance();
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
CacheStore.ConditionalUpdateStorageSize();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
void IssueRequests()
{
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FRequestTimer RequestTimer(Requests[0].Stats);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FRequestBarrier Barrier(Owner);
ForEachBatch(CacheStore.CacheRecordBatchSize, Requests.Num(),
[this](int32 BatchFirst, int32 BatchLast)
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TConstArrayView<TRequestWithStats<FCacheGetValueRequest>> Batch(Requests.GetData() + BatchFirst, BatchLast - BatchFirst + 1);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FCbWriter BatchRequest;
BatchRequest.BeginObject();
{
BatchRequest << ANSITEXTVIEW("Method") << ANSITEXTVIEW("GetCacheValues");
Add "Accept" field in Zen RPC request to enable new style RPC responses. See https://github.com/EpicGames/zen/pull/180 for Zen side change. #rb stefan.boberg zousar.shaker #preflight 6347c2568a0a7b2adcbe4d71 [CL 22505182 by dan engelbrecht in ue5-main branch]
2022-10-13 11:17:53 -04:00
BatchRequest.AddInteger(ANSITEXTVIEW("Accept"), Zen::Http::kCbPkgMagic);
Allow support for receiving local file references for cache items Companion PR in zen: https://github.com/EpicGames/zen/pull/184 #rb stefan.boberg #preflight 6368ce57d0174259cc1e7b7a [CL 23009037 by dan engelbrecht in ue5-main branch]
2022-11-07 05:33:21 -05:00
if (CacheStore.bIsLocalConnection)
{
BatchRequest.AddInteger(ANSITEXTVIEW("AcceptFlags"), static_cast<uint32_t>(Zen::Http::RpcAcceptOptions::kAllowLocalReferences));
Accept duplicated file handles in Zen RPC responses #preflight 641c11bf76461c460b8eabb4 #rb stefan.boberg per.larsson [CL 24759274 by dan engelbrecht in ue5-main branch]
2023-03-23 05:15:22 -04:00
BatchRequest.AddInteger(ANSITEXTVIEW("Pid"), FPlatformProcess::GetCurrentProcessId());
Allow support for receiving local file references for cache items Companion PR in zen: https://github.com/EpicGames/zen/pull/184 #rb stefan.boberg #preflight 6368ce57d0174259cc1e7b7a [CL 23009037 by dan engelbrecht in ue5-main branch]
2022-11-07 05:33:21 -05:00
}
Add "Accept" field in Zen RPC request to enable new style RPC responses. See https://github.com/EpicGames/zen/pull/180 for Zen side change. #rb stefan.boberg zousar.shaker #preflight 6347c2568a0a7b2adcbe4d71 [CL 22505182 by dan engelbrecht in ue5-main branch]
2022-10-13 11:17:53 -04:00
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchRequest.BeginObject(ANSITEXTVIEW("Params"));
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
ECachePolicy BatchDefaultPolicy = Batch[0].Request.Policy;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchRequest << ANSITEXTVIEW("DefaultPolicy") << *WriteToString<128>(BatchDefaultPolicy);
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
BatchRequest.AddString(ANSITEXTVIEW("Namespace"), CacheStore.Namespace);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchRequest.BeginArray("Requests");
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (const TRequestWithStats<FCacheGetValueRequest>& RequestWithStats : Batch)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
BatchRequest.BeginObject();
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchRequest << ANSITEXTVIEW("Key") << RequestWithStats.Request.Key;
if (RequestWithStats.Request.Policy != BatchDefaultPolicy)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
BatchRequest << ANSITEXTVIEW("Policy") << WriteToString<128>(RequestWithStats.Request.Policy);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
}
BatchRequest.EndObject();
}
BatchRequest.EndArray();
}
BatchRequest.EndObject();
}
BatchRequest.EndObject();
FGetValueOp* OriginalOp = this;
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
auto OnRpcComplete = [this, OpRef = TRefCountPtr<FGetValueOp>(OriginalOp), Batch](const THttpUniquePtr<IHttpResponse>& HttpResponse, const FCbPackage& Response)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
const bool bCanceled = !HttpResponse || HttpResponse->GetErrorCode() == EHttpErrorCode::Canceled;
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FRequestTimer RequestTimer(Requests[0].Stats);
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (HttpResponse)
{
const FHttpResponseStats& ResponseStats = HttpResponse->GetStats();
Requests[0].Stats.Latency = FMonotonicTimeSpan::FromSeconds(ResponseStats.GetLatency());
}
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
int32 RequestIndex = 0;
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (!bCanceled && HttpResponse->GetErrorCode() == EHttpErrorCode::None && HttpResponse->GetStatusCode() >= 200 && HttpResponse->GetStatusCode() <= 299)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
const FCbObject& ResponseObj = Response.GetObject();
for (FCbFieldView ResultField : ResponseObj[ANSITEXTVIEW("Result")])
{
if (RequestIndex >= Batch.Num())
{
++RequestIndex;
continue;
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const TRequestWithStats<FCacheGetValueRequest>& RequestWithStats = Batch[RequestIndex++];
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FCbObjectView ResultObj = ResultField.AsObjectView();
TOptional<FValue> Value;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
if (CacheStore.DebugOptions.ShouldSimulateGetMiss(RequestWithStats.Request.Key))
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Simulated miss for GetValue of '%s' from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
else
{
FCbFieldView RawHashField = ResultObj["RawHash"];
FIoHash RawHash = RawHashField.AsHash();
Fix bug in Zen DDC client. Don't assume that if the data hash of a response is available as an attachment that it should be included in the reponse. If the request is skipping data, then even if the attachment is present (eg: from another request in the batch), it should not be included when processing the request that had SkipData. #rb dan.engelbrecht [CL 28187457 by zousar shaker in ue5-main branch]
2023-09-25 10:31:36 -04:00
const FCbAttachment* Attachment = EnumHasAnyFlags(RequestWithStats.Request.Policy, ECachePolicy::SkipData) ? nullptr : Response.FindAttachment(RawHash);
if (Attachment)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Value.Emplace(Attachment->AsCompressedBinary());
}
else
{
FCbFieldView RawSizeField = ResultObj["RawSize"];
uint64 RawSize = RawSizeField.AsUInt64();
if (!RawSizeField.HasError() && !RawHashField.HasError())
{
Value.Emplace(RawHash, RawSize);
}
}
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
(bool)Value ? OnHit(RequestWithStats, MoveTemp(*Value)) : OnMiss(RequestWithStats);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
if (RequestIndex != Batch.Num())
{
Downgrade messages from ZenCacheStore from Warning to Display as they can be expected to come up due to network conditions or temporary outages and do not break functionality. #rb devin.doucette [CL 28357691 by zousar shaker in ue5-main branch]
2023-09-29 17:36:33 -04:00
UE_LOG(LogDerivedDataCache, Display,
DDC: Added more detail to warnings from the Zen cache client #preflight 62d9ced6d76ea4b503d3f487 #rb Zousar.Shaker #rnx [CL 21214276 by Devin Doucette in ue5-main branch]
2022-07-21 23:01:51 -04:00
TEXT("%s: Invalid response received from GetCacheValues RPC: %d results expected, received %d from %s"),
*CacheStore.GetName(), Batch.Num(), RequestIndex, *WriteToString<256>(*HttpResponse));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
}
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
else if (!bCanceled && (HttpResponse->GetStatusCode() != 404))
Check status code for responses received from zenserver DDC requests. #rb stefan.boberg #preflight 645a3339013f3751584a39ed [CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
{
Downgrade messages from ZenCacheStore from Warning to Display as they can be expected to come up due to network conditions or temporary outages and do not break functionality. #rb devin.doucette [CL 28357691 by zousar shaker in ue5-main branch]
2023-09-29 17:36:33 -04:00
UE_LOG(LogDerivedDataCache, Display,
Check status code for responses received from zenserver DDC requests. #rb stefan.boberg #preflight 645a3339013f3751584a39ed [CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
TEXT("%s: Error response received from GetCacheValues RPC: from %s"),
*CacheStore.GetName(), *WriteToString<256>(*HttpResponse));
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (const TRequestWithStats<FCacheGetValueRequest>& RequestWithStats : Batch.RightChop(RequestIndex))
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (bCanceled)
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
{
OnCanceled(RequestWithStats);
}
else
{
OnMiss(RequestWithStats);
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
};
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
CacheStore.EnqueueAsyncRpc(Owner, BatchRequest.Save().AsObject(), MoveTemp(OnRpcComplete));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
});
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
private:
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
void OnHit(const TRequestWithStats<FCacheGetValueRequest>& RequestWithStats, FValue&& Value)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache hit for '%s' from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
RequestWithStats.Stats.AddLogicalRead(Value);
RequestWithStats.Stats.PhysicalReadSize += Value.GetData().GetCompressedSize();
RequestWithStats.EndRequest(CacheStore, EStatus::Ok);
TRACE_COUNTER_INCREMENT(ZenDDC_GetHit);
TRACE_COUNTER_ADD(ZenDDC_BytesReceived, int64(RequestWithStats.Stats.PhysicalReadSize));
OnComplete({RequestWithStats.Request.Name, RequestWithStats.Request.Key, MoveTemp(Value), RequestWithStats.Request.UserData, EStatus::Ok});
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
};
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
void OnMiss(const TRequestWithStats<FCacheGetValueRequest>& RequestWithStats)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache miss for '%s' from '%s'"),
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
RequestWithStats.EndRequest(CacheStore, EStatus::Error);
OnComplete(RequestWithStats.Request.MakeResponse(EStatus::Error));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
};
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
void OnCanceled(const TRequestWithStats<FCacheGetValueRequest>& RequestWithStats)
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache miss with canceled request for '%s' from '%s'"),
*CacheStore.GetName(), *WriteToString<96>(RequestWithStats.Request.Key), *RequestWithStats.Request.Name);
RequestWithStats.EndRequest(CacheStore, EStatus::Canceled);
OnComplete(RequestWithStats.Request.MakeResponse(EStatus::Canceled));
};
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FZenCacheStore& CacheStore;
IRequestOwner& Owner;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TArray<TRequestWithStats<FCacheGetValueRequest>, TInlineAllocator<1>> Requests;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FOnCacheGetValueComplete OnComplete;
};
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
class FZenCacheStore::FGetChunksOp final : public FThreadSafeRefCountedObject
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
public:
FGetChunksOp(FZenCacheStore& InCacheStore,
IRequestOwner& InOwner,
const TConstArrayView<FCacheGetChunkRequest> InRequests,
FOnCacheGetChunkComplete&& InOnComplete)
: CacheStore(InCacheStore)
, Owner(InOwner)
, OnComplete(MoveTemp(InOnComplete))
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
StartRequests(Requests, InRequests, [](const FCacheGetChunkRequest& Request) { return Request.Key.Bucket; },
[](const FCacheGetChunkRequest& Request) { return Request.Id.IsNull() ? ERequestType::Value : ERequestType::Record; },
ERequestOp::GetChunk);
Algo::StableSortBy(Requests, &TRequestWithStats<FCacheGetChunkRequest>::Request, TChunkLess());
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
TRACE_COUNTER_ADD(ZenDDC_Get, int64(Requests.Num()));
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TRACE_COUNTER_ADD(ZenDDC_ChunkRequestCountInFlight, int64(Requests.Num()));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
virtual ~FGetChunksOp()
{
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
TRACE_COUNTER_SUBTRACT(ZenDDC_ChunkRequestCountInFlight, int64(Requests.Num()));
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FMonotonicTimeSpan AverageMainThreadTime = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.MainThreadTime.ToSeconds() / Requests.Num());
FMonotonicTimeSpan AverageOtherThreadTime = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.OtherThreadTime.ToSeconds() / Requests.Num());
FMonotonicTimeSpan AverageLatency = FMonotonicTimeSpan::FromSeconds(Requests[0].Stats.Latency.ToSeconds() / Requests.Num());
for (TRequestWithStats<FCacheGetChunkRequest>& Request : Requests)
{
Request.Stats.MainThreadTime = AverageMainThreadTime;
Request.Stats.OtherThreadTime = AverageOtherThreadTime;
Request.Stats.Latency = AverageLatency;
}
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
CacheStore.ConditionalEvaluatePerformance();
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
CacheStore.ConditionalUpdateStorageSize();
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
void IssueRequests()
{
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FRequestTimer RequestTimer(Requests[0].Stats);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FRequestBarrier Barrier(Owner);
ForEachBatch(CacheStore.CacheChunksBatchSize, Requests.Num(),
[this](int32 BatchFirst, int32 BatchLast)
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TConstArrayView<TRequestWithStats<FCacheGetChunkRequest>> Batch(Requests.GetData() + BatchFirst, BatchLast - BatchFirst + 1);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FCbWriter BatchRequest;
BatchRequest.BeginObject();
{
BatchRequest << ANSITEXTVIEW("Method") << "GetCacheChunks";
Add "Accept" field in Zen RPC request to enable new style RPC responses. See https://github.com/EpicGames/zen/pull/180 for Zen side change. #rb stefan.boberg zousar.shaker #preflight 6347c2568a0a7b2adcbe4d71 [CL 22505182 by dan engelbrecht in ue5-main branch]
2022-10-13 11:17:53 -04:00
BatchRequest.AddInteger(ANSITEXTVIEW("Accept"), Zen::Http::kCbPkgMagic);
Add support of partial zen DDC GetChunks requests #ushell-cherrypick of 32395474 by dan.engelbrecht #rb Devin.Doucette #jira UE-210439 [FYI] tim.doerries, Zousar.Shaker, Stefan.Boberg [CL 32452554 by dan engelbrecht in ue5-main branch]
2024-03-22 17:53:31 -04:00
uint32_t AcceptFlags = static_cast<uint32_t>(Zen::Http::RpcAcceptOptions::kAllowPartialCacheChunks);
Allow support for receiving local file references for cache items Companion PR in zen: https://github.com/EpicGames/zen/pull/184 #rb stefan.boberg #preflight 6368ce57d0174259cc1e7b7a [CL 23009037 by dan engelbrecht in ue5-main branch]
2022-11-07 05:33:21 -05:00
if (CacheStore.bIsLocalConnection)
{
Add support of partial zen DDC GetChunks requests #ushell-cherrypick of 32395474 by dan.engelbrecht #rb Devin.Doucette #jira UE-210439 [FYI] tim.doerries, Zousar.Shaker, Stefan.Boberg [CL 32452554 by dan engelbrecht in ue5-main branch]
2024-03-22 17:53:31 -04:00
AcceptFlags |= static_cast<uint32_t>(Zen::Http::RpcAcceptOptions::kAllowLocalReferences);
Accept duplicated file handles in Zen RPC responses #preflight 641c11bf76461c460b8eabb4 #rb stefan.boberg per.larsson [CL 24759274 by dan engelbrecht in ue5-main branch]
2023-03-23 05:15:22 -04:00
BatchRequest.AddInteger(ANSITEXTVIEW("Pid"), FPlatformProcess::GetCurrentProcessId());
Allow support for receiving local file references for cache items Companion PR in zen: https://github.com/EpicGames/zen/pull/184 #rb stefan.boberg #preflight 6368ce57d0174259cc1e7b7a [CL 23009037 by dan engelbrecht in ue5-main branch]
2022-11-07 05:33:21 -05:00
}
Add support of partial zen DDC GetChunks requests #ushell-cherrypick of 32395474 by dan.engelbrecht #rb Devin.Doucette #jira UE-210439 [FYI] tim.doerries, Zousar.Shaker, Stefan.Boberg [CL 32452554 by dan engelbrecht in ue5-main branch]
2024-03-22 17:53:31 -04:00
BatchRequest.AddInteger(ANSITEXTVIEW("AcceptFlags"), AcceptFlags);
Add "Accept" field in Zen RPC request to enable new style RPC responses. See https://github.com/EpicGames/zen/pull/180 for Zen side change. #rb stefan.boberg zousar.shaker #preflight 6347c2568a0a7b2adcbe4d71 [CL 22505182 by dan engelbrecht in ue5-main branch]
2022-10-13 11:17:53 -04:00
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchRequest.BeginObject(ANSITEXTVIEW("Params"));
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
ECachePolicy DefaultPolicy = Batch[0].Request.Policy;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchRequest << ANSITEXTVIEW("DefaultPolicy") << WriteToString<128>(DefaultPolicy);
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
BatchRequest.AddString(ANSITEXTVIEW("Namespace"), CacheStore.Namespace);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchRequest.BeginArray(ANSITEXTVIEW("ChunkRequests"));
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (const TRequestWithStats<FCacheGetChunkRequest>& RequestWithStats : Batch)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const FCacheGetChunkRequest& Request = RequestWithStats.Request;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
BatchRequest.BeginObject();
{
BatchRequest << ANSITEXTVIEW("Key") << Request.Key;
if (Request.Id.IsValid())
{
BatchRequest.AddObjectId(ANSITEXTVIEW("ValueId"), Request.Id);
}
if (Request.RawOffset != 0)
{
BatchRequest << ANSITEXTVIEW("RawOffset") << Request.RawOffset;
}
if (Request.RawSize != MAX_uint64)
{
BatchRequest << ANSITEXTVIEW("RawSize") << Request.RawSize;
}
if (!Request.RawHash.IsZero())
{
BatchRequest << ANSITEXTVIEW("ChunkId") << Request.RawHash;
}
if (Request.Policy != DefaultPolicy)
{
BatchRequest << ANSITEXTVIEW("Policy") << WriteToString<128>(Request.Policy);
}
}
BatchRequest.EndObject();
}
BatchRequest.EndArray();
}
BatchRequest.EndObject();
}
BatchRequest.EndObject();
Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc #preflight 63341e5a10030508062960fa #rb devin.doucette zousar.shaker [CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
FGetChunksOp* OriginalOp = this;
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
auto OnRpcComplete = [this, OpRef = TRefCountPtr<FGetChunksOp>(OriginalOp), Batch](const THttpUniquePtr<IHttpResponse>& HttpResponse, const FCbPackage& Response)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
const bool bCanceled = !HttpResponse || HttpResponse->GetErrorCode() == EHttpErrorCode::Canceled;
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
FRequestTimer RequestTimer(Requests[0].Stats);
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (HttpResponse)
{
const FHttpResponseStats& ResponseStats = HttpResponse->GetStats();
Requests[0].Stats.Latency = FMonotonicTimeSpan::FromSeconds(ResponseStats.GetLatency());
}
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc #preflight 63341e5a10030508062960fa #rb devin.doucette zousar.shaker [CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
int32 RequestIndex = 0;
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (!bCanceled && HttpResponse->GetErrorCode() == EHttpErrorCode::None && HttpResponse->GetStatusCode() >= 200 && HttpResponse->GetStatusCode() <= 299)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc #preflight 63341e5a10030508062960fa #rb devin.doucette zousar.shaker [CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
const FCbObject& ResponseObj = Response.GetObject();
for (FCbFieldView ResultView : ResponseObj[ANSITEXTVIEW("Result")])
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc #preflight 63341e5a10030508062960fa #rb devin.doucette zousar.shaker [CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
if (RequestIndex >= Batch.Num())
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc #preflight 63341e5a10030508062960fa #rb devin.doucette zousar.shaker [CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
++RequestIndex;
continue;
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const TRequestWithStats<FCacheGetChunkRequest>& RequestWithStats = Batch[RequestIndex++];
Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc #preflight 63341e5a10030508062960fa #rb devin.doucette zousar.shaker [CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
FIoHash RawHash;
bool Succeeded = false;
uint64 RawSize = 0;
FCbObjectView ResultObject = ResultView.AsObjectView();
FSharedBuffer RequestedBytes;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const FCacheGetChunkRequest& Request = RequestWithStats.Request;
Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc #preflight 63341e5a10030508062960fa #rb devin.doucette zousar.shaker [CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
if (CacheStore.DebugOptions.ShouldSimulateGetMiss(Request.Key))
{
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Simulated miss for get of '%s' from '%s'"),
*CacheStore.GetName(), *WriteToString<96>(Request.Key, '/', Request.Id), *Request.Name);
}
else
{
FCbFieldView HashView = ResultObject[ANSITEXTVIEW("RawHash")];
RawHash = HashView.AsHash();
if (!HashView.HasError())
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Add support of partial zen DDC GetChunks requests #ushell-cherrypick of 32395474 by dan.engelbrecht #rb Devin.Doucette #jira UE-210439 [FYI] tim.doerries, Zousar.Shaker, Stefan.Boberg [CL 32452554 by dan engelbrecht in ue5-main branch]
2024-03-22 17:53:31 -04:00
FIoHash AttachmentHash = RawHash;
const FCbAttachment* Attachment = nullptr;
if (EnumHasAnyFlags(RequestWithStats.Request.Policy, ECachePolicy::SkipData))
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc #preflight 63341e5a10030508062960fa #rb devin.doucette zousar.shaker [CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
FCbFieldView RawSizeField = ResultObject[ANSITEXTVIEW("RawSize")];
uint64 TotalSize = RawSizeField.AsUInt64();
Succeeded = !RawSizeField.HasError();
if (Succeeded)
{
RawSize = FMath::Min(Request.RawSize, TotalSize - FMath::Min(Request.RawOffset, TotalSize));
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
Add support of partial zen DDC GetChunks requests #ushell-cherrypick of 32395474 by dan.engelbrecht #rb Devin.Doucette #jira UE-210439 [FYI] tim.doerries, Zousar.Shaker, Stefan.Boberg [CL 32452554 by dan engelbrecht in ue5-main branch]
2024-03-22 17:53:31 -04:00
else
{
FCbFieldView FragmentOffsetField = ResultObject[ANSITEXTVIEW("FragmentOffset")];
uint64 FragmentOffset = FragmentOffsetField.AsUInt64();
FCbFieldView FragmentHashView = ResultObject[ANSITEXTVIEW("FragmentHash")];
if (FragmentHashView.IsHash())
{
FIoHash FragmentHash = FragmentHashView.AsHash();
AttachmentHash = FragmentHash;
}
if (Attachment = Response.FindAttachment(AttachmentHash); Attachment != nullptr)
{
if (FCompressedBuffer CompressedBuffer = Attachment->AsCompressedBinary(); CompressedBuffer)
{
RequestedBytes = FCompressedBufferReader(CompressedBuffer).Decompress(Request.RawOffset - FragmentOffset, Request.RawSize);
RawSize = RequestedBytes.GetSize();
Succeeded = true;
}
}
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
Fixes issue where negative legacy could be accumulated in DDC request statistics. - If a connection hasn't been established we now return a latency of 0 seconds rather than a negative number - When calculating latency we use the pretransfer time from curl which fits closer to the intention of measuring latency, specifically the time between asking for bytes and receiving the first byte. - Adds GetLatency utility method to `FHttpResponseStats` as we calculate latency in multiple places manually and all contained the same error #jira UE-206398 #rb Devin.Doucette, Zousar.Shaker [FYI] Francis.Hurteau [CL 32467988 by kevin macaulayvacher in ue5-main branch]
2024-03-25 08:45:29 -04:00
else
{
FCbFieldView HashView2 = ResultObject[ANSITEXTVIEW("RawHash")];
RawHash = HashView2.AsHash();
}
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
Succeeded ? OnHit(RequestWithStats, MoveTemp(RawHash), RawSize, MoveTemp(RequestedBytes)) : OnMiss(RequestWithStats);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
}
}
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
else if (!bCanceled && (HttpResponse->GetStatusCode() != 404))
Check status code for responses received from zenserver DDC requests. #rb stefan.boberg #preflight 645a3339013f3751584a39ed [CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
{
Downgrade messages from ZenCacheStore from Warning to Display as they can be expected to come up due to network conditions or temporary outages and do not break functionality. #rb devin.doucette [CL 28357691 by zousar shaker in ue5-main branch]
2023-09-29 17:36:33 -04:00
UE_LOG(LogDerivedDataCache, Display,
Check status code for responses received from zenserver DDC requests. #rb stefan.boberg #preflight 645a3339013f3751584a39ed [CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
TEXT("%s: Error response received from GetChunks RPC: from %s"),
*CacheStore.GetName(), *WriteToString<256>(*HttpResponse));
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
for (const TRequestWithStats<FCacheGetChunkRequest>& RequestWithStats : Batch.RightChop(RequestIndex))
Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc #preflight 63341e5a10030508062960fa #rb devin.doucette zousar.shaker [CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
if (bCanceled)
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
{
OnCanceled(RequestWithStats);
}
else
{
OnMiss(RequestWithStats);
}
Make ZenCacheStore::FGetChunksOp asynchronous using EnqueueAsyncRpc #preflight 63341e5a10030508062960fa #rb devin.doucette zousar.shaker [CL 22230515 by dan engelbrecht in ue5-main branch]
2022-09-28 16:39:24 -04:00
}
};
CacheStore.EnqueueAsyncRpc(Owner, BatchRequest.Save().AsObject(), MoveTemp(OnRpcComplete));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
});
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
private:
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
void OnHit(const TRequestWithStats<FCacheGetChunkRequest>& RequestWithStats, FIoHash&& RawHash, uint64 RawSize, FSharedBuffer&& RequestedBytes)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const FCacheGetChunkRequest& Request = RequestWithStats.Request;
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache hit for '%s' from '%s'"),
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
*CacheStore.GetName(), *WriteToString<96>(Request.Key, '/', Request.Id), *Request.Name);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
// This is a rough estimate of physical read size until Zen communicates stats with each response.
Add CPU time and latency stats to ZenCacheStore Fix LogicalReadSize for ZenCacheStore GetChunks Add missing fields for GatherAnalytics in ZenServerInterface #rb devin.doucette [CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
RequestWithStats.Stats.LogicalReadSize += RawSize;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
RequestWithStats.Stats.PhysicalReadSize += RequestedBytes.GetSize();
RequestWithStats.EndRequest(CacheStore, EStatus::Ok);
Only report local stats for Zen DDC. ZenCacheStore now only tracks what the client sees, regardless if the zenserver instance gets a hit from a configured upstream. As we don't currently have any upstreams configured in the zen server instance this should give a more accurate number for hit/miss stats that matches the file cache. #preflight 63eabb1bb96a6b2718326d15 #rb zousar.shaker devin.doucette [CL 24206594 by dan engelbrecht in ue5-main branch]
2023-02-14 03:09:44 -05:00
TRACE_COUNTER_INCREMENT(ZenDDC_GetHit);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TRACE_COUNTER_ADD(ZenDDC_BytesReceived, int64(RequestWithStats.Stats.PhysicalReadSize));
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
OnComplete({Request.Name, Request.Key, Request.Id, Request.RawOffset,
RawSize, MoveTemp(RawHash), MoveTemp(RequestedBytes), Request.UserData, EStatus::Ok});
};
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
void OnMiss(const TRequestWithStats<FCacheGetChunkRequest>& RequestWithStats)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
const FCacheGetChunkRequest& Request = RequestWithStats.Request;
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache miss with missing value '%s' for '%s' from '%s'"),
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
*CacheStore.GetName(), *WriteToString<16>(Request.Id), *WriteToString<96>(Request.Key), *Request.Name);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
RequestWithStats.EndRequest(CacheStore, EStatus::Error);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
OnComplete(Request.MakeResponse(EStatus::Error));
};
Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error. #rb devin.doucette [CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
void OnCanceled(const TRequestWithStats<FCacheGetChunkRequest>& RequestWithStats)
{
const FCacheGetChunkRequest& Request = RequestWithStats.Request;
UE_LOG(LogDerivedDataCache, Verbose, TEXT("%s: Cache miss with canceled request for '%s' from '%s'"),
*CacheStore.GetName(), *WriteToString<96>(Request.Key, '/', Request.Id), *Request.Name);
RequestWithStats.EndRequest(CacheStore, EStatus::Canceled);
OnComplete(Request.MakeResponse(EStatus::Canceled));
};
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FZenCacheStore& CacheStore;
IRequestOwner& Owner;
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
TArray<TRequestWithStats<FCacheGetChunkRequest>, TInlineAllocator<1>> Requests;
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
FOnCacheGetChunkComplete OnComplete;
};
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
class FZenCacheStore::FHealthReceiver final : public IHttpReceiver
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
{
public:
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
FHealthReceiver(const FHealthReceiver&) = delete;
FHealthReceiver& operator=(const FHealthReceiver&) = delete;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
Adjusting logs related to zen usage to: - Remove repeated log related to fetching cache stats - Ensure that the Zen cache usage has log lines explaining the status of the caches it attempts to connect to (success or failure) #rb Matt.Peters [CL 30710842 by zousar shaker in ue5-main branch]
2024-01-18 22:43:41 -05:00
explicit FHealthReceiver(EHealth& OutHealth, FString* OutResponseBody = nullptr, IHttpReceiver* InNext = nullptr)
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
: Health(OutHealth)
Adjusting logs related to zen usage to: - Remove repeated log related to fetching cache stats - Ensure that the Zen cache usage has log lines explaining the status of the caches it attempts to connect to (success or failure) #rb Matt.Peters [CL 30710842 by zousar shaker in ue5-main branch]
2024-01-18 22:43:41 -05:00
, ResponseBody(OutResponseBody)
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
, Next(InNext)
{
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
Health = EHealth::Unknown;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
}
private:
IHttpReceiver* OnCreate(IHttpResponse& Response) final
{
return &BodyReceiver;
}
IHttpReceiver* OnComplete(IHttpResponse& Response) final
{
FUtf8StringView ResponseStringView(reinterpret_cast<const UTF8CHAR*>(BodyArray.GetData()), IntCastChecked<int32>(BodyArray.Num()));
if (ResponseStringView == UTF8TEXTVIEW("OK!"))
{
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
Health = EHealth::Ok;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
}
else
{
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
Health = EHealth::Error;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
}
Adjusting logs related to zen usage to: - Remove repeated log related to fetching cache stats - Ensure that the Zen cache usage has log lines explaining the status of the caches it attempts to connect to (success or failure) #rb Matt.Peters [CL 30710842 by zousar shaker in ue5-main branch]
2024-01-18 22:43:41 -05:00
if (ResponseBody)
{
*ResponseBody = *WriteToString<64>(ResponseStringView);
}
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
return Next;
}
private:
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
EHealth& Health;
Adjusting logs related to zen usage to: - Remove repeated log related to fetching cache stats - Ensure that the Zen cache usage has log lines explaining the status of the caches it attempts to connect to (success or failure) #rb Matt.Peters [CL 30710842 by zousar shaker in ue5-main branch]
2024-01-18 22:43:41 -05:00
FString* ResponseBody;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
IHttpReceiver* Next;
TArray64<uint8> BodyArray;
FHttpByteArrayReceiver BodyReceiver{ BodyArray, this };
};
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
class FZenCacheStore::FAsyncHealthReceiver final : public FRequestBase, public IHttpReceiver
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
{
public:
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
FAsyncHealthReceiver(const FAsyncHealthReceiver&) = delete;
FAsyncHealthReceiver& operator=(const FAsyncHealthReceiver&) = delete;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
FAsyncHealthReceiver(
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
THttpUniquePtr<IHttpRequest>&& InRequest,
IRequestOwner* InOwner,
Zen::FZenServiceInstance& InZenServiceInstance,
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
FOnHealthComplete&& InOnHealthComplete)
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
: Request(MoveTemp(InRequest))
, Owner(InOwner)
, ZenServiceInstance(InZenServiceInstance)
Adjusting logs related to zen usage to: - Remove repeated log related to fetching cache stats - Ensure that the Zen cache usage has log lines explaining the status of the caches it attempts to connect to (success or failure) #rb Matt.Peters [CL 30710842 by zousar shaker in ue5-main branch]
2024-01-18 22:43:41 -05:00
, BaseReceiver(Health, nullptr, this)
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
, OnHealthComplete(MoveTemp(InOnHealthComplete))
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
{
Request->SendAsync(this, Response);
}
private:
// IRequest Interface
void SetPriority(EPriority Priority) final {}
void Cancel() final { Monitor->Cancel(); }
void Wait() final { Monitor->Wait(); }
// IHttpReceiver Interface
IHttpReceiver* OnCreate(IHttpResponse& LocalResponse) final
{
Monitor = LocalResponse.GetMonitor();
Owner->Begin(this);
return &BaseReceiver;
}
bool ShouldRecoverAndRetry(IHttpResponse& LocalResponse)
{
if (!ZenServiceInstance.IsServiceRunningLocally())
{
return false;
}
if ((LocalResponse.GetErrorCode() == EHttpErrorCode::Connect) ||
(LocalResponse.GetErrorCode() == EHttpErrorCode::TlsConnect) ||
(LocalResponse.GetErrorCode() == EHttpErrorCode::TimedOut))
{
return true;
}
return false;
}
IHttpReceiver* OnComplete(IHttpResponse& LocalResponse) final
{
Owner->End(this, [Self = this, &LocalResponse]
{
if (Self->ShouldRecoverAndRetry(LocalResponse) && Self->ZenServiceInstance.TryRecovery())
{
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
new FAsyncHealthReceiver(MoveTemp(Self->Request), Self->Owner, Self->ZenServiceInstance, MoveTemp(Self->OnHealthComplete));
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
return;
}
Self->Request.Reset();
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
if (Self->OnHealthComplete)
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
{
// Launch a task for the completion function since it can execute arbitrary code.
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
Self->Owner->LaunchTask(TEXT("ZenHealthComplete"), [Self = TRefCountPtr(Self)]
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
{
// Ensuring that the OnRpcComplete method is destroyed by the time we exit this method by moving it to a local scope variable
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
FOnHealthComplete LocalOnComplete = MoveTemp(Self->OnHealthComplete);
LocalOnComplete(Self->Response, Self->Health);
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
});
}
});
return nullptr;
}
private:
THttpUniquePtr<IHttpRequest> Request;
THttpUniquePtr<IHttpResponse> Response;
TRefCountPtr<IHttpResponseMonitor> Monitor;
IRequestOwner* Owner;
Zen::FZenServiceInstance& ZenServiceInstance;
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
EHealth Health;
FHealthReceiver BaseReceiver;
FOnHealthComplete OnHealthComplete;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
};
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
class FZenCacheStore::FCbPackageReceiver final : public IHttpReceiver
{
public:
FCbPackageReceiver(const FCbPackageReceiver&) = delete;
FCbPackageReceiver& operator=(const FCbPackageReceiver&) = delete;
explicit FCbPackageReceiver(FCbPackage& OutPackage, IHttpReceiver* InNext = nullptr)
: Package(OutPackage)
, Next(InNext)
{
}
private:
IHttpReceiver* OnCreate(IHttpResponse& Response) final
{
return &BodyReceiver;
}
IHttpReceiver* OnComplete(IHttpResponse& Response) final
{
Don't try to read the payload buffer as a package in FZenCacheStore::FCbPackageReceiver if the request failed or was cancelled Add hardening to TryLoadCbPackage to handle incomplete messages [CL 32929188 by dan engelbrecht in ue5-main branch]
2024-04-12 10:31:10 -04:00
if (Response.GetErrorCode() == EHttpErrorCode::None)
Use Http::TryLoadCbPackage as first option to parse packages from Zen. This will allow UE to talk to both old and new zen servers as we move to use the new package wire format in Zen responses. #rb devin.doucette #preflight 6334bf81691c0168b71bc0a9 [CL 22233160 by dan engelbrecht in ue5-main branch]
2022-09-28 18:22:23 -04:00
{
Don't try to read the payload buffer as a package in FZenCacheStore::FCbPackageReceiver if the request failed or was cancelled Add hardening to TryLoadCbPackage to handle incomplete messages [CL 32929188 by dan engelbrecht in ue5-main branch]
2024-04-12 10:31:10 -04:00
FMemoryView MemoryView = MakeMemoryView(BodyArray);
Use Http::TryLoadCbPackage as first option to parse packages from Zen. This will allow UE to talk to both old and new zen servers as we move to use the new package wire format in Zen responses. #rb devin.doucette #preflight 6334bf81691c0168b71bc0a9 [CL 22233160 by dan engelbrecht in ue5-main branch]
2022-09-28 18:22:23 -04:00
{
Don't try to read the payload buffer as a package in FZenCacheStore::FCbPackageReceiver if the request failed or was cancelled Add hardening to TryLoadCbPackage to handle incomplete messages [CL 32929188 by dan engelbrecht in ue5-main branch]
2024-04-12 10:31:10 -04:00
FMemoryReaderView Ar(MemoryView);
if (Zen::Http::TryLoadCbPackage(Package, Ar))
{
return Next;
}
Use Http::TryLoadCbPackage as first option to parse packages from Zen. This will allow UE to talk to both old and new zen servers as we move to use the new package wire format in Zen responses. #rb devin.doucette #preflight 6334bf81691c0168b71bc0a9 [CL 22233160 by dan engelbrecht in ue5-main branch]
2022-09-28 18:22:23 -04:00
}
Don't try to read the payload buffer as a package in FZenCacheStore::FCbPackageReceiver if the request failed or was cancelled Add hardening to TryLoadCbPackage to handle incomplete messages [CL 32929188 by dan engelbrecht in ue5-main branch]
2024-04-12 10:31:10 -04:00
FMemoryReaderView Ar(MemoryView);
Package.TryLoad(Ar);
Use Http::TryLoadCbPackage as first option to parse packages from Zen. This will allow UE to talk to both old and new zen servers as we move to use the new package wire format in Zen responses. #rb devin.doucette #preflight 6334bf81691c0168b71bc0a9 [CL 22233160 by dan engelbrecht in ue5-main branch]
2022-09-28 18:22:23 -04:00
}
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
return Next;
}
private:
FCbPackage& Package;
IHttpReceiver* Next;
TArray64<uint8> BodyArray;
FHttpByteArrayReceiver BodyReceiver{BodyArray, this};
};
class FZenCacheStore::FAsyncCbPackageReceiver final : public FRequestBase, public IHttpReceiver
{
public:
FAsyncCbPackageReceiver(const FAsyncCbPackageReceiver&) = delete;
FAsyncCbPackageReceiver& operator=(const FAsyncCbPackageReceiver&) = delete;
FAsyncCbPackageReceiver(
THttpUniquePtr<IHttpRequest>&& InRequest,
IRequestOwner* InOwner,
Add capability for things that depend on the Zen service to attempt to recover the service if there is an interruption. The Zen DDC cache store is using this to allow recovery and retry of a cache request that fails to connect or times out when communicating with a Zen service on the same machine. #rb devin.doucette #preflight 639cc26343330e63e5db1750 [CL 23541143 by Zousar Shaker in ue5-main branch]
2022-12-16 15:47:14 -05:00
Zen::FZenServiceInstance& InZenServiceInstance,
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
FOnRpcComplete&& InOnRpcComplete)
: Request(MoveTemp(InRequest))
, Owner(InOwner)
Add capability for things that depend on the Zen service to attempt to recover the service if there is an interruption. The Zen DDC cache store is using this to allow recovery and retry of a cache request that fails to connect or times out when communicating with a Zen service on the same machine. #rb devin.doucette #preflight 639cc26343330e63e5db1750 [CL 23541143 by Zousar Shaker in ue5-main branch]
2022-12-16 15:47:14 -05:00
, ZenServiceInstance(InZenServiceInstance)
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
, BaseReceiver(Package, this)
, OnRpcComplete(MoveTemp(InOnRpcComplete))
{
Request->SendAsync(this, Response);
}
private:
// IRequest Interface
void SetPriority(EPriority Priority) final {}
void Cancel() final { Monitor->Cancel(); }
void Wait() final { Monitor->Wait(); }
// IHttpReceiver Interface
IHttpReceiver* OnCreate(IHttpResponse& LocalResponse) final
{
Monitor = LocalResponse.GetMonitor();
Owner->Begin(this);
return &BaseReceiver;
}
Add capability for things that depend on the Zen service to attempt to recover the service if there is an interruption. The Zen DDC cache store is using this to allow recovery and retry of a cache request that fails to connect or times out when communicating with a Zen service on the same machine. #rb devin.doucette #preflight 639cc26343330e63e5db1750 [CL 23541143 by Zousar Shaker in ue5-main branch]
2022-12-16 15:47:14 -05:00
bool ShouldRecoverAndRetry(IHttpResponse& LocalResponse)
{
if (!ZenServiceInstance.IsServiceRunningLocally())
{
return false;
}
if ((LocalResponse.GetErrorCode() == EHttpErrorCode::Connect) ||
(LocalResponse.GetErrorCode() == EHttpErrorCode::TlsConnect) ||
(LocalResponse.GetErrorCode() == EHttpErrorCode::TimedOut))
{
return true;
}
return false;
}
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
IHttpReceiver* OnComplete(IHttpResponse& LocalResponse) final
{
Add capability for things that depend on the Zen service to attempt to recover the service if there is an interruption. The Zen DDC cache store is using this to allow recovery and retry of a cache request that fails to connect or times out when communicating with a Zen service on the same machine. #rb devin.doucette #preflight 639cc26343330e63e5db1750 [CL 23541143 by Zousar Shaker in ue5-main branch]
2022-12-16 15:47:14 -05:00
Owner->End(this, [Self = this, &LocalResponse]
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
{
Add capability for things that depend on the Zen service to attempt to recover the service if there is an interruption. The Zen DDC cache store is using this to allow recovery and retry of a cache request that fails to connect or times out when communicating with a Zen service on the same machine. #rb devin.doucette #preflight 639cc26343330e63e5db1750 [CL 23541143 by Zousar Shaker in ue5-main branch]
2022-12-16 15:47:14 -05:00
if (Self->ShouldRecoverAndRetry(LocalResponse) && Self->ZenServiceInstance.TryRecovery())
{
new FAsyncCbPackageReceiver(MoveTemp(Self->Request), Self->Owner, Self->ZenServiceInstance, MoveTemp(Self->OnRpcComplete));
return;
}
Self->Request.Reset();
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
if (Self->OnRpcComplete)
{
// Launch a task for the completion function since it can execute arbitrary code.
Self->Owner->LaunchTask(TEXT("ZenHttpComplete"), [Self = TRefCountPtr(Self)]
{
Fix memory leak caused by reference counted objects capturing references to themselves in function captures held on members of itself. This manifested as memory never being freed when performing some operations with HttpCacheStore and ZenCacheStore. #rb devin.doucette [CL 29316880 by zousar shaker in ue5-main branch]
2023-11-01 10:33:43 -04:00
// Ensuring that the OnRpcComplete method is destroyed by the time we exit this method by moving it to a local scope variable
FOnRpcComplete LocalOnComplete = MoveTemp(Self->OnRpcComplete);
LocalOnComplete(Self->Response, Self->Package);
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
});
}
});
return nullptr;
}
private:
THttpUniquePtr<IHttpRequest> Request;
THttpUniquePtr<IHttpResponse> Response;
TRefCountPtr<IHttpResponseMonitor> Monitor;
IRequestOwner* Owner;
Add capability for things that depend on the Zen service to attempt to recover the service if there is an interruption. The Zen DDC cache store is using this to allow recovery and retry of a cache request that fails to connect or times out when communicating with a Zen service on the same machine. #rb devin.doucette #preflight 639cc26343330e63e5db1750 [CL 23541143 by Zousar Shaker in ue5-main branch]
2022-12-16 15:47:14 -05:00
Zen::FZenServiceInstance& ZenServiceInstance;
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
FCbPackage Package;
FCbPackageReceiver BaseReceiver;
FOnRpcComplete OnRpcComplete;
};
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
FZenCacheStore::FZenCacheStore(
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
const FZenCacheStoreParams& InParams,
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
ICacheStoreOwner* InStoreOwner)
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
: ZenService(*InParams.Host)
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
, StoreOwner(InStoreOwner)
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
, PerformanceEvaluationRequestOwner(EPriority::Low)
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
Initialize(InParams);
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
}
FZenCacheStore::FZenCacheStore(
UE::Zen::FServiceSettings&& InSettings,
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
const FZenCacheStoreParams& InParams,
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
ICacheStoreOwner* InStoreOwner)
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
: ZenService(MoveTemp(InSettings))
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
, StoreOwner(InStoreOwner)
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
, PerformanceEvaluationRequestOwner(EPriority::Low)
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
Initialize(InParams);
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
}
DDC: Fixed leaking cache store stats in the Zen cache #jira UE-133395 #rb none #rnx [CL 25977512 by devin doucette in ue5-main branch]
2023-06-14 10:12:47 -04:00
FZenCacheStore::~FZenCacheStore()
{
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
PerformanceEvaluationRequestOwner.Cancel();
if (PerformanceEvaluationThread.IsSet())
{
PerformanceEvaluationThreadShutdownEvent.Notify();
PerformanceEvaluationThread->Join();
PerformanceEvaluationThread.Reset();
PerformanceEvaluationThreadShutdownEvent.Reset();
}
DDC: Fixed leaking cache store stats in the Zen cache #jira UE-133395 #rb none #rnx [CL 25977512 by devin doucette in ue5-main branch]
2023-06-14 10:12:47 -04:00
if (StoreStats)
{
StoreOwner->DestroyStats(StoreStats);
}
}
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
void FZenCacheStore::Initialize(const FZenCacheStoreParams& Params)
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
{
Adjusting logs related to zen usage to: - Remove repeated log related to fetching cache stats - Ensure that the Zen cache usage has log lines explaining the status of the caches it attempts to connect to (success or failure) #rb Matt.Peters [CL 30710842 by zousar shaker in ue5-main branch]
2024-01-18 22:43:41 -05:00
NodeName = Params.Name;
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
LastPerformanceEvaluationTicks.store(FDateTime::UtcNow().GetTicks(), std::memory_order_relaxed);
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
LastStorageSizeUpdateTicks.store(FDateTime::UtcNow().GetTicks(), std::memory_order_relaxed);
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
Namespace = Params.Namespace;
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
RpcUri << ZenService.GetInstance().GetURL() << ANSITEXTVIEW("/z$/$rpc");
const uint32 MaxConnections = uint32(FMath::Clamp(FPlatformMisc::NumberOfCoresIncludingHyperthreads(), 8, 64));
constexpr uint32 RequestPoolSize = 128;
constexpr uint32 RequestPoolOverflowSize = 128;
FHttpConnectionPoolParams ConnectionPoolParams;
ConnectionPoolParams.MaxConnections = MaxConnections;
ConnectionPoolParams.MinConnections = MaxConnections;
ConnectionPool = IHttpManager::Get().CreateConnectionPool(ConnectionPoolParams);
bool bReady = false;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
{
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
// Issue a synchronous health/ready request with a limited idle time
FHttpClientParams ReadinessClientParams;
ReadinessClientParams.Version = EHttpVersion::V2;
ReadinessClientParams.MaxRequests = 1;
ReadinessClientParams.MinRequests = 1;
ReadinessClientParams.LowSpeedLimit = 1;
ReadinessClientParams.LowSpeedTime = 5; // 5 second idle time limit for the initial readiness check
Add option to bypass proxies to the derived data http client (default is no bypass). Zen cache store client uses it and defaults to bypassing proxies (but can be overridden in INI). UE Cloud DDC client uses it and defaults to NOT bypassing proxies (but can be overridden in INI). #rb Devin.Doucette [FYI] dan.engelbrecht [FYI] joakim.lindqvist #jira UE-210567 [CL 32490735 by zousar shaker in ue5-main branch]
2024-03-25 18:59:10 -04:00
ReadinessClientParams.bBypassProxy = Params.bBypassProxy;
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
THttpUniquePtr<IHttpClient> ReadinessClient = ConnectionPool->CreateClient(ReadinessClientParams);
THttpUniquePtr<IHttpRequest> ReadinessRequest = ReadinessClient->TryCreateRequest({});
TAnsiStringBuilder<256> StatusUri;
StatusUri << ZenService.GetInstance().GetURL() << ANSITEXTVIEW("/health/ready");
ReadinessRequest->SetUri(StatusUri);
ReadinessRequest->SetMethod(EHttpMethod::Get);
ReadinessRequest->AddAcceptType(EHttpMediaType::Text);
Initialize an out variable to satisfy static analysis. [CL 30297745 by zousar shaker in ue5-main branch]
2023-12-13 13:17:25 -05:00
EHealth Health = EHealth::Unknown;
Adjusting logs related to zen usage to: - Remove repeated log related to fetching cache stats - Ensure that the Zen cache usage has log lines explaining the status of the caches it attempts to connect to (success or failure) #rb Matt.Peters [CL 30710842 by zousar shaker in ue5-main branch]
2024-01-18 22:43:41 -05:00
FString ResponseString;
FHealthReceiver HealthReceiver(Health, &ResponseString);
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
THttpUniquePtr<IHttpResponse> ReadinessResponse;
ReadinessRequest->Send(&HealthReceiver, ReadinessResponse);
Silence PVS warnings [CL 30653812 by marc audy in ue5-main branch]
2024-01-17 01:34:02 -05:00
bReady = Health == EHealth::Ok; // -V547
Adjusting logs related to zen usage to: - Remove repeated log related to fetching cache stats - Ensure that the Zen cache usage has log lines explaining the status of the caches it attempts to connect to (success or failure) #rb Matt.Peters [CL 30710842 by zousar shaker in ue5-main branch]
2024-01-18 22:43:41 -05:00
if (ReadinessResponse->GetErrorCode() == EHttpErrorCode::None &&
(ReadinessResponse->GetStatusCode() >= 200 && ReadinessResponse->GetStatusCode() <= 299))
{
UE_LOG(LogDerivedDataCache, Display,
TEXT("%s: Using ZenServer HTTP service at %s with namespace %s status: %s."),
*GetName(), ZenService.GetInstance().GetURL(), *Namespace, *ResponseString);
}
else
{
if (ZenService.GetInstance().IsServiceRunningLocally())
{
UE_LOG(LogDerivedDataCache, Warning,
TEXT("%s: Unable to reach ZenServer HTTP service at %s with namespace %s. Status: %d . Response: %s"),
*GetName(), ZenService.GetInstance().GetURL(), *Namespace, ReadinessResponse->GetStatusCode(),
*ResponseString);
}
else
{
UE_LOG(LogDerivedDataCache, Display,
TEXT("%s: Unable to reach ZenServer HTTP service at %s with namespace %s. Status: %d . Response: %s"),
*GetName(), ZenService.GetInstance().GetURL(), *Namespace, ReadinessResponse->GetStatusCode(),
*ResponseString);
}
}
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
}
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
FHttpClientParams ClientParams;
ClientParams.MaxRequests = RequestPoolSize + RequestPoolOverflowSize;
ClientParams.MinRequests = RequestPoolSize;
ClientParams.LowSpeedLimit = 1;
ClientParams.LowSpeedTime = 25;
Add option to bypass proxies to the derived data http client (default is no bypass). Zen cache store client uses it and defaults to bypassing proxies (but can be overridden in INI). UE Cloud DDC client uses it and defaults to NOT bypassing proxies (but can be overridden in INI). #rb Devin.Doucette [FYI] dan.engelbrecht [FYI] joakim.lindqvist #jira UE-210567 [CL 32490735 by zousar shaker in ue5-main branch]
2024-03-25 18:59:10 -04:00
ClientParams.bBypassProxy = Params.bBypassProxy;
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
RequestQueue.Initialize(*ConnectionPool, ClientParams);
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
Add NoZenLocalFallback and InstalledNoZenLocalFallback DDC graphs for use in scenarios where users are unable to run a local zenserver for any reason. Also fix a mis-labeling of a local cache store as being remote if it is deactivated. #rb devin.doucette #jira UE-208878 [CL 32022101 by zousar shaker in ue5-main branch]
2024-03-05 11:24:22 -05:00
bIsLocalConnection = ZenService.GetInstance().IsServiceRunningLocally() || ZenService.GetInstance().GetServiceSettings().IsAutoLaunch();
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
bIsUsable = true;
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
if (StoreOwner)
{
// Default to locally launched service getting the Local cache store flag. Can be overridden by explicit value in config.
bool bLocal = Params.bLocal.Get(bIsLocalConnection);
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
// Default to non-locally launched service getting the Remote cache store flag. Can be overridden by explicit value in config.
// In the future this could be extended to allow the Remote flag by default (even for locally launched instances) if they have upstreams configured.
bool bRemote = Params.bRemote.Get(!bIsLocalConnection);
DeactivateAtMs = Params.DeactivateAtMs;
ECacheStoreFlags Flags = ECacheStoreFlags::Query;
Flags |= Params.bReadOnly ? ECacheStoreFlags::None : ECacheStoreFlags::Store;
Flags |= bLocal ? ECacheStoreFlags::Local : ECacheStoreFlags::None;
Flags |= bRemote ? ECacheStoreFlags::Remote : ECacheStoreFlags::None;
OperationalFlags = Flags;
if (!bReady)
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
{
Add NoZenLocalFallback and InstalledNoZenLocalFallback DDC graphs for use in scenarios where users are unable to run a local zenserver for any reason. Also fix a mis-labeling of a local cache store as being remote if it is deactivated. #rb devin.doucette #jira UE-208878 [CL 32022101 by zousar shaker in ue5-main branch]
2024-03-05 11:24:22 -05:00
Flags = OperationalFlags & ~(ECacheStoreFlags::Store | ECacheStoreFlags::Query);
if (bIsLocalConnection)
{
UE_LOG(LogDerivedDataCache, Display,
TEXT("%s: Readiness check failed. "
"It will be deactivated until responsiveness improves. "
"If this is consistent, consider disabling this cache store through "
"the use of the '-ddc=NoZenLocalFallback' or '-ddc=InstalledNoZenLocalFallback' "
"commandline arguments."),
*GetName());
}
else
{
UE_LOG(LogDerivedDataCache, Display,
TEXT("%s: Readiness check failed. "
"It will be deactivated until responsiveness improves. "
"If this is consistent, consider disabling this cache store through "
"environment variables or other configuration."),
*GetName());
}
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
bDeactivatedForPerformance.store(true, std::memory_order_relaxed);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
}
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
StoreOwner->Add(this, Flags);
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
TStringBuilder<256> Path(InPlace, ZenService.GetInstance().GetPath(), TEXTVIEW(" ("), Namespace, TEXTVIEW(")"));
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
StoreStats = StoreOwner->CreateStats(this, Flags, TEXT("Zen"), *Params.Name, Path);
bTryEvaluatePerformance = !GIsBuildMachine && (StoreStats != nullptr) && (DeactivateAtMs > 0.0f);
StoreStats->SetAttribute(TEXTVIEW("Namespace"), Namespace);
if (!bReady)
{
UpdateStatus();
ActivatePerformanceEvaluationThread();
}
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
// Issue a request for stats as it will be fetched asynchronously and issuing now makes them available sooner for future callers.
Zen::FZenCacheStats ZenStats;
ZenService.GetInstance().GetCacheStats(ZenStats);
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
MaxBatchPutKB = Params.MaxBatchPutKB;
CacheRecordBatchSize = Params.RecordBatchSize;
CacheChunksBatchSize = Params.ChunksBatchSize;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
bool FZenCacheStore::IsServiceReady()
{
return ZenService.GetInstance().IsServiceReady();
}
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
FCompositeBuffer FZenCacheStore::SaveRpcPackage(const FCbPackage& Package)
2nd submit: Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 #preflight 629642a1be0ae0b33a46d012 [CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
{
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
FLargeMemoryWriter Memory;
Zen::Http::SaveCbPackage(Package, Memory);
uint64 PackageMemorySize = Memory.TotalSize();
return FCompositeBuffer(FSharedBuffer::TakeOwnership(Memory.ReleaseOwnership(), PackageMemorySize, FMemory::Free));
2nd submit: Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 #preflight 629642a1be0ae0b33a46d012 [CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
}
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
void FZenCacheStore::CreateRpcRequest(IRequestOwner& Owner, FHttpRequestQueue::FOnRequest&& OnRequest)
2nd submit: Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 #preflight 629642a1be0ae0b33a46d012 [CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
RequestQueue.CreateRequestAsync(Owner, {}, [this, OnRequest = MoveTemp(OnRequest)](THttpUniquePtr<IHttpRequest>&& Request)
{
if (Request)
{
Request->SetUri(RpcUri);
Request->SetMethod(EHttpMethod::Post);
Request->AddAcceptType(EHttpMediaType::CbPackage);
}
OnRequest(MoveTemp(Request));
});
2nd submit: Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 #preflight 629642a1be0ae0b33a46d012 [CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
}
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
void FZenCacheStore::EnqueueAsyncRpc(IRequestOwner& Owner, FCbObject RequestObject, FOnRpcComplete&& OnComplete)
2nd submit: Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 #preflight 629642a1be0ae0b33a46d012 [CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
CreateRpcRequest(Owner, [this, &Owner, RequestObject, OnComplete = MoveTemp(OnComplete)](THttpUniquePtr<IHttpRequest>&& Request) mutable
{
if (UNLIKELY(!Request))
{
OnComplete({}, {});
return;
}
Request->SetContentType(EHttpMediaType::CbObject);
Request->SetBody(RequestObject.GetBuffer().MakeOwned());
new FAsyncCbPackageReceiver(MoveTemp(Request), &Owner, ZenService.GetInstance(), MoveTemp(OnComplete));
});
DDC: Refactored the HTTP API to add configurability and remove reliance on types from DDC - FHttpSharedData is now IHttpConnectionPool. It is a connection pool of configurable size. - FHttpRequestPool is now IHttpClient. It is a request pool of configurable size and the point at which most behaviors are configured. - FHttpRequest is now split into IHttpRequest and IHttpResponse. This separation allows requests to be released back to the client for reuse while the response is still being processed. - FScopedHttpPoolRequestPtr is now THttpUniquePtr. It is used uniformly with the pool, client, request, and response. - The response body is now handled by IHttpReceiver, with a basic FHttpByteArrayReceiver provided to capture the whole body into an array. - The response now includes stats about how much time was spent on various parts of the request, as well as how much data was sent and received and at what rate. - The response supports cancellation at any point before it is complete, and IHttpResponseMonitor is provided to support thread-safe cancellation and waiting without requiring locks around the unique owner of the response. - The response will be canceled automatically when destroyed through its THttpUniquePtr. - The client does not directly support blocking on request creation. Blocking creation of requests is now provided by FHttpRequestQueue. - The HTTP thread now relies on curl_multi_poll, available in newer libcurl versions, to allow immediate waking when a handle needs to be added or removed. #preflight 62d580dd2e3e5993c3b678dd #rb Zousar.Shaker [CL 21149626 by Devin Doucette in ue5-main branch]
2022-07-18 12:23:16 -04:00
}
void FZenCacheStore::EnqueueAsyncRpc(IRequestOwner& Owner, const FCbPackage& RequestPackage, FOnRpcComplete&& OnComplete)
{
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
CreateRpcRequest(Owner, [this, &Owner, RequestPackage, OnComplete = MoveTemp(OnComplete)](THttpUniquePtr<IHttpRequest>&& Request) mutable
{
if (UNLIKELY(!Request))
{
OnComplete({}, {});
return;
}
Request->SetContentType(EHttpMediaType::CbPackage);
Request->SetBody(SaveRpcPackage(RequestPackage));
new FAsyncCbPackageReceiver(MoveTemp(Request), &Owner, ZenService.GetInstance(), MoveTemp(OnComplete));
});
2nd submit: Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 #preflight 629642a1be0ae0b33a46d012 [CL 20438549 by Zousar Shaker in ue5-main branch]
2022-05-31 12:42:17 -04:00
}
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
void FZenCacheStore::ActivatePerformanceEvaluationThread()
{
if (!PerformanceEvaluationThread.IsSet())
{
PerformanceEvaluationThread.Emplace(TEXT("ZenCacheStore Performance Evaluation"), [this]
{
while (!PerformanceEvaluationThreadShutdownEvent.WaitFor(FMonotonicTimeSpan::FromSeconds(30.0)))
{
IRequestOwner& Owner(PerformanceEvaluationRequestOwner);
FRequestBarrier Barrier(Owner);
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
RequestQueue.CreateRequestAsync(Owner, {}, [this, &Owner](THttpUniquePtr<IHttpRequest>&& Request)
{
if (UNLIKELY(!Request))
{
return;
}
TAnsiStringBuilder<256> StatusUri;
StatusUri << ZenService.GetInstance().GetURL() << ANSITEXTVIEW("/health/ready");
Request->SetUri(StatusUri);
Request->SetMethod(EHttpMethod::Get);
Request->AddAcceptType(EHttpMediaType::Text);
new FAsyncHealthReceiver(MoveTemp(Request), &Owner, ZenService.GetInstance(), [this, StartTime = FMonotonicTimePoint::Now()](const THttpUniquePtr<IHttpResponse>& HttpResponse, EHealth Health)
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
{
if (Health != EHealth::Ok)
{
// Any non-ok health means we hold off any possibility of reactivating and we don't add the lagency to the stats
// as the failure may be client-side and instantaneous which will create an artificially low latency measurement.
return;
}
Fixes issue where negative legacy could be accumulated in DDC request statistics. - If a connection hasn't been established we now return a latency of 0 seconds rather than a negative number - When calculating latency we use the pretransfer time from curl which fits closer to the intention of measuring latency, specifically the time between asking for bytes and receiving the first byte. - Adds GetLatency utility method to `FHttpResponseStats` as we calculate latency in multiple places manually and all contained the same error #jira UE-206398 #rb Devin.Doucette, Zousar.Shaker [FYI] Francis.Hurteau [CL 32467988 by kevin macaulayvacher in ue5-main branch]
2024-03-25 08:45:29 -04:00
const double LatencySec = HttpResponse->GetStats().GetLatency();
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
StoreStats->AddLatency(StartTime, FMonotonicTimePoint::Now(), FMonotonicTimeSpan::FromSeconds(LatencySec));
if (!bTryEvaluatePerformance || (StoreStats->GetAverageLatency() * 1000 <= DeactivateAtMs))
{
if (PerformanceEvaluationThread.IsSet())
{
PerformanceEvaluationThreadShutdownEvent.Notify();
}
if (bDeactivatedForPerformance.load(std::memory_order_relaxed))
{
StoreOwner->SetFlags(this, OperationalFlags);
UE_LOG(LogDerivedDataCache, Display,
TEXT("%s: Performance has improved and meets minimum performance criteria. "
"It will be reactivated now."),
*GetName());
bDeactivatedForPerformance.store(false, std::memory_order_relaxed);
UpdateStatus();
}
}
});
DDC: Eliminated most blocking during requests in ZenCacheStore #rb Zousar.Shaker [CL 33874002 by devin doucette in ue5-main branch]
2024-05-23 16:37:20 -04:00
});
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
}
});
}
}
For local Zen DDC connections show the path where data is store in the Cache Statistics tab for DDC instead of local ip address For Zen DDC connection show the storage size #jira UE-199929 #rb Devin.Doucette [CL 30501101 by dan engelbrecht in ue5-main branch]
2024-01-09 06:08:29 -05:00
void FZenCacheStore::ConditionalUpdateStorageSize()
{
if (!StoreStats)
{
return;
}
// Look for an opportunity to measure and evaluate if storage size is acceptable.
int64 LocalStorageSizeUpdateTicks = LastStorageSizeUpdateTicks.load(std::memory_order_relaxed);
FTimespan TimespanSinceLastStorageSizeUpdate = FDateTime::UtcNow() - FDateTime(LocalStorageSizeUpdateTicks);
if (TimespanSinceLastStorageSizeUpdate < FTimespan::FromSeconds(30))
{
return;
}
if (!LastStorageSizeUpdateTicks.compare_exchange_strong(LocalStorageSizeUpdateTicks, FDateTime::UtcNow().GetTicks()))
{
return;
}
bool PhysicalSizeIsValid = false;
double PhysicalSize = 0.0;
Zen::FZenCacheStats ZenCacheStats;
if (ZenService.GetInstance().GetCacheStats(ZenCacheStats))
{
PhysicalSize += ZenCacheStats.General.Size.Disk + static_cast<double>(ZenCacheStats.CID.Size.Total);
PhysicalSizeIsValid = true;
}
Zen::FZenProjectStats ZenProjectStats;
if (ZenService.GetInstance().GetProjectStats(ZenProjectStats))
{
PhysicalSize += ZenProjectStats.General.Size.Disk;
PhysicalSizeIsValid = true;
}
if (PhysicalSizeIsValid)
{
StoreStats->SetTotalPhysicalSize(static_cast<uint64>(PhysicalSize));
}
}
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
void FZenCacheStore::ConditionalEvaluatePerformance()
{
if (!bTryEvaluatePerformance)
{
return;
}
// Look for an opportunity to measure and evaluate if performance is acceptable.
int64 LocalLastPerfEvaluationTicks = LastPerformanceEvaluationTicks.load(std::memory_order_relaxed);
FTimespan TimespanSinceLastPerfEval = FDateTime::UtcNow() - FDateTime(LocalLastPerfEvaluationTicks);
if (TimespanSinceLastPerfEval < FTimespan::FromSeconds(30))
{
return;
}
if (!LastPerformanceEvaluationTicks.compare_exchange_strong(LocalLastPerfEvaluationTicks, FDateTime::UtcNow().GetTicks()))
{
return;
}
// We won the race and get to do the performance check
if (PerformanceEvaluationThread.IsSet() && PerformanceEvaluationThreadShutdownEvent.IsNotified())
{
// Join and cleanup old thread before we consider whether we need to start a new one
PerformanceEvaluationThread->Join();
PerformanceEvaluationThread.Reset();
PerformanceEvaluationThreadShutdownEvent.Reset();
}
if (StoreStats->GetAverageLatency() * 1000 > DeactivateAtMs)
{
if (!bDeactivatedForPerformance.load(std::memory_order_relaxed))
{
Add NoZenLocalFallback and InstalledNoZenLocalFallback DDC graphs for use in scenarios where users are unable to run a local zenserver for any reason. Also fix a mis-labeling of a local cache store as being remote if it is deactivated. #rb devin.doucette #jira UE-208878 [CL 32022101 by zousar shaker in ue5-main branch]
2024-03-05 11:24:22 -05:00
StoreOwner->SetFlags(this, OperationalFlags & ~(ECacheStoreFlags::Store | ECacheStoreFlags::Query));
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
UE_LOG(LogDerivedDataCache, Display,
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
TEXT("%s: Performance does not meet minimum criteria. "
"It will be deactivated until performance measurements improve. "
"If this is consistent, consider disabling this cache store through "
"environment variables or other configuration."),
*GetName());
bDeactivatedForPerformance.store(true, std::memory_order_relaxed);
UpdateStatus();
}
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
ActivatePerformanceEvaluationThread();
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
}
}
void FZenCacheStore::UpdateStatus()
{
if (StoreStats)
{
if (bDeactivatedForPerformance.load(std::memory_order_relaxed))
{
Change the way readiness of ZenCacheStore is evaluated and re-evaluated. Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit. If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both: - Health is Ok - Performance criteria (if any has been configured and is in use) is met At which point it is activated. Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change. The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup. #rb Devin.Doucette [CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
StoreStats->SetStatus(ECacheStoreStatusCode::Warning, NSLOCTEXT("DerivedDataCache", "DeactivatedForPerformanceOrReadiness", "Deactivated for performance or readiness"));
Add performance disabling (via latency) for ZenCacheStore. #rb devin.doucette [CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
}
else
{
StoreStats->SetStatus(ECacheStoreStatusCode::None, {});
}
}
}
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
void FZenCacheStore::LegacyStats(FDerivedDataCacheStatsNode& OutNode)
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
checkNoEntry();
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
bool FZenCacheStore::LegacyDebugOptions(FBackendDebugOptions& InOptions)
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
{
DebugOptions = InOptions;
return true;
}
void FZenCacheStore::Put(
const TConstArrayView<FCachePutRequest> Requests,
IRequestOwner& Owner,
FOnCachePutComplete&& OnComplete)
{
TRACE_CPUPROFILER_EVENT_SCOPE(ZenDDC::PutCachedRecord);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
TRefCountPtr<FPutOp> PutOp = MakeAsyncOp<FPutOp>(*this, Owner, Requests, MoveTemp(OnComplete));
PutOp->IssueRequests();
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
void FZenCacheStore::Get(
const TConstArrayView<FCacheGetRequest> Requests,
IRequestOwner& Owner,
FOnCacheGetComplete&& OnComplete)
{
TRACE_CPUPROFILER_EVENT_SCOPE(ZenDDC::GetCacheRecord);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
TRefCountPtr<FGetOp> GetOp = MakeAsyncOp<FGetOp>(*this, Owner, Requests, MoveTemp(OnComplete));
GetOp->IssueRequests();
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
void FZenCacheStore::PutValue(
TConstArrayView<FCachePutValueRequest> Requests,
IRequestOwner& Owner,
FOnCachePutValueComplete&& OnComplete)
{
TRACE_CPUPROFILER_EVENT_SCOPE(ZenDDC::PutValue);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
TRefCountPtr<FPutValueOp> PutValueOp = MakeAsyncOp<FPutValueOp>(*this, Owner, Requests, MoveTemp(OnComplete));
PutValueOp->IssueRequests();
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
void FZenCacheStore::GetValue(
TConstArrayView<FCacheGetValueRequest> Requests,
IRequestOwner& Owner,
FOnCacheGetValueComplete&& OnComplete)
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
{
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
TRACE_CPUPROFILER_EVENT_SCOPE(ZenDDC::GetValue);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
TRefCountPtr<FGetValueOp> GetValueOp = MakeAsyncOp<FGetValueOp>(*this, Owner, Requests, MoveTemp(OnComplete));
GetValueOp->IssueRequests();
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
void FZenCacheStore::GetChunks(
TConstArrayView<FCacheGetChunkRequest> Requests,
IRequestOwner& Owner,
FOnCacheGetChunkComplete&& OnComplete)
{
TRACE_CPUPROFILER_EVENT_SCOPE(ZenDDC::GetChunks);
Change Zen cache store to issue non-blocking HTTP requests. GetChunks remains blocking because of the sequential nature of the way it consumes results. This can be changed in the future. Fixed a bug in the code for async POST operations to ensure it sets the post size appropriately. This change requires a fix (in progress) to EditorDomain to avoid a deadlock. #rb devin.doucette #fyi stefan.boberg #preflight 62a3619670680d6143dc5425 [CL 20598472 by Zousar Shaker in ue5-main branch]
2022-06-10 12:56:08 -04:00
TRefCountPtr<FGetChunksOp> GetChunksOp = MakeAsyncOp<FGetChunksOp>(*this, Owner, Requests, MoveTemp(OnComplete));
GetChunksOp->IssueRequests();
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
void FZenCacheStoreParams::Parse(const TCHAR* NodeName, const TCHAR* Config)
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
Name = NodeName;
if (FString ServerId; FParse::Value(Config, TEXT("ServerID="), ServerId))
{
FString ServerEntry;
const TCHAR* ServerSection = TEXT("StorageServers");
if (GConfig->GetString(ServerSection, *ServerId, ServerEntry, GEngineIni))
{
Parse(NodeName, *ServerEntry);
}
else
{
UE_LOG(LogDerivedDataCache, Warning, TEXT("%s: Using ServerID=%s which was not found in [%s]"), NodeName, *ServerId, ServerSection);
}
}
FParse::Value(Config, TEXT("Host="), Host);
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
Add support for a Zen cache store Host to be None (implying that it is disabled) and to have both commandline and environment overrides (similar to the HTTP cache store). #rb devin.doucette [CL 23815887 by zousar shaker in ue5-main branch]
2023-01-23 12:48:38 -05:00
FString OverrideName;
if (FParse::Value(Config, TEXT("EnvHostOverride="), OverrideName))
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
FString HostEnv = FPlatformMisc::GetEnvironmentVariable(*OverrideName);
if (!HostEnv.IsEmpty())
Add support for a Zen cache store Host to be None (implying that it is disabled) and to have both commandline and environment overrides (similar to the HTTP cache store). #rb devin.doucette [CL 23815887 by zousar shaker in ue5-main branch]
2023-01-23 12:48:38 -05:00
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
Host = HostEnv;
UE_LOG(LogDerivedDataCache, Log, TEXT("%s: Found environment override for Host %s=%s"), NodeName, *OverrideName, *Host);
Add support for a Zen cache store Host to be None (implying that it is disabled) and to have both commandline and environment overrides (similar to the HTTP cache store). #rb devin.doucette [CL 23815887 by zousar shaker in ue5-main branch]
2023-01-23 12:48:38 -05:00
}
}
if (FParse::Value(Config, TEXT("CommandLineHostOverride="), OverrideName))
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
if (FParse::Value(FCommandLine::Get(), *(OverrideName + TEXT("=")), Host))
Add support for a Zen cache store Host to be None (implying that it is disabled) and to have both commandline and environment overrides (similar to the HTTP cache store). #rb devin.doucette [CL 23815887 by zousar shaker in ue5-main branch]
2023-01-23 12:48:38 -05:00
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
UE_LOG(LogDerivedDataCache, Log, TEXT("%s: Found command line override for Host %s=%s"), NodeName, *OverrideName, *Host);
Add support for a Zen cache store Host to be None (implying that it is disabled) and to have both commandline and environment overrides (similar to the HTTP cache store). #rb devin.doucette [CL 23815887 by zousar shaker in ue5-main branch]
2023-01-23 12:48:38 -05:00
}
}
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
FParse::Value(Config, TEXT("Namespace="), Namespace);
FParse::Value(Config, TEXT("StructuredNamespace="), Namespace);
FParse::Bool(Config, TEXT("ReadOnly="), bReadOnly);
// Sandbox and flush configuration for use in Cold/Warm type use cases
FParse::Value(Config, TEXT("Sandbox="), Sandbox);
FParse::Bool(Config, TEXT("Flush="), bFlush);
Add option to bypass proxies to the derived data http client (default is no bypass). Zen cache store client uses it and defaults to bypassing proxies (but can be overridden in INI). UE Cloud DDC client uses it and defaults to NOT bypassing proxies (but can be overridden in INI). #rb Devin.Doucette [FYI] dan.engelbrecht [FYI] joakim.lindqvist #jira UE-210567 [CL 32490735 by zousar shaker in ue5-main branch]
2024-03-25 18:59:10 -04:00
FParse::Bool(Config, TEXT("BypassProxy="), bBypassProxy);
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
// Performance deactivation
FParse::Value(Config, TEXT("DeactivateAt="), DeactivateAtMs);
// Explicit local and remote configuration
if (bool bExplicitLocal = false; FParse::Bool(Config, TEXT("Local="), bExplicitLocal))
Add support for a Zen cache store Host to be None (implying that it is disabled) and to have both commandline and environment overrides (similar to the HTTP cache store). #rb devin.doucette [CL 23815887 by zousar shaker in ue5-main branch]
2023-01-23 12:48:38 -05:00
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
bLocal = bExplicitLocal;
}
if (bool bExplicitRemote = false; FParse::Bool(Config, TEXT("Remote="), bExplicitRemote))
{
bRemote = bExplicitRemote;
}
// Request batch fracturing configuration
FParse::Value(Config, TEXT("MaxBatchPutKB="), MaxBatchPutKB);
FParse::Value(Config, TEXT("RecordBatchSize="), RecordBatchSize);
FParse::Value(Config, TEXT("ChunksBatchSize="), ChunksBatchSize);
}
ILegacyCacheStore* CreateZenCacheStore(const TCHAR* NodeName, const TCHAR* Config, ICacheStoreOwner* Owner)
{
FZenCacheStoreParams Params;
Params.Parse(NodeName, Config);
if (Params.Host == TEXTVIEW("None"))
{
UE_LOG(LogDerivedDataCache, Log, TEXT("%s: Disabled because Host is set to 'None'"), NodeName);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
return nullptr;
Add support for a Zen cache store Host to be None (implying that it is disabled) and to have both commandline and environment overrides (similar to the HTTP cache store). #rb devin.doucette [CL 23815887 by zousar shaker in ue5-main branch]
2023-01-23 12:48:38 -05:00
}
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
if (Params.Namespace.IsEmpty())
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
Params.Namespace = FApp::GetProjectName();
UE_LOG(LogDerivedDataCache, Warning, TEXT("%s: Missing required parameter 'Namespace', falling back to '%s'"), NodeName, *Params.Namespace);
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
bool bHasSandbox = !Params.Sandbox.IsEmpty();
ZenCacheStore - ignore "None" as an override value for a local DDC path when using sandboxes (in line with past behavior for non-default local caches like Warm/Cold). #rb devin.doucette [CL 23658565 by zousar shaker in ue5-main branch]
2023-01-11 20:57:13 -05:00
bool bUseLocalDataCachePathOverrides = !bHasSandbox;
Ensure that Zen as a local service can be configured to follow the same commandline overrides as a filesystem cache store. Also ensure that if the configured override mechanisms for cache store paths yield a path of "None" that we disable the Zen DDC cache. These are in support of using Zen as a drop-in replacement for an existing filesystem DDC store. #rb devin.doucette #preflight 6398ea4e680483bcb1057b0a [CL 23517045 by Zousar Shaker in ue5-main branch]
2022-12-14 15:06:30 -05:00
FString CachePathOverride;
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
if (bUseLocalDataCachePathOverrides && UE::Zen::Private::IsLocalAutoLaunched(Params.Host) && UE::Zen::Private::GetLocalDataCachePathOverride(CachePathOverride))
Ensure that Zen as a local service can be configured to follow the same commandline overrides as a filesystem cache store. Also ensure that if the configured override mechanisms for cache store paths yield a path of "None" that we disable the Zen DDC cache. These are in support of using Zen as a drop-in replacement for an existing filesystem DDC store. #rb devin.doucette #preflight 6398ea4e680483bcb1057b0a [CL 23517045 by Zousar Shaker in ue5-main branch]
2022-12-14 15:06:30 -05:00
{
if (CachePathOverride == TEXT("None"))
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
UE_LOG(LogDerivedDataCache, Log, TEXT("%s: Disabled because path is set to 'None'"), NodeName);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
return nullptr;
Ensure that Zen as a local service can be configured to follow the same commandline overrides as a filesystem cache store. Also ensure that if the configured override mechanisms for cache store paths yield a path of "None" that we disable the Zen DDC cache. These are in support of using Zen as a drop-in replacement for an existing filesystem DDC store. #rb devin.doucette #preflight 6398ea4e680483bcb1057b0a [CL 23517045 by Zousar Shaker in ue5-main branch]
2022-12-14 15:06:30 -05:00
}
}
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
TUniquePtr<FZenCacheStore> Backend;
ZenCacheStore - ignore "None" as an override value for a local DDC path when using sandboxes (in line with past behavior for non-default local caches like Warm/Cold). #rb devin.doucette [CL 23658565 by zousar shaker in ue5-main branch]
2023-01-11 20:57:13 -05:00
if (bHasSandbox)
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
{
Zen::FServiceSettings DefaultServiceSettings;
DefaultServiceSettings.ReadFromConfig();
if (!DefaultServiceSettings.IsAutoLaunch())
{
Fix a typo in a log message. [CL 30100313 by zousar shaker in ue5-main branch]
2023-12-04 18:28:00 -05:00
UE_LOG(LogDerivedDataCache, Warning, TEXT("%s: Attempting to use a sandbox when there is no default autolaunch configured to inherit settings from. Cache will be disabled."), NodeName);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
return nullptr;
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
}
// Make a unique local instance (not the default local instance) of ZenServer
Zen::FServiceSettings ServiceSettings;
ServiceSettings.SettingsVariant.Emplace<Zen::FServiceAutoLaunchSettings>();
Zen::FServiceAutoLaunchSettings& AutoLaunchSettings = ServiceSettings.SettingsVariant.Get<Zen::FServiceAutoLaunchSettings>();
const Zen::FServiceAutoLaunchSettings& DefaultAutoLaunchSettings = DefaultServiceSettings.SettingsVariant.Get<Zen::FServiceAutoLaunchSettings>();
AutoLaunchSettings = DefaultAutoLaunchSettings;
// Default as one more than the default port to not collide. Multiple sandboxes will share a desired port, but will get differing effective ports.
AutoLaunchSettings.DesiredPort++;
FPaths::NormalizeDirectoryName(AutoLaunchSettings.DataPath);
AutoLaunchSettings.DataPath += TEXT("_");
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
AutoLaunchSettings.DataPath += Params.Sandbox;
Fix a collection of bugs related to ZenServer launching: - Fix a message that had the text and caption fields inverted. - Ensure that sandbox instances don't overwrite the .runcontext file used by non-sandbox instances. - Fix a bug where the Zen data path when following DDC path settings wasn't normalizing the path leading to needless shutdown and startup of zenserver. #rb matt.peters [CL 26205402 by zousar shaker in ue5-main branch]
2023-06-23 00:49:34 -04:00
AutoLaunchSettings.bIsDefaultSharedRunContext = false;
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
// The unique local instances will always limit process lifetime for now to avoid accumulating many of them
AutoLaunchSettings.bLimitProcessLifetime = true;
// Flush the cache if requested.
MPCook: Add global function UE::GetMultiprocessId to handle the commandline parsing of MultiprocessId for CookWorkers. #rnx #rb Zousar.Shaker [CL 28938402 by matt peters in ue5-main branch]
2023-10-19 18:18:01 -04:00
uint32 MultiprocessId = UE::GetMultiprocessId();
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
if (Params.bFlush && (MultiprocessId == 0))
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
{
bool bStopped = true;
if (UE::Zen::IsLocalServiceRunning(*AutoLaunchSettings.DataPath))
{
bStopped = UE::Zen::StopLocalService(*AutoLaunchSettings.DataPath);
}
if (bStopped)
{
IFileManager::Get().DeleteDirectory(*(AutoLaunchSettings.DataPath / TEXT("")), /*bRequireExists*/ false, /*bTree*/ true);
}
else
{
UE_LOG(LogDerivedDataCache, Warning, TEXT("%s: Zen DDC could not be flushed due to an existing instance not shutting down when requested."), NodeName);
}
}
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
Backend = MakeUnique<FZenCacheStore>(MoveTemp(ServiceSettings), Params, Owner);
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
}
else
{
Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later. #rb Devin.Doucette [CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
Backend = MakeUnique<FZenCacheStore>(Params, Owner);
Add support for Zen DDC sandboxes which are separate instances of zenserver with separate data paths and can be optionally flushed from run-to-run. It is used to implement Warm and Cold DDC graphs using Zen sandboxes for projects/platforms that use Zen for local DDC by default, and to provide new ZenColdDDC and ZenWarmDDC as new graphs that have Cold/Warm behavior for projects that don't use Zen for local DDC by default. #rb devin.doucette #preflight 63b493d5202bee5e27a645ed [CL 23569648 by Zousar Shaker in ue5-main branch]
2023-01-03 15:58:30 -05:00
}
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
if (!Backend->IsUsable())
{
Downgrade warning severity messages to display severity when we are unable to reach a remote zenserver (as opposed to a local one). #rb Devin.Doucette [CL 29761186 by zousar shaker in ue5-main branch]
2023-11-15 17:17:29 -05:00
if (Backend->IsLocalConnection())
{
UE_LOG(LogDerivedDataCache, Warning, TEXT("%s: Failed to contact the service (%s), will not use it."), NodeName, *Backend->GetName());
}
else
{
UE_LOG(LogDerivedDataCache, Display, TEXT("%s: Failed to contact the service (%s), will not use it."), NodeName, *Backend->GetName());
}
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
Backend.Reset();
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
return nullptr;
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
}
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
return Backend.Release();
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
} // namespace UE::DerivedData
#else
namespace UE::DerivedData
{
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
ILegacyCacheStore* CreateZenCacheStore(const TCHAR* NodeName, const TCHAR* Config, ICacheStoreOwner* Owner)
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
{
DDC: Removed the legacy cache implementation from the clients for Horde Storage and Zen #preflight 62967fd60c151996e13038ec #rb Zousar.Shaker [CL 20445538 by Devin Doucette in ue5-main branch]
2022-05-31 22:01:53 -04:00
UE_LOG(LogDerivedDataCache, Warning, TEXT("%s: Zen cache is not yet supported in the current build configuration."), NodeName);
DDC: Added ICacheStoreStats to the Zen cache #jira UE-133395 #rb Zousar.Shaker #rnx [CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
return nullptr;
Switch Zen cache store to use the HTTP request codepath in the Jupiter cache store while taking some of the improvements that had been made when that code was branched off to make the Zen HTTP codepath. This is a precursor for getting Zen cache store using non-blocking operations like Jupiter cache store - this has started to be stubbed in with the EnqueueAsyncRpc methods that are stubbed in but not used from the Zen cache store. Eventually I'd like this HTTP code to move out of the DerivedDataCache module and be usable elsewhere and supplant the HTTP code currently in the Zen module, but this will require some decisions about whether IRequestOwner should move out of the DerivedDataCache module or not. Tested Zen operation using ShooterGame editor. Tested Jupiter operation using CitySample editor (empty local cache). #rb devin.doucette #fyi stefan.boberg #preflight 62953e65401169f78c57c298 [CL 20433163 by Zousar Shaker in ue5-main branch]
2022-05-30 18:14:02 -04:00
}
} // UE::DerivedData
ZenServer CachePolicy support. Add batched Put and PutValue. Update Batched Get to send the RecordPolicy with each key. Update GetChunks to send the CachePolicy as string and to handle SkipData in the response. Add more unit tests for upstream behavior and GetChunks. Update the CacheRecordPolicy Save/Load to match the new variable names on FCacheRecordPolicy. Deploy new zenserver binaries with the required matching server-side changes. #rb Per.Larsson,Devin.Doucette #rnx #preflight 61f94e5980608c7029b8fc5c #ROBOMERGE-AUTHOR: matt.peters #ROBOMERGE-SOURCE: CL 18808743 in //UE5/Release-5.0/... via CL 18809399 via CL 18822531 #ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v908-18788545) [CL 18823289 by matt peters in ue5-main branch]
2022-02-02 05:33:16 -05:00
#endif // UE_WITH_ZEN
Reference in New Issue Copy Permalink