Commit Graph

63 Commits

Author SHA1 Message Date
zousar shaker
557cdf9e71 Initialize an out variable to satisfy static analysis.
[CL 30297745 by zousar shaker in ue5-main branch]
2023-12-13 13:17:25 -05:00
zousar shaker
ae71242205 Change the way readiness of ZenCacheStore is evaluated and re-evaluated.
Instead of relying on a one-time blocking check with no reponse timeout, we now issue a blocking request with a 5 second idle time limit.  If it fails, the store will still be created, but it will go into the same asyncronous re-evaluation loop as when performance is below the acceptable threshold and re-evaluate at 30 second intervals until both:

- Health is Ok
- Performance criteria (if any has been configured and is in use) is met

At which point it is activated.  Health checks have been changed from the health/status endpoint to health/ready because we don't want to act if the server is running but not ready for requests (eg: during the time when it may be wiping data during a schema change.

The overall goal is that we don't force the entire session to execute without zenserver if zenserver was not ready at startup.

#rb Devin.Doucette

[CL 30295095 by zousar shaker in ue5-main branch]
2023-12-13 11:34:21 -05:00
zousar shaker
49d387e6c5 Use the low speed limit with zen cache store client, but with really tolerant threshold (only require 30 bytes over 60 seconds).
#rb dan.engelbrecht, Devin.Doucette

[CL 30265167 by zousar shaker in ue5-main branch]
2023-12-12 11:23:43 -05:00
zousar shaker
93a052ff58 Fix a typo in a log message.
[CL 30100313 by zousar shaker in ue5-main branch]
2023-12-04 18:28:00 -05:00
zousar shaker
81874f5865 Change the way ZenCacheStore configuration and presentation is implemented to align with the pattern used in HttpCacheStore so that we can use a ServerID parameter to refer to a configuration in the StorageServers section of the config while still retaining the ability to override values as needed. No config changes at this point, those will happen later.
#rb Devin.Doucette

[CL 29926397 by zousar shaker in ue5-main branch]
2023-11-25 16:22:26 -05:00
zousar shaker
922be064fe Downgrade warning severity messages to display severity when we are unable to reach a remote zenserver (as opposed to a local one).
#rb Devin.Doucette

[CL 29761186 by zousar shaker in ue5-main branch]
2023-11-15 17:17:29 -05:00
zousar shaker
540ba6901f Add performance disabling (via latency) for ZenCacheStore.
#rb devin.doucette

[CL 29537962 by zousar shaker in ue5-main branch]
2023-11-07 17:56:15 -05:00
zousar shaker
cce1a015d6 Fix memory leak caused by reference counted objects capturing references to themselves in function captures held on members of itself. This manifested as memory never being freed when performing some operations with HttpCacheStore and ZenCacheStore.
#rb devin.doucette

[CL 29316880 by zousar shaker in ue5-main branch]
2023-11-01 10:33:43 -04:00
matt peters
e53bae0868 MPCook: Add global function UE::GetMultiprocessId to handle the commandline parsing of MultiprocessId for CookWorkers.
#rnx
#rb Zousar.Shaker

[CL 28938402 by matt peters in ue5-main branch]
2023-10-19 18:18:01 -04:00
zousar shaker
2a2910744f Downgrade messages from ZenCacheStore from Warning to Display as they can be expected to come up due to network conditions or temporary outages and do not break functionality.
#rb devin.doucette

[CL 28357691 by zousar shaker in ue5-main branch]
2023-09-29 17:36:33 -04:00
zousar shaker
8e368cea22 Fix bug in Zen DDC client. Don't assume that if the data hash of a response is available as an attachment that it should be included in the reponse. If the request is skipping data, then even if the attachment is present (eg: from another request in the batch), it should not be included when processing the request that had SkipData.
#rb dan.engelbrecht

[CL 28187457 by zousar shaker in ue5-main branch]
2023-09-25 10:31:36 -04:00
dan engelbrecht
e517a3bc58 Add CPU time and latency stats to ZenCacheStore
Fix LogicalReadSize for ZenCacheStore GetChunks
Add missing fields for GatherAnalytics in ZenServerInterface

#rb devin.doucette

[CL 28028176 by dan engelbrecht in ue5-main branch]
2023-09-20 02:58:55 -04:00
zousar shaker
d4e2723732 Fix handling of cancelation in Zen DDC client to avoid having it treated as equivalent to an unexpected error.
#rb devin.doucette

[CL 27969813 by zousar shaker in ue5-main branch]
2023-09-18 15:22:34 -04:00
zousar shaker
8c75ff393b ZenDashboard refresh to add project stats and clean up reporting of disk size.
#rb dan.engelbrecht

[CL 27546624 by zousar shaker in ue5-main branch]
2023-08-31 22:30:53 -04:00
devin doucette
402d04cde9 DDC: Added request name to FCacheStoreRequestStats
#jira UE-133395
#rb Zousar.Shaker
#rnx

[CL 26210120 by devin doucette in ue5-main branch]
2023-06-23 10:51:52 -04:00
zousar shaker
f92dc5c090 Fix a collection of bugs related to ZenServer launching:
- Fix a message that had the text and caption fields inverted.
- Ensure that sandbox instances don't overwrite the .runcontext file used by non-sandbox instances.
- Fix a bug where the Zen data path when following DDC path settings wasn't normalizing the path leading to needless shutdown and startup of zenserver.

#rb matt.peters

[CL 26205402 by zousar shaker in ue5-main branch]
2023-06-23 00:49:34 -04:00
dan engelbrecht
46b88104b8 Return error if we receive a partial record from zenserver
#rb devin.doucette
#prefligth 648a3d2f88385dd88cfdda4a

[CL 26006460 by dan engelbrecht in ue5-main branch]
2023-06-15 04:54:40 -04:00
dan engelbrecht
a46531dd35 Make sure record policy is set up correctly in batch requests to Zen
#rb devin.doucette

[CL 25982752 by dan engelbrecht in ue5-main branch]
2023-06-14 12:31:13 -04:00
devin doucette
ab371b64e2 DDC: Fixed leaking cache store stats in the Zen cache
#jira UE-133395
#rb none
#rnx

[CL 25977512 by devin doucette in ue5-main branch]
2023-06-14 10:12:47 -04:00
devin doucette
d3b72ca4be DDC: Added ICacheStoreStats to the Zen cache
#jira UE-133395
#rb Zousar.Shaker
#rnx

[CL 25964028 by devin doucette in ue5-main branch]
2023-06-13 17:14:59 -04:00
zousar shaker
b7989e78f3 Fix crash when using -ddc=Cold on projects using ZenServer as Local DDC and Multi-Process Cook simultaneously.
#rb devin.doucette

[CL 25649292 by zousar shaker in ue5-main branch]
2023-05-26 13:52:11 -04:00
dan engelbrecht
97a2f5064b Don't continue initialization of ZenCacheStore of backend is deemed unusable (unable to connect)
#rb per.larsson
#preflight 646cb0a22e78be9841ff6463

[CL 25580746 by dan engelbrecht in ue5-main branch]
2023-05-23 08:45:48 -04:00
dan engelbrecht
9388d8e1bd Check status code for responses received from zenserver DDC requests.
#rb stefan.boberg
#preflight 645a3339013f3751584a39ed

[CL 25385641 by dan engelbrecht in ue5-main branch]
2023-05-09 08:08:07 -04:00
devin doucette
8ff0cd05ca DDC: Added EHttpVersion to HttpClient
#rb Zousar.Shaker
#rnx

[CL 25184555 by devin doucette in ue5-main branch]
2023-04-25 13:54:42 -04:00
zousar shaker
2a4b4cdaf5 Change Zen DDC cache store to allow:
- Local storage flag to be inferred by whether the service is run locally or set explicitly in config.
- Remote storage flag to be inferred by whether the service is NOT run locally or set explicitly in config.
- ReadOnly storage flag to be set explicitly in config (defaults to true).

#rb devin.doucette

[CL 24822351 by zousar shaker in ue5-main branch]
2023-03-28 13:43:50 -04:00