Commit Graph

177 Commits

Author SHA1 Message Date
Joakim Lindqvist
a7ce9dd454 Unreal Cloud DDC - Missing file that resolves test building error
#rb none

[CL 28340314 by Joakim Lindqvist in ue5-main branch]
2023-09-29 09:09:04 -04:00
Joakim Lindqvist
5cafdd47ff Unreal Cloud DDC - Added open telemetry metric for number of considered refs for GC.
#rb none

[CL 28338706 by Joakim Lindqvist in ue5-main branch]
2023-09-29 06:37:45 -04:00
Joakim Lindqvist
1dd3133dc3 Unreal Cloud DDC - Fixed issue with being unable to determine current region in some worker processes occasionally.
#rb none

[CL 28338696 by Joakim Lindqvist in ue5-main branch]
2023-09-29 06:36:08 -04:00
Joakim Lindqvist
b40cfd19c2 Unreal Cloud DDC - Added namespace to metrics collected by the metrics service
#rb none

[CL 28338493 by Joakim Lindqvist in ue5-main branch]
2023-09-29 06:13:30 -04:00
Joakim Lindqvist
77f9f4ddde Unreal Cloud DDC - Bumped batch size when cleaning the filesystem store, for most nodes this doesn't matter as they will only cleanup as many files as needed to reach the target. But for really large nodes, and esspecially if they then do see a lot of new files all the time, this low number can result in the GC never reaching target and thus the cache grows to large over time.
#rb none

[CL 28267925 by Joakim Lindqvist in ue5-main branch]
2023-09-27 06:08:17 -04:00
Joakim Lindqvist
d9e8de109d Unreal Cloud DDC - Made the metrics service more resillient to errors as its likely to encounter some malformed data when scanning through everything like this.
#rb none

[CL 28267765 by Joakim Lindqvist in ue5-main branch]
2023-09-27 05:57:06 -04:00
Joakim Lindqvist
072fa88ada Unreal Cloud DDC - Improved logging when we are unable to determine if a blob exists remotley due to missing settings, also avoid reading the settings if we can be certain it exists somewhere.
#rb none

[CL 28267687 by Joakim Lindqvist in ue5-main branch]
2023-09-27 05:54:22 -04:00
Joakim Lindqvist
f72e1d50eb Unreal Cloud DDC - Change order that refs are cleaned up, start with the old refs and if its considered alive then check if its finalzied (as that requires another db operation to determine) - speeds up GC and reduces pressure on DB.
#rb none

[CL 28267659 by Joakim Lindqvist in ue5-main branch]
2023-09-27 05:52:30 -04:00
Joakim Lindqvist
85109a5709 Unreal Cloud DDC - Ignore missing refs when trying to cleanup non-finalized refs (these are cleaned up later)
#rb none

[CL 28226896 by Joakim Lindqvist in ue5-main branch]
2023-09-26 10:11:43 -04:00
Joakim Lindqvist
f0577c8394 Unreal Cloud DDC - Delete non-finalized refs that are older then 2 hours during ref GC
#rb none

[CL 28222071 by Joakim Lindqvist in ue5-main branch]
2023-09-26 08:18:00 -04:00
Joakim Lindqvist
449c4ea62f Unreal Cloud DDC - polished some logging and error handling from running metrics calculations
#rb none

[CL 28221995 by Joakim Lindqvist in ue5-main branch]
2023-09-26 08:14:00 -04:00
Joakim Lindqvist
3bda16c9b5 Unreal Cloud DDC - Added test that we can enumerate buckets without throwing and fixed issue with it (missing parameter ctor)
#rb none

[CL 28220915 by Joakim Lindqvist in ue5-main branch]
2023-09-26 06:55:29 -04:00
Joakim Lindqvist
338c27145b Unreal Cloud DDC - Added missing tests for metrics service
#rb none

[CL 28220825 by Joakim Lindqvist in ue5-main branch]
2023-09-26 06:45:15 -04:00
Joakim Lindqvist
ac205b30b4 Unreal Cloud DDC - Further fixes to issues listing buckets during metrics calculations
#rb none

[CL 28220495 by Joakim Lindqvist in ue5-main branch]
2023-09-26 06:18:26 -04:00
Joakim Lindqvist
1e4abf7842 Unreal Cloud DDC - Fixed issue listing ref buckets in Scylla
#rb none

[CL 28220097 by Joakim Lindqvist in ue5-main branch]
2023-09-26 05:54:57 -04:00
Ben Marsh
977dd674d2 Horde: Fix presubmit tests.
[CL 28200228 by Ben Marsh in ue5-main branch]
2023-09-25 15:24:54 -04:00
Ben Marsh
eac890dc3d UnrealCloudDDC: Add an analyze build configuration which outputs warnings at build time. Fix some fussy violations (mainly unused using directives).
#fyi Joakim.Lindqvist

[CL 28193854 by Ben Marsh in ue5-main branch]
2023-09-25 13:29:04 -04:00
Joakim Lindqvist
5a13dbcb9a Unreal Cloud DDC - Delete last access tracking of object when we are deleting the object. Fixes issue were we attempt to delete object that doesn't exist.
#rb none

[CL 28185727 by Joakim Lindqvist in ue5-main branch]
2023-09-25 09:18:06 -04:00
Ben Marsh
3766809fbc UnrealCloudDDC: Fix build errors caused by EpicGames.Horde changes.
[CL 28185467 by Ben Marsh in ue5-main branch]
2023-09-25 09:03:25 -04:00
Joakim Lindqvist
4a1afcfbce Unreal Cloud DDC - Specifying version of newtonsoft as we get this from a dependency and it should be a newer version then what we do get.
#rb none

[CL 28133446 by Joakim Lindqvist in ue5-main branch]
2023-09-22 09:46:27 -04:00
Joakim Lindqvist
0cf4ea1b5a Unreal Cloud DDC - Further disabling of tests to get a build thru
#rb none

[CL 28131604 by Joakim Lindqvist in ue5-main branch]
2023-09-22 08:32:16 -04:00
Joakim Lindqvist
4a0890ffb8 Unreal Cloud DDC - Temporarily disabled some code to handle bundles to be able to get a new build out.
#rb none

[CL 28131319 by Joakim Lindqvist in ue5-main branch]
2023-09-22 08:23:16 -04:00
Joakim Lindqvist
30988e83be Unreal Cloud DDC - Fixed build error with GetStats not being implemented
#rb none

[CL 28130752 by Joakim Lindqvist in ue5-main branch]
2023-09-22 08:05:34 -04:00
Ben Marsh
71f5476473 Fix Jupiter tests.
[CL 28083591 by Ben Marsh in ue5-main branch]
2023-09-21 08:35:43 -04:00
Ben Marsh
3c0aeb2d2b UnrealCloudDDC: Fix build errors.
[CL 28083425 by Ben Marsh in ue5-main branch]
2023-09-21 08:31:47 -04:00