* Once a log is retrieved, server adds an entry with this log id to a sorted set in Redis, scored by expiry time and broadcasts it to any other server instances.
* If a log is being tailed, server keeps the total number of lines in a Redis key and appends chunks of log data to a sorted set for that log scored by index of the first line.
* Chunks are split on fixed boundaries, in order to allow older entries to be purged by score without having to count the number of lines they contain first.
* Agent polls for requests to provide tail data via LogRpcService.UpdateLogTail, which calls LogTailService.WaitForTailNext. WaitForTailNext returns the index of the total line count for this log if it is being tailed (indicating the index of the next line that the agent should send to the server), or blocks until the log is being tailed.
* Once data is flushed to persistent storage, the number of flushed lines is added to LogRpcService._trimQueue and the line data is removed from Redis after LogRpcService.TrimAfter.
* Log node contains a "complete" flag indicating whether it is necessary to check for tail data.
#preflight none
[CL 23215856 by Ben Marsh in ue5-main branch]
Only expose ConnectionPool from RedisService to avoid using Database or ConnectionMultiplexer directly.
#preflight 62f2661d3b773d041627aad0
[CL 21286130 by carl bystrom in ue5-main branch]
#fyi joe.kirchoff
Original CL Desc
-----------------------------------------------------------------
EpicGames.Redis: net6.0 upgrade (with warnings)
Remaining warnings:
RedisChannel.cs(15,25,15,37): warning CA1066: Type EpicGames.Redis.RedisChannel<T> should implement IEquatable<T> because it overrides Equals
Utility\RedisLock.cs(14,15,14,24): warning CA1063: Provide an overridable implementation of Dispose(bool) on 'RedisLock' or mark the type as sealed. A call to Dispose(false) should only clean up native resources. A call to Dispose(true) should clean up both managed and native resources.
Utility\RedisLock.cs(39,26,39,38): warning CA1816: Change RedisLock.DisposeAsync() to call GC.SuppressFinalize(object). This will prevent derived types that introduce a finalizer from needing to re-implement 'IDisposable' to call it.
Utility\RedisLock.cs(79,11,79,96): warning CA2008: Do not create tasks without passing a TaskScheduler
Utility\RedisLock.cs(33,15,33,22): warning CA1063: Modify 'RedisLock.Dispose' so that it calls Dispose(true), then calls GC.SuppressFinalize on the current object instance ('this' or 'Me' in Visual Basic), and then returns
Utility\RedisLock.cs(33,15,33,22): warning CA1816: Change RedisLock.Dispose() to call GC.SuppressFinalize(object). This will prevent derived types that introduce a finalizer from needing to re-implement 'IDisposable' to call it.
#rnx
#rb none
#preflight 629e7910cb76a3f715a5cc4f
[CL 20540282 by joe kirchoff in ue5-main branch]
Remaining warnings:
RedisChannel.cs(15,25,15,37): warning CA1066: Type EpicGames.Redis.RedisChannel<T> should implement IEquatable<T> because it overrides Equals
Utility\RedisLock.cs(14,15,14,24): warning CA1063: Provide an overridable implementation of Dispose(bool) on 'RedisLock' or mark the type as sealed. A call to Dispose(false) should only clean up native resources. A call to Dispose(true) should clean up both managed and native resources.
Utility\RedisLock.cs(39,26,39,38): warning CA1816: Change RedisLock.DisposeAsync() to call GC.SuppressFinalize(object). This will prevent derived types that introduce a finalizer from needing to re-implement 'IDisposable' to call it.
Utility\RedisLock.cs(79,11,79,96): warning CA2008: Do not create tasks without passing a TaskScheduler
Utility\RedisLock.cs(33,15,33,22): warning CA1063: Modify 'RedisLock.Dispose' so that it calls Dispose(true), then calls GC.SuppressFinalize on the current object instance ('this' or 'Me' in Visual Basic), and then returns
Utility\RedisLock.cs(33,15,33,22): warning CA1816: Change RedisLock.Dispose() to call GC.SuppressFinalize(object). This will prevent derived types that introduce a finalizer from needing to re-implement 'IDisposable' to call it.
#rnx
#rb none
#preflight 629e7910cb76a3f715a5cc4f
[CL 20528968 by joe kirchoff in ue5-main branch]
Two sets of data are returned - one detailing the timeline of the task on the server (being queued, through dispatched, to completed), and one detailing the timeline of the task on the remote agent (downloading data, executing the task, and uploading the result).
The standalone compute command implementation in Horde.Agent also logs its own timeline of events.
#preflight none
[CL 19675739 by Ben Marsh in ue5-main branch]
- Each tool can have a number of deployments which are rolled out over time. Clients can specify a "phase" value indicating where in the queue they want to take updates (ie. whether they want to be early- or late-adopters), and deployments can be paused and cancelled.
- Tool channels are configured through the global configuration file. Tools can be marked "public", meaning that they do not require authentication against Horde for access. This can be useful for auto-update scenarios.
- The most recent 5 deployments are kept in the tool at any time.
The collection and controller implementation here is an experiment in reducing the amount of boilerplate currently adopted as a pattern in Horde. Notably:
- Model and collection classes are concrete rather than interfaces (since we don't generally mock them anyway).
- The same model class with annotations is used for response objects (paving the way to supporting patching and filtering using the same public model as the internal model).
- The VersionedCollection class is used to handle automatic migration of documents across schema versions, and cache document values with Redis.
- MongoDB property names are explicit and short, rather than just taking variable names.
#preflight none
[CL 19636699 by Ben Marsh in ue5-main branch]
