When FileNode is marked as compressed, the appropriate method for download is used as compressed blobs are references by their uncompressed hash.
Files are still downloaded as-is and are not decompressed by the agent executor.
Flagging compression for files during upload is still left to be implemented.
#preflight 6287b26d5b568e5761f9a1a1
[CL 20295190 by carl bystrom in ue5-main branch]
New AsSegment() method allows creating a segment, while explicit extension methods now exist for explicitly slicing List<T>.
#preflight 627dddaa830b40b66810acd3
[CL 20178032 by Ben Marsh in ue5-main branch]
Reads are typically lock free. Writes require a lock but otherwise complete quickly and in constant time. The store is transactional, and is designed not to lose data if the process is terminated between flushes. To accomplish this, blocks are not reused until the store is flushed via a call to Save().
Blobs allocated in the store may be split into multiple cells, so do not support access to a contiguous underlying memory region. The cells comprising a blob may be enumerated and accessed individually via a handle to the blob.
The store also supports storage of tree objects which can be addressed by hash. Internally, these objects are reference counted and maintain a list of references to other blobs separately to their payload.
If the store runs out of space, supplemental stores may be allocated in granularity of <see cref="LocalBlobStoreOptions.ExpandStepSize"/>. These stores are merged into the main one during the next Save() call.
#preflight none
#preflight 625872ee7f628a9018e480e5
[CL 19761303 by Ben Marsh in ue5-main branch]
Two sets of data are returned - one detailing the timeline of the task on the server (being queued, through dispatched, to completed), and one detailing the timeline of the task on the remote agent (downloading data, executing the task, and uploading the result).
The standalone compute command implementation in Horde.Agent also logs its own timeline of events.
#preflight none
[CL 19675739 by Ben Marsh in ue5-main branch]
- Node state is now separated into a separate class to allow atomic transitions between imported/exported/standalone states.
- A semaphore is used to limit concurrent reads from underlying storage, so bundles now implement IDisposable.
- DirectoryNode.CopyToDirectoryAsync() spawns multiple tasks to extract data in parallel. The current tree traversal is fairly naive; since it doesn't know which objects are currently in the cache, it can result in a lot of thrashing if the cache is full as objects are purged and re-read to match the traversal order.
- Mutation of the tree is still restricted to a single thread.
#preflight none
[CL 19405679 by Ben Marsh in ue5-main branch]
- Fixed tracking of parent ownership for child nodes when adding new nodes to the tree.
- Fixed incorrect offsets in exports when multiple items are in a single compression packet.
- Document and enforce invariants when switching nodes between different states. Nodes read from refs are now immediately turned into standalone nodes. Removes unnecessary logic for caching/decompressing nodes in the root ref on demand.
#preflight none
[CL 19368974 by Ben Marsh in ue5-main branch]
