Commit Graph

9 Commits

Author SHA1 Message Date
Yuriy ODonnell
7389b826b6 unsync - Initial implementation of the "scavenge" mode which aims to reduce download times by reusing data outside of the final sync directory
This is useful for people who want to download each data set into a uniquely named directory, instead of always patching the last downloaded version.

Current implementation is highly experimental and should only be used for testing purposes.

#jira UE-178864
#rb none
#preflight skip

[CL 25190443 by Yuriy ODonnell in ue5-main branch]
2023-04-25 17:41:49 -04:00
Yuriy ODonnell
1702de7c3f unsync - Add available disk check before starting sync, add anonymized machine name to telemetry
#rb none
#preflight none

[CL 22490002 by Yuriy ODonnell in ue5-main branch]
2022-10-12 20:45:24 -04:00
Yuriy ODonnell
2cfe8f70ad unsync - Resolve letter-based source paths to universal form when syncing
#rb Robert.Millar
#preflight skip

[CL 21861981 by Yuriy ODonnell in ue5-main branch]
2022-09-07 16:50:30 -04:00
Robert Millar
73fc9b9a36 Add --include argument to sync subcommand to limit the files synced
Add --files flag to info subcommand to list files in manifests

#jira none
#rb yuriy.odonnell
#preflight there are no horde jobs that build unsync

[CL 21487301 by Robert Millar in ue5-main branch]
2022-08-22 14:00:37 -04:00
Yuriy ODonnell
b59ee6402f unsync - Add --overlay option to sync command to copy files from multiple sources into a single merged target, bump version to 1.0.43
#preflight skip
#rb none

[CL 20997805 by Yuriy ODonnell in ue5-main branch]
2022-07-07 22:20:00 -04:00
Yuriy ODonnell
da7c737ec5 unsync - Enable quick sync mode by default, reduce log verbosity, fix file attribute cache lookup, bump version to 1.0.40
#rb none
#preflight skip

[CL 20351881 by Yuriy ODonnell in ue5-main branch]
2022-05-24 13:55:09 -04:00
Yuriy ODonnell
fa2684b9d7 unsync - Remove std::filesystem namespace alias and add wrappers for various filesystem functions
#rb none
#preflight skip

[CL 19301694 by Yuriy ODonnell in ue5-main branch]
2022-03-08 08:17:08 -05:00
Yuriy ODonnell
41d40191b5 unsync - Fix few badly auto-converted function names
Also fix missing header include

#rb none (trivial)
#preflight skip

[CL 19300753 by Yuriy ODonnell in ue5-main branch]
2022-03-08 05:13:41 -05:00
Yuriy ODonnell
b5709042fb Import Unsync into the main source tree
This is a binary patching and incremental downloading tool, similar to rsync or zsync. It aims to improve the large binary download processes that previously were served by robocopy (i.e. full packages produced by the build farm).

The original code can be found in `//depot/usr/yuriy.odonnell/unsync`. This commit is a branch from the original location to preserve history.

While the codebase is designed to be self-contained and does not depend on any engine libraries, it mostly follows the UE coding guidelines and can be built with UBT.

Currently only Windows is supported, however the tool is expected to also work on Mac and Linux in the future.

#rb Martin.Ridgers
#preflight skip

[CL 18993571 by Yuriy ODonnell in ue5-main branch]
2022-02-15 04:30:27 -05:00