Commit Graph

24 Commits

Author SHA1 Message Date
Yuriy ODonnell
146aec0adb unsync - Load trusted certificates from unsync.cer if it exists next to unsync.exe
#preflight skip

[CL 21261258 by Yuriy ODonnell in ue5-main branch]
2022-08-05 18:51:23 -04:00
Yuriy ODonnell
1ef1ccee59 unsync - Add a way to exclude certain files/directories from cleanup, bump version to 1.0.45
* Use --cleanup-exclude [comma,separated,words] option to skip deleting specific files/directories after sync (such as saved logs, configs, etc.)
* Use UNSYNC_CLEANUP_EXCLUDE environment variable to provide extra exclusion rules

#rb none
#preflight skip

[CL 21226431 by Yuriy ODonnell in ue5-main branch]
2022-07-22 14:45:34 -04:00
Yuriy ODonnell
0bed95e711 unsync - Use TLS during handshake if running with --tls option, skip all certificate validation with --insecure option
#preflight skip

[CL 21066879 by Yuriy ODonnell in ue5-main branch]
2022-07-12 22:10:46 -04:00
Yuriy ODonnell
10b0e773a3 unsync - Fix crash when initial proxy handshake fails
Also use plain (non-TLS) connection for handshake by default

#preflight none
#rb none

[CL 21039956 by Yuriy ODonnell in ue5-main branch]
2022-07-11 10:31:33 -04:00
Yuriy ODonnell
72d8aecd98 unsync - Add client version string to telemetry data
#preflight skip
#rb none

[CL 20998114 by Yuriy ODonnell in ue5-main branch]
2022-07-07 22:34:43 -04:00
Yuriy ODonnell
b59ee6402f unsync - Add --overlay option to sync command to copy files from multiple sources into a single merged target, bump version to 1.0.43
#preflight skip
#rb none

[CL 20997805 by Yuriy ODonnell in ue5-main branch]
2022-07-07 22:20:00 -04:00
Yuriy ODonnell
15ae7848a8 unsync - Send basic sync stat telemetry to the server (if server supports it)
#preflight skip

[CL 20938659 by Yuriy ODonnell in ue5-main branch]
2022-07-04 23:38:13 -04:00
Yuriy ODonnell
4b06bf80b0 unsync - NormalizeFilenameUtf8 now also makes path absolute
#preflight skip

[CL 20938650 by Yuriy ODonnell in ue5-main branch]
2022-07-04 23:36:49 -04:00
Yuriy ODonnell
481883b4a3 unsync - Add a special case for utf8<->wide string conversion when all characters are ASCII
#rb none
#preflight skip

[CL 20924899 by Yuriy ODonnell in ue5-main branch]
2022-07-01 22:15:28 -04:00
Yuriy ODonnell
175fc7e33c unsync - Fix compile errors on Linux
#rb none
#preflight skip

[CL 20924894 by Yuriy ODonnell in ue5-main branch]
2022-07-01 22:14:07 -04:00
Yuriy ODonnell
81ef9ed72e unsync - Handle local file name case differences during sync, bump version to 1.0.42
When running on case-insensitive file systems, local files and directories will be renamed to match the case specified in remote directory manifest.

#rb Dave.Barrett
#preflight skip

[CL 20642233 by Yuriy ODonnell in ue5-main branch]
2022-06-14 01:41:56 -04:00
Yuriy ODonnell
22e1d59a0f sync - Change query mirrors command output to use integer milliseconds formatting for ping times, bump version to 1.0.41
#rb none
#preflight skip

[CL 20407486 by Yuriy ODonnell in ue5-main branch]
2022-05-28 10:41:29 -04:00
Yuriy ODonnell
da7c737ec5 unsync - Enable quick sync mode by default, reduce log verbosity, fix file attribute cache lookup, bump version to 1.0.40
#rb none
#preflight skip

[CL 20351881 by Yuriy ODonnell in ue5-main branch]
2022-05-24 13:55:09 -04:00
Yuriy ODonnell
d61913c81a unsync - Suspend and resume sync task dispatcher using condition variable when scheduling a new task is not possible instead of using a poll loop
Also fix the background task memory usage estimation.

#rb Rex.Hill
#preflight skip

[CL 20298829 by Yuriy ODonnell in ue5-main branch]
2022-05-20 13:55:32 -04:00
Yuriy ODonnell
f1e8eddb1f unsync - Check directory manifest entry block size and mtime fields and report errors if invalid entries are found
#preflight skip

[CL 20288973 by Yuriy ODonnell in ue5-main branch]
2022-05-19 19:18:28 -04:00
Yuriy ODonnell
738c0f421c unsync - Handle long paths on Windows, bump version to 1.0.39
#rb none
#preflight skip

[CL 19333815 by Yuriy ODonnell in ue5-main branch]
2022-03-10 03:23:41 -05:00
Yuriy ODonnell
fa2684b9d7 unsync - Remove std::filesystem namespace alias and add wrappers for various filesystem functions
#rb none
#preflight skip

[CL 19301694 by Yuriy ODonnell in ue5-main branch]
2022-03-08 08:17:08 -05:00
Yuriy ODonnell
41d40191b5 unsync - Fix few badly auto-converted function names
Also fix missing header include

#rb none (trivial)
#preflight skip

[CL 19300753 by Yuriy ODonnell in ue5-main branch]
2022-03-08 05:13:41 -05:00
Yuriy ODonnell
29db42b0dd unsync - Add query command, which can be used to get information from the server (such as list mirrors)
Also implement header-only HTTP response handling.

#rb Martin.Ridgers
#preflight skip

[CL 19196474 by Yuriy ODonnell in ue5-main branch]
2022-03-01 05:53:29 -05:00
Yuriy ODonnell
44ee3bd688 unsync - Fix few minor build issues
* Suppress warnings in CLI11 headers
* Use WIN32_LEAN_AND_MEAN=1 and include specific headers when they're needed
Some windows headers (at least in some versions of the SDKs) do not pop the warning suppression states, which causes inconsistent warning reporting whenever Windows.h is included.
By adding WIN32_LEAN_AND_MEAN, we avoid the issue at least in the common case.
* Suppress various deprecation warnings in Windows headers (_SILENCE_CXX17_OLD_ALLOCATOR_MEMBERS_DEPRECATION_WARNING, _WINSOCK_DEPRECATED_NO_WARNINGS, _SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING, _CRT_SECURE_NO_WARNINGS)
* Fix blake3 include path when using cmake

This fixes compilation warnings when using Windows SDK 10.0.22000.0.

#rb Martin.Ridgers
#preflight 6216532d0f71e491cc023eec

[CL 19111912 by Yuriy ODonnell in ue5-main branch]
2022-02-24 06:20:58 -05:00
Yuriy ODonnell
53695fbd49 unsync - Fix compile errors on Mac
#rb Martin.Ridgers
#preflight skip

[CL 19070501 by Yuriy ODonnell in ue5-main branch]
2022-02-22 05:37:15 -05:00
Yuriy ODonnell
bae920b60e unsync - Fix build errors on Linux, remove the bundled blake3 version (use existing UE package)
#rb Martin.Ridgers
#preflight skip

[CL 19069588 by Yuriy ODonnell in ue5-main branch]
2022-02-22 02:09:06 -05:00
Yuriy ODonnell
a25c902605 unsync - Rename weak hash ban list variable
#rb none (trivial)
#preflight skip

[CL 18994082 by Yuriy ODonnell in ue5-main branch]
2022-02-15 06:14:58 -05:00
Yuriy ODonnell
b5709042fb Import Unsync into the main source tree
This is a binary patching and incremental downloading tool, similar to rsync or zsync. It aims to improve the large binary download processes that previously were served by robocopy (i.e. full packages produced by the build farm).

The original code can be found in `//depot/usr/yuriy.odonnell/unsync`. This commit is a branch from the original location to preserve history.

While the codebase is designed to be self-contained and does not depend on any engine libraries, it mostly follows the UE coding guidelines and can be built with UBT.

Currently only Windows is supported, however the tool is expected to also work on Mac and Linux in the future.

#rb Martin.Ridgers
#preflight skip

[CL 18993571 by Yuriy ODonnell in ue5-main branch]
2022-02-15 04:30:27 -05:00