Commit Graph

42 Commits

Author SHA1 Message Date
yuriy odonnell
abf5a85ab6 unsync - Fix various runtime issues on linux
* Wide string formatting
* Path normalization

#rb none

[CL 27856783 by yuriy odonnell in ue5-main branch]
2023-09-13 18:33:59 -04:00
yuriy odonnell
4fd271fff2 unsync - Fix clang/linux build errors
#rb none

[CL 27821338 by yuriy odonnell in ue5-main branch]
2023-09-13 02:10:23 -04:00
yuriy odonnell
a3bd8e9bb6 unsync - Add messages in addition to error codes in various places, fix CLI11 non-ascii printing, add --version argument
#rb none

[CL 27781280 by yuriy odonnell in ue5-main branch]
2023-09-11 22:49:17 -04:00
yuriy odonnell
a02ea83d83 unsync - Use multiple concurrent connections during file download query
* Bump version to 1.0.54

#rb none

[CL 27714879 by yuriy odonnell in ue5-main branch]
2023-09-08 12:19:57 -04:00
yuriy odonnell
f455aaecc7 unsync - Add support for HTTP HEAD requests
Also minor fixes and improvements:

* Fix error code handling when connecting a socket
* Fix some of the naming convention violations
* Add --print-http-header option to login command

#rb none

[CL 27702511 by yuriy odonnell in ue5-main branch]
2023-09-07 23:20:15 -04:00
yuriy odonnell
7a75fdfe69 unsync - Trim old cached manifest files during sync
#rb none

[CL 27541251 by yuriy odonnell in ue5-main branch]
2023-08-31 18:07:30 -04:00
yuriy odonnell
01b2df4dba unsync - Add quick login option
* Skip remote token validation in --quick mode, just check expiry locally

#rb none

[CL 27538554 by yuriy odonnell in ue5-main branch]
2023-08-31 17:15:46 -04:00
yuriy odonnell
8c7bdecb04 unsync - Add support for direct manifest download from proxy server (bypassing SMB)
* Add --login flag to sync command
* Add in-process token cache
* Add auth token when quering server features
* Use direct download path when supported by server

#jira UE-192913
#rb none

[CL 27535921 by yuriy odonnell in ue5-main branch]
2023-08-31 16:20:09 -04:00
yuriy odonnell
74e05813ad unsync - Skip automatic proxy selection for query command, use dedicated ping endpoint, refactor query helpers and download result mechanism
#rb none

[CL 27153277 by yuriy odonnell in ue5-main branch]
2023-08-16 16:34:36 -04:00
yuriy odonnell
c45fdca170 unsync - Automatically find closest proxy server by default
* Use --no-proxy-select to skip server selection
* Bump version to 1.0.52

#rb none

[CL 26157514 by yuriy odonnell in ue5-main branch]
2023-06-21 15:01:35 -04:00
Yuriy ODonnell
f4a7dc800b unsync - Improved manifest info command
* Allow --include and --exclude options
* Compute approximate patch size from manifest A to B

#rb none
#preflight skip

[CL 25437799 by Yuriy ODonnell in ue5-main branch]
2023-05-11 18:34:56 -04:00
Yuriy ODonnell
bb51087f42 unsync - Finish up --scavenge mode implementation, bump version to 1.0.51
* Count scavenged blocks as "base" data for stats/telemetry
* Gracefully handle missing/invalid scavenge root path
* Cancel reading blocks from a particular source if a hash mismatch was found

#rb none
#jira UE-178864
#preflight skip

[CL 25326911 by Yuriy ODonnell in ue5-main branch]
2023-05-03 17:28:38 -04:00
Yuriy ODonnell
eca8ba2215 unsync - Deduplicate blocks within the same file while building scavenge database
* Log progress when scavenging blocks from local files
* Move prorress logging helper code to dedicated files

#rb none
#preflight skip

[CL 25247737 by Yuriy ODonnell in ue5-main branch]
2023-04-28 20:35:07 -04:00
Yuriy ODonnell
7389b826b6 unsync - Initial implementation of the "scavenge" mode which aims to reduce download times by reusing data outside of the final sync directory
This is useful for people who want to download each data set into a uniquely named directory, instead of always patching the last downloaded version.

Current implementation is highly experimental and should only be used for testing purposes.

#jira UE-178864
#rb none
#preflight skip

[CL 25190443 by Yuriy ODonnell in ue5-main branch]
2023-04-25 17:41:49 -04:00
Yuriy ODonnell
ee7d1f6a92 unsync - Fix global progress reporting
#rb none
#preflight skip

[CL 25153462 by Yuriy ODonnell in ue5-main branch]
2023-04-21 16:54:22 -04:00
Yuriy ODonnell
40d4991232 unsync - Download small files in batches to reduce amount of network requests
* This improves download performance of data sets with large numbers of small files due to reduced impact of latency
* Small files are grouped into ~4MB batches

#jira UE-175966
#preflight skip
#rb robert.millar

[CL 25150472 by Yuriy ODonnell in ue5-main branch]
2023-04-21 14:43:23 -04:00
Yuriy ODonnell
7820db9550 unsync - Bump version to 1.0.50
#rb none
#preflight skip

[CL 24903755 by Yuriy ODonnell in ue5-main branch]
2023-04-03 19:10:25 -04:00
Yuriy ODonnell
0e88e06c7c unsync - Files that are excluded from sync via --exclude [string] argument will now be deleted
* This removes any leftover files from a previous sync that may have used a different exclusion filter
* Stale files in some cases may cause errors (for example, serialization scheme may change and new executables may no longer be compatible with old data)
* If necessary, --cleanup-exclude [string] may still be used to explicitly prevent deletion of specific stale files
* Improved `info` command (which displays basic information about a manifest file) to detect if input path is a directory or a manifest file itself

#rb none
#preflight skip

[CL 24885471 by Yuriy ODonnell in ue5-main branch]
2023-04-01 18:48:15 -04:00
Yuriy ODonnell
e9d32e1c81 unsync - Fix --full-diff option, bump version to 1.0.49
#rb none
#preflight skip

[CL 24385847 by Yuriy ODonnell in ue5-main branch]
2023-02-23 14:14:52 -05:00
Yuriy ODonnell
f89d532f08 unsync - Workaround for ICE when using MSVC toolchain 14.34.31942
#rb none (trivial)
#preflight skip

[CL 24351444 by Yuriy ODonnell in ue5-main branch]
2023-02-21 17:29:00 -05:00
Yuriy ODonnell
1702de7c3f unsync - Add available disk check before starting sync, add anonymized machine name to telemetry
#rb none
#preflight none

[CL 22490002 by Yuriy ODonnell in ue5-main branch]
2022-10-12 20:45:24 -04:00
Yuriy ODonnell
2cfe8f70ad unsync - Resolve letter-based source paths to universal form when syncing
#rb Robert.Millar
#preflight skip

[CL 21861981 by Yuriy ODonnell in ue5-main branch]
2022-09-07 16:50:30 -04:00
Robert Millar
0b02de6b01 Bump unsync.exe version
#fyi yuriy.odonnell
#jira none
#preflight none

[CL 21488301 by Robert Millar in ue5-main branch]
2022-08-22 14:34:29 -04:00
Robert Millar
73fc9b9a36 Add --include argument to sync subcommand to limit the files synced
Add --files flag to info subcommand to list files in manifests

#jira none
#rb yuriy.odonnell
#preflight there are no horde jobs that build unsync

[CL 21487301 by Robert Millar in ue5-main branch]
2022-08-22 14:00:37 -04:00
Yuriy ODonnell
1ef1ccee59 unsync - Add a way to exclude certain files/directories from cleanup, bump version to 1.0.45
* Use --cleanup-exclude [comma,separated,words] option to skip deleting specific files/directories after sync (such as saved logs, configs, etc.)
* Use UNSYNC_CLEANUP_EXCLUDE environment variable to provide extra exclusion rules

#rb none
#preflight skip

[CL 21226431 by Yuriy ODonnell in ue5-main branch]
2022-07-22 14:45:34 -04:00