Commit Graph

93 Commits

Author SHA1 Message Date
yuriy odonnell
c65a2c0128 unsync - Add query http-get <url> command to easily issue use Horde and Unsync HTTP APIs
* Add extended error messages to various places

[CL 34701319 by yuriy odonnell in ue5-main branch]
2024-06-27 00:49:36 -04:00
yuriy odonnell
accfc7e7fe unsync - Improve error checking and handling
* Handle most errors during Horde block download as non-fatal (allow other files to be downloaded if one file hits an error)
* Account for block cache creation time when logging individual file download rate
* Add explicit EDownloadRetryMode::Disconnect code to prevent subsequent server requests, distinct from from Abort code
* Propagate errors from few more places when decoding auth tokens

[CL 34514495 by yuriy odonnell in ue5-main branch]
2024-06-19 17:14:41 -04:00
yuriy odonnell
31b376099f unsync - Add --no-compression command line argument to prefer uncompressed block downloads when possible
* Fix block size validation when downloading from Horde without compression
* Add more explicit error reporting when attempting to download data from a server without a valid connection pool

[CL 34474355 by yuriy odonnell in ue5-main branch]
2024-06-18 15:36:15 -04:00
yuriy odonnell
19fa4207c8 unsync - Fix directory separators when syncing from Horde
* Also handle some more errors that may be encountered during unsuccessful sync

[CL 34220309 by yuriy odonnell in ue5-main branch]
2024-06-07 20:17:08 -04:00
yuriy odonnell
c3f2a984c2 unsync - Groundwork for downloading Horde artifacts
* Recognize Horde artifact URLs
* Download and decode manifest from Horde

[CL 34189213 by yuriy odonnell in ue5-main branch]
2024-06-07 00:03:58 -04:00
yuriy odonnell
0bc836c31d unsync - Use deferred open mechanism for base files
* Avoid opening base files unnecessarily

[CL 34126244 by yuriy odonnell in ue5-main branch]
2024-06-05 11:21:32 -04:00
yuriy odonnell
5eee19aea6 unsync - Add experimental support for pure HTTP proxy server API, instead of custom binary protocol
[CL 34107282 by yuriy odonnell in ue5-main branch]
2024-06-04 17:15:24 -04:00
yuriy odonnell
02bac68ee6 unsync - Detect and report errros when downloading pack index files
[CL 34068576 by yuriy odonnell in ue5-main branch]
2024-06-03 12:28:21 -04:00
Yuriy ODonnell
d0e30d27ab unsync - Fix compile error on Linux
#jira none

[CL 33955096 by Yuriy ODonnell in ue5-main branch]
2024-05-28 17:55:18 -04:00
Yuriy ODonnell
3ee01ac3cc unsync - Use pack files when downloading blocks from server when possible
#jira none

[CL 33954792 by Yuriy ODonnell in ue5-main branch]
2024-05-28 17:45:20 -04:00
Yuriy ODonnell
0979b26d4a unsync - Add an abstraction for remote file system listing and manifest file downloads
#jira none

[CL 33925622 by Yuriy ODonnell in ue5-main branch]
2024-05-27 12:44:20 -04:00
Yuriy ODonnell
8812f02854 unsync - Further split utility code into dedicated modules
#jira none

[CL 33917971 by Yuriy ODonnell in ue5-main branch]
2024-05-26 13:37:37 -04:00
Yuriy ODonnell
81ab3f820c unsync - Move various utility code from UnsyncCore to separate modules
#jira none

[CL 33917087 by Yuriy ODonnell in ue5-main branch]
2024-05-26 02:04:36 -04:00
Yuriy ODonnell
50f882f5ec unsync - Groundwork for packing small files into a custom container during manifest generation
#jira none

[CL 33912288 by Yuriy ODonnell in ue5-main branch]
2024-05-24 20:21:04 -04:00
Yuriy ODonnell
ad086b7e64 unsync - Remove ConcRT/PPL, use custom scheduler by default
#jira none

[CL 33595690 by Yuriy ODonnell in ue5-main branch]
2024-05-11 20:20:25 -04:00
Yuriy ODonnell
0acabe5490 unsync - Added a mechanism for unified proxy server and source directory specification to simply sync command line
Example command line `unsync.exe sync https://foo.bar/directory/path <dest>` is equivalent to `unsync.exe sync --proxy https://foo.bar directory/path <dest>`

#jira none

[CL 33569778 by Yuriy ODonnell in ue5-main branch]
2024-05-10 00:54:55 -04:00
Yuriy ODonnell
a63405040e unsync - Add a basic custom thread scheduler for Mac/Linux
#jira none

[CL 33556179 by Yuriy ODonnell in ue5-main branch]
2024-05-09 15:37:00 -04:00
Yuriy ODonnell
833acc62ca unsync - Groundwork for a cross-platform scheduler
#jira none

[CL 33531808 by Yuriy ODonnell in ue5-main branch]
2024-05-08 18:13:41 -04:00
Yuriy ODonnell
bb18878804 unsync - Fix manifest search, output HTTP status codes on query errors
#jira none

[CL 33507065 by Yuriy ODonnell in ue5-main branch]
2024-05-08 01:56:22 -04:00
Yuriy ODonnell
e5fea36fe7 unsync - Search for .unsyncmanifest as an alternative for .unsync/manifest.bin when downloading manifests from remote server
#jira none

[CL 33492631 by Yuriy ODonnell in ue5-main branch]
2024-05-07 15:57:57 -04:00
Yuriy ODonnell
926e87e4bd unsync - ListDirectory and DownloadFile helpers now take FHttpConnection
#jira none

[CL 33487910 by Yuriy ODonnell in ue5-main branch]
2024-05-07 13:10:40 -04:00
Yuriy ODonnell
ca60aac7fc unsync - Fix downloading uncompressed blocks, tweak direct file download connection limit and chunk size
#jira none

[CL 33418009 by Yuriy ODonnell in ue5-main branch]
2024-05-02 20:46:49 -04:00
yuriy odonnell
f7f7485244 unsync - Fix inconsistent serialized directory separators in the manifest
* Fixed syncing data sets that had manifests generated on non-windows platform

#rb Dmytro.Ivanov

[CL 33205064 by yuriy odonnell in ue5-main branch]
2024-04-24 13:32:27 -04:00
yuriy odonnell
351f099b51 unsync - Attempt to establish TLS connections by default and allow fallback unless explicitly connecting to HTTPS port or using authentication
[CL 33127126 by yuriy odonnell in ue5-main branch]
2024-04-19 22:35:37 -04:00
yuriy odonnell
4ba5f8d39d unsync - Automatically enable TLS when server port is standard https/443
[CL 33095223 by yuriy odonnell in ue5-main branch]
2024-04-18 22:55:33 -04:00