yuriy odonnell
3661ad26bc
unsync - Fix GetRelativePath() for network paths
...
[CL 35101305 by yuriy odonnell in ue5-main branch]
2024-07-25 20:45:40 -04:00
yuriy odonnell
b247041845
unsync - Implement support for overlays when syncing from Horde
...
[CL 34948454 by yuriy odonnell in ue5-main branch]
2024-07-19 18:21:08 -04:00
yuriy odonnell
22adbd7f05
unsync - Increase default socket timeout from 1 minute to 15
...
[CL 34902765 by yuriy odonnell in ue5-main branch]
2024-07-18 11:44:09 -04:00
yuriy odonnell
d0aa0bd51c
unsync - Fix manifest caching when using overlays
...
* Virtualized file system was created for the main sync source and erroneously was used when attempting to fetch overlay manifest
[CL 34816579 by yuriy odonnell in ue5-main branch]
2024-07-15 12:53:03 -04:00
yuriy odonnell
88769697df
unsync - Add early exit if non-success HTTP code is encountered during query http-get command
...
* Update version to 1.0.82
[CL 34752406 by yuriy odonnell in ue5-main branch]
2024-06-28 19:51:06 -04:00
yuriy odonnell
c65a2c0128
unsync - Add query http-get <url> command to easily issue use Horde and Unsync HTTP APIs
...
* Add extended error messages to various places
[CL 34701319 by yuriy odonnell in ue5-main branch]
2024-06-27 00:49:36 -04:00
yuriy odonnell
fc93606223
unsync - Fix horde query parsing when it only contains the artifact id
...
[CL 34696081 by yuriy odonnell in ue5-main branch]
2024-06-26 18:46:20 -04:00
yuriy odonnell
19d713baee
unsync - Fix compile error
...
[CL 34661850 by yuriy odonnell in ue5-main branch]
2024-06-25 18:45:05 -04:00
yuriy odonnell
8a833fe9ff
unsync - Add Horde artifact listing syntax backwards-compatible with filesystem based paths
...
* Specify artifact filters using `KeyA=ValueA;KeyB=ValueB` syntax
* Specify artifact listing formating and directory structure using `{FieldA}{dir}{FieldB}` syntax, where {FieldA/B} are substituted with artifact metadata and {dir} creates a virtual sub-directory
* Missing fields are substituted with `[unknown]`
[CL 34660906 by yuriy odonnell in ue5-main branch]
2024-06-25 18:31:35 -04:00
yuriy odonnell
653714087b
unsync - Implement experimental mechanism to list Horde artifacts as a virtual directory structure
...
* Use 5 minute threshold by default for triggering auth token refresh
[CL 34549235 by yuriy odonnell in ue5-main branch]
2024-06-20 18:37:09 -04:00
yuriy odonnell
7a8cebabab
unsync - Use common implementation of server directory listing for query list and search commands
...
[CL 34518988 by yuriy odonnell in ue5-main branch]
2024-06-19 20:06:19 -04:00
yuriy odonnell
accfc7e7fe
unsync - Improve error checking and handling
...
* Handle most errors during Horde block download as non-fatal (allow other files to be downloaded if one file hits an error)
* Account for block cache creation time when logging individual file download rate
* Add explicit EDownloadRetryMode::Disconnect code to prevent subsequent server requests, distinct from from Abort code
* Propagate errors from few more places when decoding auth tokens
[CL 34514495 by yuriy odonnell in ue5-main branch]
2024-06-19 17:14:41 -04:00
yuriy odonnell
31b376099f
unsync - Add --no-compression command line argument to prefer uncompressed block downloads when possible
...
* Fix block size validation when downloading from Horde without compression
* Add more explicit error reporting when attempting to download data from a server without a valid connection pool
[CL 34474355 by yuriy odonnell in ue5-main branch]
2024-06-18 15:36:15 -04:00
yuriy odonnell
ad9fde4431
unsync - Propagate DecodeJwtPayload() error from AcquireAuthToken()
...
* Fix typo in Horde block validation error message
* Remove auth URL logging
[CL 34467219 by yuriy odonnell in ue5-main branch]
2024-06-18 12:45:29 -04:00
yuriy odonnell
9bdf70ec4e
unsync - Handle 0-sized blocks when downloading data from Horde
...
[CL 34464902 by yuriy odonnell in ue5-main branch]
2024-06-18 12:10:20 -04:00
yuriy odonnell
d3052b8198
unsync - Use current time if file in the manifest doesn't have a valid timestamp
...
[CL 34220430 by yuriy odonnell in ue5-main branch]
2024-06-07 20:30:08 -04:00
yuriy odonnell
19fa4207c8
unsync - Fix directory separators when syncing from Horde
...
* Also handle some more errors that may be encountered during unsuccessful sync
[CL 34220309 by yuriy odonnell in ue5-main branch]
2024-06-07 20:17:08 -04:00
yuriy odonnell
703013dc43
unsync - Basic implementation of Horde blob download
...
[CL 34218083 by yuriy odonnell in ue5-main branch]
2024-06-07 18:21:05 -04:00
yuriy odonnell
fbd81d0c69
unsync - Add --protocol argument to explicitly specify the server protocol flavor to use
...
[CL 34207269 by yuriy odonnell in ue5-main branch]
2024-06-07 13:00:34 -04:00
yuriy odonnell
82f4423109
unsync - Remove leftover dev comment
...
[CL 34191467 by yuriy odonnell in ue5-main branch]
2024-06-07 02:08:54 -04:00
yuriy odonnell
c3f2a984c2
unsync - Groundwork for downloading Horde artifacts
...
* Recognize Horde artifact URLs
* Download and decode manifest from Horde
[CL 34189213 by yuriy odonnell in ue5-main branch]
2024-06-07 00:03:58 -04:00
yuriy odonnell
5e75c65ac5
unsync - Reduce chunking test buffer to from 1GB to 128MB, add validation for number of generated chunks and report median chunk size
...
[CL 34171426 by yuriy odonnell in ue5-main branch]
2024-06-06 16:01:58 -04:00
yuriy odonnell
608ab7041f
unsync - Add unit tests for chunking algorithm, add utility function to compute rolling window hash threshold to achieve specific average block size
...
[CL 34144345 by yuriy odonnell in ue5-main branch]
2024-06-05 19:22:24 -04:00
yuriy odonnell
0bc836c31d
unsync - Use deferred open mechanism for base files
...
* Avoid opening base files unnecessarily
[CL 34126244 by yuriy odonnell in ue5-main branch]
2024-06-05 11:21:32 -04:00
yuriy odonnell
d5e9515dd3
unsync - Send strong hash algorithm id with the block requests
...
[CL 34112795 by yuriy odonnell in ue5-main branch]
2024-06-04 20:45:07 -04:00
