57 Commits

Author SHA1 Message Date
lorry li
937421b24f Add the interface to set activity timeout per request.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]jon.cook
#rb jon.cook, michael.atchison
#tests Passed WebTests test case

[CL 32500358 by lorry li in 5.4 branch]
2024-03-26 03:01:14 -04:00
lorry li
7ba954943b Make sure to re-open the file when retry http request with streaming upload request.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]justin.marcus
#rb Justin.Marcus, michael.atchison

[CL 32500026 by lorry li in 5.4 branch]
2024-03-26 02:48:59 -04:00
lorry li
d230e9db55 Fix a random dead lock when shut down http manager.
Reason: Currently when shutdown and flush in http manager, it gives up and cancels remaining requests after waiting a short period. When cancel, it holds the RequestLock, and in FHttpRequestCommon::CancelRequest, it tries to access HttpTaskTimerHandleCriticalSection to stop activity timer. In the mean time, the request itself could be in activity or total timeout callback, which is holding the HttpTaskTimerHandleCriticalSection, and trying to AbortRequest which will try to access RequstLock in HttpManager.IsValidRequest.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb michael.atchison, Michael.Kirzinger, Rafa.Lecina
#tests Reproduced with test case in WebTests and fixed it. Tried launch and quit game on Win64.

[CL 32499301 by lorry li in 5.4 branch]
2024-03-26 02:18:50 -04:00
lorry li
abe7f74335 Fix the dead lock by reducing the scope of request lock in http manager to make sure when trigger http request complete callback in Tick, it doesn't hold the requests lock;
Deperecate the FGenericPlatformHttp::UsesThreadedHttp interface and remove corresponding impls, now all platforms are using threaded http requests;
Re-enable retry system off-game thread support.

Reason:
When accessing requests lock and trigger complete callback in http manager Tick from main thread, the complete callback could have been bound to retry manager which will try to access retry manager lock, while retry manager lock can be holding by FHttpRetrySystem::FManager::RetryHttpRequestWithDelay, which triggered by timer callback from http thread and is trying to access http manager's requests lock in order to add the retry request to http manager.

[REVIEW] [at]michael.atchison [at]michael.kirzinger [at]rafa.lecina
#rb michael.atchison

[CL 32493084 by lorry li in 5.4 branch]
2024-03-25 19:55:05 -04:00
marc audy
ac171e0025 Make sure don't pass received data to stream interface after cancelling the http request or after shutdowning http module;
Add test cases to test that after canceling/shut down request, the response body receive delegate won't ever been called.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]jens.petersam
#jira UE-207423, FORT-706402
#rb michael.atchison, Rafa.Lecina

[CL 32493078 by marc audy in 5.4 branch]
2024-03-25 19:54:54 -04:00
lorry li
ad12c40d95 Improve default HTTP retry behavior when services do not indicate how long to wait for retries
- Retry period changed from linear to exponential;
- Adds random jitter multiplication to computed backoff.

Final default retry lockout period formula looks like this:
RandRange(JitterMin, JitterMax) * Pow(Base, RetryNumber + 1 + Bias) where RetryNumber is 0, 1, 2, 3...

Defaults:
JitterMin = 0.5
JitterMax = 1.0
Base = 2.0
Bias = 1.0

[REVIEW] [at]david.parton [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb Rafa.Lecina

[CL 32492903 by lorry li in 5.4 branch]
2024-03-25 19:49:47 -04:00
richard smith
956fa0880e Integrating fix for #sh-03-08-7k-server-crashes-due-to-fort-714642-in--ext
Make sure to stop activity timeout timer when total timeout triggered;
Only output log if request had completed when activity timeout, it can continue to run even if that happened.

[REVIEW] [at]michael.atchison [at]michael.kirzinger [at]rafa.lecina
#rb Rafa.Lecina

#p4v-cherrypick 31948855

[CL 32141241 by richard smith in 5.4 branch]
2024-03-08 22:52:54 -05:00
lorry li
941e18477a In http sony impl change it to read response headers once received status code;
Use cvar for the new flow just in case the headers needs to be read when complete for some urls.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb Chris.Adams, michael.atchison
#tests Tried through WebTests and the game

[CL 31596531 by lorry li in 5.4 branch]
2024-02-16 21:21:18 -05:00
lorry li
79da16fea6 Added optional, separated retry limit support for connection error in http retry system.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]daniel.lamb
#jira UE-206119
#rb Daniel.Lamb, michael.atchison, Rafa.Lecina
#tests Passed new test case in WebTests

[CL 31349566 by lorry li in 5.4 branch]
2024-02-09 14:01:46 -05:00
lorry li
56b45f3e0c Add more test cases about retry behaviors in WebTests;
Make sure when timed out in retry system, during back off or during retrying, it will return the last response instead of empty response.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
[FYI] daniel.lamb
#rb Rafa.Lecina
#tests Tried in WebTests

[CL 31304508 by lorry li in 5.4 branch]
2024-02-08 16:23:28 -05:00
lorry li
227ef19932 Stop activity timeout timer after canceling request;
Stop total timeout timer after request completed in retry manager;
Move the virtual impls of FHttpRequestAdapter to cpp file.

[REVIEW] [at]michael.kirzinger [at]rafa.lecina [at]michael.atchison
#jira UE-197485
[FYI] [at]jens.petersam
#rb Jens.Petersam, Rafa.Lecina
#tests Repro and fixed in WebTests

[CL 31298382 by lorry li in 5.4 branch]
2024-02-08 14:23:20 -05:00
lorry li
85c1ad8a82 Make sure python packages can be updated in WebTestsServer;
Added more checks in test case "Can do blocking call" of WebTests;
Added duplicated call CancelRequest in test case "Cancel http request connect before timeout" of  WebTests;
Some other code clean in WebTests.

[REVIEW] [at]stephen.ma [at]michael.kirzinger [at]rafa.lecina
#rb stephen.ma

[CL 31289772 by lorry li in 5.4 branch]
2024-02-08 09:19:12 -05:00
lorry li
20e2d52297 Add GetEffectiveURL interface to get the effective url after redirection.
#jira UE-204044
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]jens.petersam
#rb Rafa.Lecina
#tests Passed WebTests test case, launched the game to main menu.

[CL 31178090 by lorry li in 5.4 branch]
2024-02-05 12:29:09 -05:00
lorry li
4f801e27f5 Fix the failure in test case "Can do blocking call" in WebTests by removing the complete delegate.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina

[CL 31172218 by lorry li in 5.4 branch]
2024-02-05 09:35:54 -05:00
lorry li
d9de04ec69 Add check in WebTests test case to validate the received data.
[REVIEW] [at]michael.kirzinger [at]justin.marcus [at]michael.atchison [at]rafa.lecina

[CL 31171508 by lorry li in 5.4 branch]
2024-02-05 09:09:10 -05:00
lorry li
a69a2f0914 Fix the http dead lock when Flush while AbortRequest;
Add the ProcessRequestUntilComplete sync call to IHttpRequest.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]robert.millar [at]laura.hermanns
#jira UE-197485
#rb Laura.Hermanns, michael.atchison, Rafa.Lecina
#tests Passed tests in WebTests.

[CL 30946224 by lorry li in 5.4 branch]
2024-01-26 18:03:04 -05:00
lorry li
0980390e94 Migrate retry system total timeout for http request, by adding total timeout feature in http request;
Deprecate HttpTimeout config, use HttpActivityTimeout or HttpTotalTimeout instead;
Deprecate HttpSendTimeout config, only use HttpActivityTimeout;
Make HttpActivityTimeout work on all platforms, not only CurlHttp;
Added corresponding http tests;
Now because timeout migrated, enable new flow by default in retry system to have non-game thread support.

#jira UE-197485, UE-202201
[REVIEW] [at]michael.atchison [at]michael.kirzinger [at]rafa.lecina
#rb michael.atchison, Michael.Kirzinger
#tests Tested through WebTests project on all platforms, also tried the game on PC.

[CL 30817277 by lorry li in ue5-main branch]
2024-01-23 16:08:08 -05:00
lorry li
05a45465b3 Add status code received callback for http and implement it for all platforms.
#jira UE-203992
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]jans.petersam
#rb michael.atchison, Rafa.Lecina
#tests Tested through new test cases in WebTests project

[CL 30726774 by lorry li in ue5-main branch]
2024-01-19 15:35:30 -05:00
lorry li
9ab1c34e02 Add mock_latency method in web tests server, prepare for timeout tests;
Add chunk latency param for streaming api in web tests server;
Make sure streaming api in web tests server can actually return response data by chunks in django asgi server.

#jira UE-197485
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb michael.atchison, Michael.Kirzinger, Rafa.Lecina
#tests Tried in WebTests project

[CL 30669130 by lorry li in ue5-main branch]
2024-01-17 16:29:34 -05:00
lorry li
340a38a9df Add test code to cover the case that download progress get cleaned when retry.
[REVIEW] [at]michael.kirzinger [at]rex.hill
#rb Rafa.Lecina, Rex.Hill, stephen.ma
#tests Tested through WebTests in local

[CL 30639345 by lorry li in ue5-main branch]
2024-01-16 14:08:02 -05:00
lorry li
882df0d083 Add test case to validate the http header receive callback.
[REVIEW] [at]justin.marcus [at]michael.kirzinger [at]rafa.lecina [at]michael.atchison
#rb Justin.Marcus, michael.atchison, Michael.Kirzinger, Rafa.Lecina
#tests Tested through WebTests preflight

[CL 30639160 by lorry li in ue5-main branch]
2024-01-16 14:00:28 -05:00
RiotJoshuaGlazer
70b5e2b144 PR #10520: Fix issues with non blocking sockets in SocketBSD preventing http server from serving large files on linux
#jira UE-188438
[REVIEW] [at]michael.kirzinger [at]michael.atchison
#rb jon.fairchild, Ryan.Gerleve
#tests Reproduced and fixed it in WebTests project by a test case.

[CL 30298475 by RiotJoshuaGlazer in ue5-main branch]
2023-12-13 13:40:37 -05:00
lorry li
22aaf876f0 Make all http impls respect the connect timeout set in http module.
#jira UE-201788
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb Michael.Kirzinger, Rafa.Lecina
#tests Test covered in WebTests project

[CL 30245270 by lorry li in ue5-main branch]
2023-12-11 14:19:43 -05:00
stephen ma
542bc9dff6 Move HTTP Decode tests to WebTests
[CL 29914944 by stephen ma in ue5-main branch]
2023-11-23 17:52:59 -05:00
lorry li
e371c162e5 Revert the changes to limit schemes to http and https. We already have support to config the allowed schemes through FURLRequestFilter.
#jira UE-198427
[REVIEW] [at]michael.kirzinger [at]rafa.lecina
#rb Michael.Kirzinger, Rafa.Lecina

[CL 29841236 by lorry li in ue5-main branch]
2023-11-20 09:55:01 -05:00