Commit Graph

63 Commits

Author SHA1 Message Date
lorry li
076a9f64d5 Fix the potential http crash when shared ptr bound to data receiving delegate got destroyed.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb michael.atchison, Michael.Kirzinger, Rafa.Lecina
#tests Validated through test cases in WebTests project

[CL 33200296 by lorry li in ue5-main branch]
2024-04-24 11:09:32 -04:00
lorry li
3273aa98e7 Fix the test case crash "Shutdown http module without issue when there are ongoing download http requests" in WebTests.
Reason: UserStreamingInstance instance got deleted before http module shutdown, the data received delegates only unbind when http module shutdown.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb michael.atchison, Michael.Kirzinger

[CL 32946476 by lorry li in ue5-main branch]
2024-04-12 17:41:58 -04:00
lorry li
c29cce2d5a Fix the test case "Http request connect timeout" for xCurl by increasing UE_HTTP_CONNECTION_TIMEOUT_MAX_DEVIATION;
Filter out the test case "Cancel http request connect before timeout" for platforms which doesn't support connect timeout retry with precise timeout value.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb michael.atchison

[CL 32840155 by lorry li in ue5-main branch]
2024-04-09 17:22:33 -04:00
lorry li
219ee093c2 Refactor function FHttpRetrySystem::FManager::ShouldRetry to use early returns.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb Rafa.Lecina
#tests Covered by test case in WebTests.

[CL 32735428 by lorry li in ue5-main branch]
2024-04-04 14:01:11 -04:00
lorry li
e8f537554e Make sure when cancel http request from retry system, it can still trigger complete delegate when non-game thread support enabled.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb michael.atchison, Rafa.Lecina

[CL 32673430 by lorry li in ue5-main branch]
2024-04-02 13:51:50 -04:00
lorry li
94234107c7 Add the interface to set activity timeout per request.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]jon.cook
#rb jon.cook, michael.atchison
#tests Passed WebTests test case

[CL 32409091 by lorry li in ue5-main branch]
2024-03-21 15:03:46 -04:00
lorry li
1e0719ce28 Make sure to re-open the file when retry http request with streaming upload request.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]justin.marcus
#rb Justin.Marcus, michael.atchison

[CL 32371902 by lorry li in ue5-main branch]
2024-03-20 13:04:21 -04:00
lorry li
acbaf16fb7 Fix a random dead lock when shut down http manager.
Reason: Currently when shutdown and flush in http manager, it gives up and cancels remaining requests after waiting a short period. When cancel, it holds the RequestLock, and in FHttpRequestCommon::CancelRequest, it tries to access HttpTaskTimerHandleCriticalSection to stop activity timer. In the mean time, the request itself could be in activity or total timeout callback, which is holding the HttpTaskTimerHandleCriticalSection, and trying to AbortRequest which will try to access RequstLock in HttpManager.IsValidRequest.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb michael.atchison, Michael.Kirzinger, Rafa.Lecina
#tests Reproduced with test case in WebTests and fixed it. Tried launch and quit game on Win64.

[CL 32281657 by lorry li in ue5-main branch]
2024-03-15 15:12:05 -04:00
lorry li
9f7b96d4f7 When http request total timeout during lockout, make sure it can trigger complete delegate by http delegate thread policy;
Use FTSTicker in http manager instead to manage game thread tasks.

#jira UE-163631
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb Rafa.Lecina
#tests Passed all tests in WebTests project

[CL 32052075 by lorry li in ue5-main branch]
2024-03-06 09:41:35 -05:00
lorry li
efdbf20e85 Make sure to stop activity timeout timer when total timeout triggered;
Only output log if request had completed when activity timeout, it can continue to run even if that happened.

[REVIEW] [at]michael.atchison [at]michael.kirzinger [at]rafa.lecina
#rb Rafa.Lecina

[CL 31948912 by lorry li in ue5-main branch]
2024-03-01 13:19:35 -05:00
lorry li
1d79a2d847 Fix the test case "Optionally retry limit can be set differently for connection error" by using latest expected back off value;
Make the test case "Test platform request requests limits" covers the cancelling using CompleteOnHttpThread as delegate thread policy.

[REVIEW] [at]michael.atchison [at]michael.kirzinger [at]rafa.lecina [at]stephen.ma
#rb Rafa.Lecina
#tests Passed the WebTests.

[CL 31879765 by lorry li in ue5-main branch]
2024-02-28 14:33:34 -05:00
lorry li
fa91b21b64 Fix the dead lock by reducing the scope of request lock in http manager to make sure when trigger http request complete callback in Tick, it doesn't hold the requests lock;
Deperecate the FGenericPlatformHttp::UsesThreadedHttp interface and remove corresponding impls, now all platforms are using threaded http requests;
Re-enable retry system off-game thread support.

Reason:
When accessing requests lock and trigger complete callback in http manager Tick from main thread, the complete callback could have been bound to retry manager which will try to access retry manager lock, while retry manager lock can be holding by FHttpRetrySystem::FManager::RetryHttpRequestWithDelay, which triggered by timer callback from http thread and is trying to access http manager's requests lock in order to add the retry request to http manager.

[REVIEW] [at]michael.atchison [at]michael.kirzinger [at]rafa.lecina
#rb michael.atchison

[CL 31821069 by lorry li in ue5-main branch]
2024-02-26 18:12:55 -05:00
lorry li
0cd2b1ff8d Make sure don't pass received data to stream interface after cancelling the http request or after shutdowning http module;
Add test cases to test that after canceling/shut down request, the response body receive delegate won't ever been called.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]jens.petersam
#jira UE-207423, FORT-706402
#rb michael.atchison, Rafa.Lecina

[CL 31820574 by lorry li in ue5-main branch]
2024-02-26 18:06:19 -05:00
lorry li
0d51ce6853 Fix the test case "Optionally retry limit can be set differently for connection error", can't really retry when activity timeout, because the response is not null when get connection error. User code might still want to access the response when activity timeout. So changed the test case to use connection time out instead;
Get rid of platform specific code in WebTests, define macros in Http.Build.cs instead.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#jira UE-207067, UE-207061
#rb Rafa.Lecina

[CL 31777105 by lorry li in ue5-main branch]
2024-02-23 16:26:05 -05:00
lorry li
67a08f1634 Improve default HTTP retry behavior when services do not indicate how long to wait for retries
- Retry period changed from linear to exponential;
- Adds random jitter multiplication to computed backoff.

Final default retry lockout period formula looks like this:
RandRange(JitterMin, JitterMax) * Pow(Base, RetryNumber + 1 + Bias) where RetryNumber is 0, 1, 2, 3...

Defaults:
JitterMin = 0.5
JitterMax = 1.0
Base = 2.0
Bias = 1.0

[REVIEW] [at]david.parton [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb Rafa.Lecina

[CL 31776593 by lorry li in ue5-main branch]
2024-02-23 16:17:01 -05:00
lorry li
30335b12e1 In http sony impl change it to read response headers once received status code;
Use cvar for the new flow just in case the headers needs to be read when complete for some urls.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
#rb Chris.Adams, michael.atchison
#tests Tried through WebTests and the game

[CL 31601881 by lorry li in ue5-main branch]
2024-02-17 12:48:36 -05:00
lorry li
ba0c68c448 Only output warning when request stay in http thread queue for too long;
Added the MaxTimeToWaitInQueue stat in HttpStats.

#jira UE-204405
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]alejandro.aguilar
#rb alejandro.aguilar, michael.atchison, Rafa.Lecina

[CL 31441575 by lorry li in ue5-main branch]
2024-02-13 14:53:52 -05:00
lorry li
2c729b929d Make the http request don't output warning if the request get canceled;
Move the default max concurrent http requests limit to compile time decided by different platforms.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]alejandro.aguilar [at]stephen.ma [at]rafa.lecina
#rb michael.atchison, Rafa.Lecina, stephen.ma
#tests Passed WebTests project

[CL 31350710 by lorry li in ue5-main branch]
2024-02-09 14:31:55 -05:00
lorry li
7922ee5565 Added optional, separated retry limit support for connection error in http retry system.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]daniel.lamb
#jira UE-206119
#rb Daniel.Lamb, michael.atchison, Rafa.Lecina
#tests Passed new test case in WebTests

[CL 31350103 by lorry li in ue5-main branch]
2024-02-09 14:14:37 -05:00
stephen ma
9b3bdebaa6 Adds tests for LAUNCHREQUEST 1, 20, 50, 100, 500, 1000 reqeusts.
Also adds tests for canceling 1,20,50,100,500,1000 requests.

[FYI] Benjamin.Dohrmann, Ryan.Hairyes, Chris.Varnsverry
#rb alejandro.aguilar, Chris.Varnsverry, lorry.li

[CL 31313811 by stephen ma in ue5-main branch]
2024-02-08 19:27:07 -05:00
lorry li
b2820a2752 Add more test cases about retry behaviors in WebTests;
Make sure when timed out in retry system, during back off or during retrying, it will return the last response instead of empty response.

[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina
[FYI] daniel.lamb
#rb Rafa.Lecina
#tests Tried in WebTests

[CL 31310240 by lorry li in ue5-main branch]
2024-02-08 18:13:40 -05:00
lorry li
702eeb4a13 Stop activity timeout timer after canceling request;
Stop total timeout timer after request completed in retry manager;
Move the virtual impls of FHttpRequestAdapter to cpp file.

[REVIEW] [at]michael.kirzinger [at]rafa.lecina [at]michael.atchison
#jira UE-197485
[FYI] [at]jens.petersam
#rb Jens.Petersam, Rafa.Lecina
#tests Repro and fixed in WebTests

[CL 31304925 by lorry li in ue5-main branch]
2024-02-08 16:31:09 -05:00
lorry li
881a7127a7 Make sure python packages can be updated in WebTestsServer;
Added more checks in test case "Can do blocking call" of WebTests;
Added duplicated call CancelRequest in test case "Cancel http request connect before timeout" of  WebTests;
Some other code clean in WebTests.

[REVIEW] [at]stephen.ma [at]michael.kirzinger [at]rafa.lecina
#rb stephen.ma

[CL 31289833 by lorry li in ue5-main branch]
2024-02-08 09:21:28 -05:00
lorry li
f1f1946eee Add GetEffectiveURL interface to get the effective url after redirection.
#jira UE-204044
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina [at]jens.petersam
#rb Rafa.Lecina
#tests Passed WebTests test case, launched the game to main menu.

[CL 31180593 by lorry li in ue5-main branch]
2024-02-05 13:23:49 -05:00
lorry li
a5c3cb5473 Fix the failure in test case "Can do blocking call" in WebTests by removing the complete delegate.
[REVIEW] [at]michael.kirzinger [at]michael.atchison [at]rafa.lecina

[CL 31172233 by lorry li in ue5-main branch]
2024-02-05 09:36:17 -05:00