Clear caches and update a canary value across the fork boundary to avoid touching pages that were allocated pre-fork to minimize unnecessary duplication of shared pages.
Savings depend on pre-fork fragmentation/slack and number of concurrent child processes.
#rb louisphilippe.seguin, johan.torp, arciel.rekman
#ROBOMERGE-OWNER: robert.millar
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19057122 via CL 19057644 via CL 19057693 via CL 19057718 via CL 19057737 via CL 19059617
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)
[CL 19066314 by robert millar in ue5-main branch]
Adding optional child process timeout for WAIT_AND_FORK_RESPONSE_SIGNAL
Added logging if we fail to set the child command line
#rb louisphilippe.seguin, brandon.schaefer
#ROBOMERGE-AUTHOR: brian.bekich
#ROBOMERGE-SOURCE: CL 19050891 via CL 19050899 via CL 19050911 via CL 19057425
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)
[CL 19066039 by brian bekich in ue5-main branch]
#rb none
#ROBOMERGE-AUTHOR: robert.millar
#ROBOMERGE-SOURCE: CL 19038494 via CL 19038585 via CL 19038593 via CL 19039371 via CL 19039472 via CL 19039992
#ROBOMERGE-BOT: UE5 (Release-Engine-Staging -> Main) (v918-19018356)
[CL 19040355 by robert millar in ue5-main branch]
Set to 1 if we should try to use /proc/self/smaps_rollup in FUnixPlatformMemory::GetExtendedStats().
There is a potential tradeoff in that smaps_rollup appears to be quite a bit faster in
straight testing, but it also looks like it blocks mmap() calls on other threads until it finishes.
Also move /proc/self/smaps code to use open/read/close and read + parse in 512 byte chunks instead of
fopen to avoid malloc calls in fopen/fgets.
[REVIEW] [at]Brandon.Schaefer, [at]Ilya.Loshchinin
#preflight trivial
#ROBOMERGE-OWNER: michael.sartain
#ROBOMERGE-AUTHOR: michael.sartain
#ROBOMERGE-SOURCE: CL 19001314 via CL 19004045 via CL 19004552 via CL 19005213 via CL 19008130 via CL 19008739
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v917-18934589)
#ROBOMERGE-CONFLICT from-shelf
[CL 19009893 by michael sartain in ue5-main branch]
fopen/fgets both allocate memory and we're potentially hitting contention when multithreaded
Also remove the freebsd code and clean up everything to use common ReadProcFields function
#jira none
#review-18926623 @Brandon.Schaefer, @Robert.Seiver, @Ilya.Loshchinin
#preflight trivial
[CL 18953210 by Michael Sartain in ue5-main branch]
* Added -forkautostatsport cmdline option that automatically sets a unique port for each forked child. Ports used will start at statsPort+1.
* Improved statsport /exec listener so it returns a different error message when the exec callback isn't registered, the cmd itself was not processed by the exec or the cmd is missing the 'c=' param.
Forking
* Exposed ChildProcessIndex in ForkHelper
[REVIEW] Brandon.Schaeffer, Brian.Bekich
#ROBOMERGE-AUTHOR: louisphilippe.seguin
#ROBOMERGE-SOURCE: CL 18770568 via CL 18770579 via CL 18770587 via CL 18774066 via CL 18774960
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v903-18687472)
[CL 18775041 by louisphilippe seguin in ue5-main branch]
smaps_rollup was added in v4.14. Details on this page:
https://kernelnewbies.org/Linux_4.14
It should be a constant ~1ms vs 6.8ms (and far higher) when reading entire smaps file
(Tested on my 5.15 kernel)
smaps_rollup info:
https://patchwork.kernel.org/project/linux-fsdevel/patch/20170812022148.178293-1-dancol[at]google.com/
Smaps info:
> FPlatformMemory::GetExtendedStats() is slow and took 28ms in my local tests.
> Moreover, it can be even slower because the time it takes is a (hopefully
> linear) function of the number of memory allocations.
#jira UE-139749, UE-52126
[REVIEW] Brandon.Schaefer, Arciel.Rekman
[FYI] Brian.Bekich, Ilya.Loshchinin, Robert.Seiver
#preflight trivial
#ROBOMERGE-AUTHOR: michael.sartain
#ROBOMERGE-SOURCE: CL 18683396 via CL 18683443 via CL 18683460 via CL 18684028 via CL 18684035 via CL 18684055
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v902-18672795)
[CL 18684069 by michael sartain in ue5-main branch]
This is similar to CL 2421073, 2424491 for FApplePlatformFile.
[at]Brandon.Schaefer, [at]Robert.Seiver
[FYI] Will.Damon
#jira none
#preflight trivial
#ROBOMERGE-AUTHOR: michael.sartain
#ROBOMERGE-SOURCE: CL 18604349 in //UE5/Release-5.0/... via CL 18604376 via CL 18604394
#ROBOMERGE-BOT: UE5 (Release-Engine-Test -> Main) (v899-18417669)
[CL 18604413 by michael sartain in ue5-main branch]
Changed underlying API from posix_spawn to posix_spawnp, which only differs in behavior if the provided path fragment does not contain any path separators. Matches within the base directory are still prioritized if they exist. Failures are no longer treated as fatal.
#jira UE-138792
#rb Brandon.Schaefer, Michael.Sartain
#preflight 61ddd69fcd2cced71060929a
#ROBOMERGE-AUTHOR: zach.brockway
#ROBOMERGE-SOURCE: CL 18575611 in //UE5/Release-5.0/... via CL 18575644
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)
[CL 18575658 by zach brockway in ue5-release-engine-test branch]
Also move stack to mmap'd memory instead of alloc'ing in heap
Should allow us to set & reduce size of 200k stack significantly via command line.
Brandon.Schaefer
#jira none
#preflight 61dc61cb1f62d3ad4d8079d4
#ROBOMERGE-AUTHOR: michael.sartain
#ROBOMERGE-SOURCE: CL 18561402 in //UE5/Release-5.0/... via CL 18561404
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)
[CL 18561421 by michael sartain in ue5-release-engine-test branch]
Windows needs to set permissions accordingly.
#rb Brandon.Schaefer
[FYI] Josh.Adams
#preflight 61d79ed84c252480ca31e640
#ROBOMERGE-AUTHOR: jack.porter
#ROBOMERGE-SOURCE: CL 18539045 in //UE5/Release-5.0/... via CL 18539064
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v899-18417669)
[CL 18539081 by jack porter in ue5-release-engine-test branch]
#tests Crashed on Linux with nullptr and non-null ptr
#rb Brandon.Schaefer
#preflight 61b245cafb3115178686b4a7
#ROBOMERGE-AUTHOR: ilya.loshchinin
#ROBOMERGE-SOURCE: CL 18422416 via CL 18422444 via CL 18422465 via CL 18435085 via CL 18437383
#ROBOMERGE-BOT: STARSHIP (Release-Engine-Staging -> Release-Engine-Test) (v897-18405271)
[CL 18437661 by ilya loshchinin in ue5-release-engine-test branch]
Add stderr parameter to CreateProc
Add exec-process test to TestPAL
PR #6919: Add a new FPlatformProcess::CreateProc() overload to allow separating stderr and stdout (Contributed by geordiemhall)
#rb Brandon.Schaefer, James.Singer, Robert.Seiver, Will.Damon
#jira UE-91758, UE-92964
#ROBOMERGE-SOURCE: CL 17290306 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v861-17282326)
[CL 17290325 by geordiemhall in ue5-release-engine-test branch]
Change all FPlatformMemory::OnOutOfMemory implementations to [[noreturn]]
#jira UE-93728
#rb steve.robb, arciel.rekman
#ROBOMERGE-SOURCE: CL 17212898 in //UE5/Main/...
#ROBOMERGE-BOT: STARSHIP (Main -> Release-Engine-Test) (v855-17104924)
[CL 17212911 by johan torp in ue5-release-engine-test branch]
