Commit Graph

41 Commits

Author SHA1 Message Date
henrik karlsson
3ea6ef91db [UBA]
* Removed error on linux for failing poll call

[CL 30987771 by henrik karlsson in ue5-main branch]
2024-01-29 23:59:19 -05:00
zack neyland
505cac5c83 UBA: Add if some defs around POLLUP for platform differences
[CL 30983584 by zack neyland in ue5-main branch]
2024-01-29 21:01:39 -05:00
zack neyland
59077f46f4 UBA: Close fds when done with polling to stop leaking on MacOS.
[CL 30980705 by zack neyland in ue5-main branch]
2024-01-29 18:45:46 -05:00
henrik karlsson
0d51a68c53 [Uba]
* Fixed file descriptor leak when running native processes on posix platforms
* Removed hangup test on poll for pipe file descriptors since it exited to early and missed reads

[CL 30963156 by henrik karlsson in ue5-main branch]
2024-01-29 02:13:15 -05:00
josh adams
2c6d16634e Fixing working dir on Mac
#rb henrik.karlsson

[CL 30891145 by josh adams in ue5-main branch]
2024-01-25 12:31:40 -05:00
zack neyland
c10257601e UBA: Bump cmdline limit to 64k
[CL 30888128 by zack neyland in ue5-main branch]
2024-01-25 11:10:13 -05:00
henrik karlsson
def11cc5d0 [UBA]
* Improved error handling for setpriority even more. There is a risk that a non-detoured process have exited already when we get to this function

[CL 30878827 by henrik karlsson in ue5-main branch]
2024-01-25 01:12:09 -05:00
henrik karlsson
29914bf077 [UBA]
* Fixed so if setpriority fails on linux because of not having permissions we don't fail the build

[CL 30878645 by henrik karlsson in ue5-main branch]
2024-01-25 01:03:44 -05:00
henrik karlsson
abae032917 [UBA]
* Improved error description when failing to create pipe on posix

[CL 30875937 by henrik karlsson in ue5-main branch]
2024-01-24 22:22:28 -05:00
henrik karlsson
659c4c9101 [UBA]
* Changed so com memory is not split up in read and write since they never overlap.. so both read and write can use all the memory

[CL 30871523 by henrik karlsson in ue5-main branch]
2024-01-24 20:33:10 -05:00
henrik karlsson
ade0725e14 [UBA]
* Fixed so imagehlp.dll and dbghelp.dll are detoured when loaded. Detour ImageGetDigestStream and SymLoadModuleExW because both of them cause trouble on wine
* Fixed so reuse of processes also honor log files so a new log file is created when reuse happen
* Improved log file naming so host can set name and log files are sent back properly with the right name

[CL 30638193 by henrik karlsson in ue5-main branch]
2024-01-16 13:24:28 -05:00
henrik karlsson
4b1c00af7d [UBA]
* Fixed stats reporting bug for process reuse

[CL 30588189 by henrik karlsson in ue5-main branch]
2024-01-12 01:51:57 -05:00
henrik karlsson
22cf61f84b [UBA]
* Added GetNextProcess as a real message type (not using Custom) to be able to stop fetching work if helper is being terminated by aws or if we want to scale down number of workers
* Did some cleanup in the Tcp backend code and made sure not to call close socket multiple times (since it can cause the code to close a different socket than what is owned)

[CL 30584707 by henrik karlsson in ue5-main branch]
2024-01-11 21:01:56 -05:00
henrik karlsson
9f12e202b4 [UBA]
* Added ProcessStartInfo::writeOutputFilesOnFail that can be set to true if we want the output files to be written/sent back even though process exited with errors
* Changed so session client use ProcessStartInfoHolder and moved serialization code to that class

[CL 30576299 by henrik karlsson in ue5-main branch]
2024-01-11 15:42:50 -05:00
henrik karlsson
8827665b22 [UBA]
* Beeeautiful hack to try to get around ETXTBSY error when spawning ispc processes on linux. Our theory is that with perfectly bad timing the symlink creation of ispc could happen at the exact same time as a posix_spawn happening and inside symlink there is a temporarily open file descriptor that gets cloned into the child and kept open. and this causes ETXTBSY when trying to open that binary.

Solution is simply to have a writelock around the symlink creation and a readlock around posix_spawn

[CL 30513020 by henrik karlsson in ue5-main branch]
2024-01-09 14:26:32 -05:00
henrik karlsson
e0e929073b [Uba]
* Added RegisterDeleteFile to external api that should be used when file is deleted outside of uba but uba needs to know about it
* Reduced lock scope around FlushDeadProcesses
* Fixed disconnect issue crash in ubaagent that could cause access violation in UbaStorage. Fix was to cleanup when leaving StorageClient::SendAllSegments
* Changed so UbaRequestNextProcess is doing a full environment update (not resetting stats) if no next process is found
* Fixed so Scheduler enableProcessReuse=false works properly
* Improved some assert descriptions
* Fixed so files in m_outputFiles in session client is erased when flushed to server
* Fixed so Rpc_GetFullName cleans up .. in paths
* Fixed so files that existed but has been deleted by external process is seen as not existing by remote detoured process.

[CL 30507663 by henrik karlsson in ue5-main branch]
2024-01-09 12:04:16 -05:00
henrik karlsson
ea673427b2 [UBA]
* Implemented process reuse logic in UbaScheduler. It is now possible for running processes to fetch more work
* Fixed bugs in custom message path
* Fixed bugs in FlushWrittenFiles message
* Changed So UpdateEnvironment is resetting stats
* Fixed potential race condition related to process reuse
* Added special rule for ShaderCompileWorker.exe which detours ImageGetDigestStream in Imagehlp.dll (because wine implementation does not match windows implementation)
* Renamed "exitedUserData" to just userData since it is used for more than when exiting process
* Added a couple more dlls to "known system dlls"
* Fixed so visualizer can visualize process reuse properly

[CL 30462059 by henrik karlsson in ue5-main branch]
2024-01-02 17:42:42 -05:00
henrik karlsson
79fad227cd [UBA]
* Fixed linux compile error

[CL 30407078 by henrik karlsson in ue5-main branch]
2023-12-19 20:02:10 -05:00
josh adams
91a7b42e3e - Removed unneeded PLATFORM_MAC
#rb henrik.karlsson

[CL 30403458 by josh adams in ue5-main branch]
2023-12-19 16:32:11 -05:00
josh adams
5c95a66a67 - Lowered the priority of the processes Uba spawns, so that too many clangs don't starve the session process. Doesn't always repro, but it was causing some networking errors when talking to horde
#rb henrik.karlsson

[CL 30403426 by josh adams in ue5-main branch]
2023-12-19 16:31:15 -05:00
henrik karlsson
bb93072bd6 [Uba]
* #if 0 on code that does not exist on mac

[CL 30389279 by henrik karlsson in ue5-main branch]
2023-12-19 02:50:28 -05:00
josh adams
53e1c01e3f Fixed UBA on Mac to hve children die when session process goes away even if not notified
#rb henrik.karlsson

[CL 30385745 by josh adams in ue5-main branch]
2023-12-18 18:48:15 -05:00
henrik karlsson
5ebc3e8754 [UBA]
* Fixed one more stats visualization number that was wrong (not taking frequency in to account)

[CL 30364013 by henrik karlsson in ue5-main branch]
2023-12-15 17:57:18 -05:00
henrik karlsson
054d26f056 [UBA]
* Fixed so all visualization of times in visualizer is taking timer frequency into account

[CL 30344197 by henrik karlsson in ue5-main branch]
2023-12-15 03:06:12 -05:00
henrik karlsson
f0e6540060 [UBA]
* Added retry for ETXTBSY failures when doing posix_spawn. Have no idea how these can happen but they do.

[CL 30311542 by henrik karlsson in ue5-main branch]
2023-12-13 19:52:40 -05:00