Commit Graph

27 Commits

Author SHA1 Message Date
henrik karlsson
2ecaf5607e [UBA]
* Fixed unit test compile errors
* Fixed race condition in unit test shutdown

[CL 31374569 by henrik karlsson in ue5-main branch]
2024-02-11 18:22:11 -05:00
henrik karlsson
34b52deafb [UBA]
* Added hint to unmapfileview to be able to add useful information to asserts on fails
* Added error handling for vsnprintf returning error
* Fixed memory stomp in unit test
* Removed usage of %hs on non-windows since asan complains about it
* Changed UBA_USE_MIMALLOC to always be defined and be 0 or 1 instead of using ifdef
* Enabled mimalloc on linux

[CL 31355077 by henrik karlsson in ue5-main branch]
2024-02-09 16:12:12 -05:00
henrik karlsson
eee33ba668 [UBA]
* Fixed so stdout/err is redirected to pipe on detoured processes on linux/mac.. this should fix so errors are visualized in the right place
* Added -nostdout to UbaCli to be able to check that redirected stdout/err works
* Added version for process messages to catch issues where ppl have old ubaagent.exe but new ubadetours.dll
* Added capacity parameter to FixPath and added asserts to check that we never write outside capacity
* Fixed code creating g_virtualApplication buffer.
* Disabled asserts for mac non debug builds
* Removed detoured write/fwrite for posix now when we instead redirect stdout/err
* Enabled some unit tests for linux now when they work as intended regarding redirected stdout/err

[CL 31029918 by henrik karlsson in ue5-main branch]
2024-01-30 19:05:51 -05:00
henrik karlsson
0eaf4b8111 [UBA]
* Fixed some error handling for posix
* Improved some logging
* Disabled some unit tests on linux because they always fail

[CL 30991796 by henrik karlsson in ue5-main branch]
2024-01-30 02:17:34 -05:00
henrik karlsson
9366bc899f [Uba]
* Skip clang test if clang does not exists on machine

[CL 30984644 by henrik karlsson in ue5-main branch]
2024-01-29 21:46:35 -05:00
henrik karlsson
6aeefa8f73 [Uba]
* Disable storage unit test for linux for now. need help getting access to a machine that this fails on.. all machines I've tested on works

[CL 30963185 by henrik karlsson in ue5-main branch]
2024-01-29 02:16:16 -05:00
henrik karlsson
7eaa01d676 [Uba]
* Fixed linux unit test errors

[CL 30878478 by henrik karlsson in ue5-main branch]
2024-01-25 00:54:43 -05:00
henrik karlsson
b2caa842dc [Uba]
* Added more info to test error output

[CL 30872986 by henrik karlsson in ue5-main branch]
2024-01-24 21:18:45 -05:00
henrik karlsson
659c4c9101 [UBA]
* Changed so com memory is not split up in read and write since they never overlap.. so both read and write can use all the memory

[CL 30871523 by henrik karlsson in ue5-main branch]
2024-01-24 20:33:10 -05:00
zack neyland
718701a101 UBA: Remove Touch test from Mac running.
[CL 30699426 by zack neyland in ue5-main branch]
2024-01-18 17:53:11 -05:00
zack neyland
e35aaea9ec UBA: Fix up tests on Mac.
* Need to grab the actual clang binary location and the SDK path.

[CL 30670808 by zack neyland in ue5-main branch]
2024-01-17 17:20:04 -05:00
henrik karlsson
460bdb7637 [UBA/Shader]
* Changed Scheduler_GetStats to separate active into activeLocal and activeRemote
* Added Storage_DeleteFile that can be used to delete cas entries if we know a file will never be used again (no point taking up space)
* Fixed so ubaagents spawned through horde has a idle timeout of 15 seconds (also made them quiet)
* Changed so UbaJobProcessor shutdown uba after 100 seconds instead of 10 seconds of idling without tasks
* Added so .uba.in and .uba.out is deleted from cas storage after used.. no point storing them there since they will never be used again
* Fixed so number of local SCW instances are decreased when more remote helpers are running
* Fixed race condition where tasks could be added when uba was shutting down causing tasks to be cancelled

[CL 30607957 by henrik karlsson in ue5-main branch]
2024-01-12 18:46:59 -05:00
henrik karlsson
22cf61f84b [UBA]
* Added GetNextProcess as a real message type (not using Custom) to be able to stop fetching work if helper is being terminated by aws or if we want to scale down number of workers
* Did some cleanup in the Tcp backend code and made sure not to call close socket multiple times (since it can cause the code to close a different socket than what is owned)

[CL 30584707 by henrik karlsson in ue5-main branch]
2024-01-11 21:01:56 -05:00
henrik karlsson
9b81535da2 [UBA]
* Added unit test for scheduler

[CL 30523388 by henrik karlsson in ue5-main branch]
2024-01-09 20:29:17 -05:00
henrik karlsson
ea673427b2 [UBA]
* Implemented process reuse logic in UbaScheduler. It is now possible for running processes to fetch more work
* Fixed bugs in custom message path
* Fixed bugs in FlushWrittenFiles message
* Changed So UpdateEnvironment is resetting stats
* Fixed potential race condition related to process reuse
* Added special rule for ShaderCompileWorker.exe which detours ImageGetDigestStream in Imagehlp.dll (because wine implementation does not match windows implementation)
* Renamed "exitedUserData" to just userData since it is used for more than when exiting process
* Added a couple more dlls to "known system dlls"
* Fixed so visualizer can visualize process reuse properly

[CL 30462059 by henrik karlsson in ue5-main branch]
2024-01-02 17:42:42 -05:00
henrik karlsson
9a536983eb [Uba]
* Removed usage of "select" in tcp socket connect code because it can't handle file descriptors over 1024 so segfaults (phew, hard one to find)
* Added option to have clients send back log to server (SessionServerCreateInfo.remoteLogEnabled) .. this forced a network protocol bump
* Improved asserts for when network messages are corrupt
* Some small fixes of asserts in debug in win32 detoured code

[CL 30343891 by henrik karlsson in ue5-main branch]
2023-12-15 02:18:13 -05:00
henrik karlsson
f86ddc7b73 [UBA]
* Added code to prevent recursion of already handled directories (caused by symlinks)
* Fixed path issue with double forward slashes on macos
* Added unit test for double slash
* Improved assert description

[CL 30254027 by henrik karlsson in ue5-main branch]
2023-12-11 19:35:57 -05:00
henrik karlsson
ca04570337 [Uba]
* Fixed handling of paths that has dirs that has double dots in it.. still a valid path

[CL 30247269 by henrik karlsson in ue5-main branch]
2023-12-11 15:21:10 -05:00
henrik karlsson
d3d5176d3b [UBA]
* Implement simple commandline parser and added unit tests
* Removed usage of wordexp on non-windows platforms because it seems to have a _huge_ penalty on macos..
* Added trace scope around WriteFilesToDisk
* Fixed assert in stdout/err pipes on macos and added handling for poll hangup
*

[CL 30225204 by henrik karlsson in ue5-main branch]
2023-12-09 03:07:04 -05:00
henrik karlsson
37b5c7298e [Uba]
* Implemented RunProcess which can be used to spawn and wait on raw processes (not detoured). (Note, calling program needs to report file system modifications since it is not known to UBA)
* Changed so UBAExecutor is using UBA for non-detoured processes as well. This will give correct profiling tracking and reuse existing code.
* Fixed handle leak in windows
* Fixed race condition on linux since wordexp() is n ot thread safe
* Added tests
* New binaries

[CL 30216046 by henrik karlsson in ue5-main branch]
2023-12-08 13:44:47 -05:00
henrik karlsson
e9238f52e0 [UBA]
* Fixed uba unit test compile error

[CL 30144731 by henrik karlsson in ue5-main branch]
2023-12-05 20:09:55 -05:00
henrik karlsson
909f4ef9e0 [UBA]
* Fixed bug with time stamps on linux.. was using seconds accuracy
* Fixed a few bugs on linux paths related to not updating directory caches properly when files changed
* Fixed so asserts are logging properly on linux
* Added debug logger for session side to be able to track all tracking of file caches (disabled by default)

[CL 30104061 by henrik karlsson in ue5-main branch]
2023-12-04 19:43:52 -05:00
henrik karlsson
348667e01f [UBA]
* Fixed so visualizer is showing correct ping numbers when showing linux builds

[CL 30069648 by henrik karlsson in ue5-main branch]
2023-12-03 02:50:01 -05:00
henrik karlsson
7262ce9635 [Uba]
* Fixed usage of wrong define name
* Renamed options in batchfiles using old options
* Removed old code not used
* Added one more test to the list of "all tests"

[CL 30019208 by henrik karlsson in ue5-main branch]
2023-11-30 13:11:39 -05:00
henrik karlsson
c155e57afc [Uba]
* Added unit test for inverted connect path (client listen and server connects)
* Changed so AddClient waits for connect to succeed/fail before returning

[CL 30018826 by henrik karlsson in ue5-main branch]
2023-11-30 12:58:02 -05:00