Commit Graph

141 Commits

Author SHA1 Message Date
henrik karlsson
5c8c7121f1 [UBA]
* Fixed allocation to make sure it honors alignments
* Added error handling in trace code. It will stop writing when running out of shared mem

[CL 31533514 by henrik karlsson in ue5-main branch]
2024-02-15 15:36:01 -05:00
henrik karlsson
6a0ab32772 [UBA]
* Reverting some allocations to use new[] on windows. There has been weird hangs on the farm since last binaries and this is one of the changes that I can imagine can make a difference.

[CL 31525372 by henrik karlsson in ue5-main branch]
2024-02-15 13:03:29 -05:00
henrik karlsson
532603ab8c [UBA]
* Fixed code to handle extremely long error log entries

[CL 31507612 by henrik karlsson in ue5-main branch]
2024-02-14 20:54:26 -05:00
henrik karlsson
5673117e65 [UBA]
* Added code to handling uba::ReadFile trying to read file with less bytes than requested. It will try for 3 seconds and then fail with a message saying it read 0 bytes for 3 seconds
* Added more information to error messages

[CL 31461069 by henrik karlsson in ue5-main branch]
2024-02-14 01:37:37 -05:00
henrik karlsson
141b357011 [Uba]
* Added code to validate that connections from server to client are all from the same server. Solved by giving servers a unique id and made sure it is part of connection handshake with client. Client will just discard connections that is providing a different server uid than the first connection

This will hopefully fix the bug seen on our farm a couple days ago. Theory is that two servers managed to connect to the same helper. First server is just about to start connecting to a helper when helper is brought down and restarted to help another server. Some sort of stall happens on first server and once it starts the actual tcp connection the helper has been brought up to help the other server. The helper allows more than one tcp connection for performance reasons so will accept tcp connection from both old and new server. In this messed up scenario some messages would go to one server and some to the other.. causing all kinds of weird things. It is critical that all connections go to the same server since all messages are just round robin the tcp connections.

[CL 31460950 by henrik karlsson in ue5-main branch]
2024-02-14 01:23:19 -05:00
henrik karlsson
217b710d5a [UBA]
* Added code to silence STATUS_NO_MEDIA_IN_DEVICE errors when caching folders.

[CL 31387245 by henrik karlsson in ue5-main branch]
2024-02-12 12:19:14 -05:00
henrik karlsson
13fd19980f [UBA]
* Fixed mac compile errors by adding explicit ctor

[CL 31374671 by henrik karlsson in ue5-main branch]
2024-02-11 18:45:11 -05:00
henrik karlsson
28a07b16d2 [Uba]
* Fixed crash in trace system when log lines are insanely long (over 256k)

[CL 31374618 by henrik karlsson in ue5-main branch]
2024-02-11 18:31:11 -05:00
henrik karlsson
2ecaf5607e [UBA]
* Fixed unit test compile errors
* Fixed race condition in unit test shutdown

[CL 31374569 by henrik karlsson in ue5-main branch]
2024-02-11 18:22:11 -05:00
henrik karlsson
eda007f51f [UBA]
* Added support for most features in UbaScheduler. Dependencies, weights etc etc.
* Added support for loading yaml file with processes for UbaScheduler.
* Added so UbaCli interprets yaml file as a file with processes use it to populate a scheduler which is then executed
* Fixed so all threads inside same process spawned by uba ends up in the same thread group.
* Fixed linux crash where process comunication memory was deleted when cancel event was called (added lock around code)
* Fixed deadlock that could happen if flush dead processes were called at the after lock but before processhandle dtor in Session::ProcessExited
* Changed new[]/delete[] to aligned_alloc/free because for some reason new/delete trigger asan on linux and don't know why.

[CL 31372220 by henrik karlsson in ue5-main branch]
2024-02-11 04:00:51 -05:00
henrik karlsson
bbbcf50893 [UBA]
* Fixed so GetFullPathNameW for linux works for non-existing paths (we just want it to fix it up)

[CL 31368282 by henrik karlsson in ue5-main branch]
2024-02-10 12:51:07 -05:00
henrik karlsson
c5b06cbba0 [UBA]
* Added more information for when process dies without sending exit message

[CL 31367591 by henrik karlsson in ue5-main branch]
2024-02-10 12:38:52 -05:00
henrik karlsson
f4741cbdf6 [UBA]
* Fixed memory stomp in code writing memory to compressed file. Found by asan on linux

[CL 31358532 by henrik karlsson in ue5-main branch]
2024-02-09 17:18:03 -05:00
henrik karlsson
34b52deafb [UBA]
* Added hint to unmapfileview to be able to add useful information to asserts on fails
* Added error handling for vsnprintf returning error
* Fixed memory stomp in unit test
* Removed usage of %hs on non-windows since asan complains about it
* Changed UBA_USE_MIMALLOC to always be defined and be 0 or 1 instead of using ifdef
* Enabled mimalloc on linux

[CL 31355077 by henrik karlsson in ue5-main branch]
2024-02-09 16:12:12 -05:00
henrik karlsson
c48ddbdd7d [Uba]
* Fixed so uba uses virtual file index numbers of "real" index is (u64)-1. This can be used for systems like ProjFS where we don't know the file index until it is hydrated but we need a proper number in order for applications like clang to work properly (clang uses file index as unique key for files)
* Added some more debug code

[CL 31297871 by henrik karlsson in ue5-main branch]
2024-02-08 14:12:18 -05:00
henrik karlsson
cca66a4ea3 [UBA]
* Removed logging entries that was only added to make horde not timeout.. since this was implemented horde has added ping logic so this is not needed anymore
* Fixed bug in callstack logging on posix
* Changed a bunch of %hs to %s
* Added allocator debug code to try to figure out asan issues

[CL 31281576 by henrik karlsson in ue5-main branch]
2024-02-07 20:36:18 -05:00
henrik karlsson
b3e84767c7 [UBA]
* Added support for PagingFiles registry key containing drive letter '?'. Not sure exactly what this mean except that it can exist if system automatically manage page files for all drives. We resort to use first entry in ExistingPageFiles registry key to figure out drive

[CL 31198385 by henrik karlsson in ue5-main branch]
2024-02-05 19:09:12 -05:00
sergio gardeazabal
9728fc156b [UBA] Disable power throttling in created processes to ensure P-Cores are preferred over E-Cores on Intel Hybrid architectures platforms
#jira UE-205966

#rb henrik.karlsson

[CL 31193922 by sergio gardeazabal in ue5-main branch]
2024-02-05 17:56:36 -05:00
henrik karlsson
b4552f4816 [UBA]
* Fixed logic calculating max page file size. System-managed page file logic was wrong giving calculations that were way to high

[CL 31161437 by henrik karlsson in ue5-main branch]
2024-02-03 23:55:33 -05:00
henrik karlsson
03e44346be [UBA]
* Fixed bug where error messages written to trace stream could be bigger than max size for a write scope. Previous number was 4k, now changed to 256k instead (I don't expect an individual error message to be bigger than 256k at least!)

[CL 31135204 by henrik karlsson in ue5-main branch]
2024-02-02 14:32:34 -05:00
henrik karlsson
b35413a335 [UBA]
* Proper fix of bug where code could write 8 bytes outside memory allocation

[CL 31127482 by henrik karlsson in ue5-main branch]
2024-02-02 11:27:44 -05:00
henrik karlsson
85b847a049 [Uba]
* Fixed bug in storage client code that directly decompress network data to disk/mem. It could write outside buffer by 8 bytes
* Added UbaCli -checkcas2  .. this codepath creates a storage server and client and networks all cas files over.. just to check so code paths properly work
* Fixed UbaCli -checkaws bug

[CL 31118503 by henrik karlsson in ue5-main branch]
2024-02-02 02:00:29 -05:00
henrik karlsson
59cd3b1849 [UBA]
* Added support for custom text in trace/visualizer... can be used to report things like horde status etc

[CL 31118070 by henrik karlsson in ue5-main branch]
2024-02-02 00:30:28 -05:00
henrik karlsson
85837b6ef5 [UBA]
* Fixed so renaming from castemp to real destination handle cross-device

[CL 31058118 by henrik karlsson in ue5-main branch]
2024-01-31 14:16:07 -05:00
henrik karlsson
eee33ba668 [UBA]
* Fixed so stdout/err is redirected to pipe on detoured processes on linux/mac.. this should fix so errors are visualized in the right place
* Added -nostdout to UbaCli to be able to check that redirected stdout/err works
* Added version for process messages to catch issues where ppl have old ubaagent.exe but new ubadetours.dll
* Added capacity parameter to FixPath and added asserts to check that we never write outside capacity
* Fixed code creating g_virtualApplication buffer.
* Disabled asserts for mac non debug builds
* Removed detoured write/fwrite for posix now when we instead redirect stdout/err
* Enabled some unit tests for linux now when they work as intended regarding redirected stdout/err

[CL 31029918 by henrik karlsson in ue5-main branch]
2024-01-30 19:05:51 -05:00