Commit Graph

35 Commits

Author SHA1 Message Date
henrik karlsson
f1ee094c7e [UBA]
* Fix for when absolute path was sent in to SearchPathForFile

[CL 34394134 by henrik karlsson in ue5-main branch]
2024-06-15 03:26:48 -04:00
henrik karlsson
9333b3232e [UBA]
* Added support for having multiple tilde in same path
* Added IsAbsolutePath function

[CL 34364566 by henrik karlsson in ue5-main branch]
2024-06-14 02:30:57 -04:00
henrik karlsson
e573e1da40 [UBA]
* Fixed so mode is propagated when closing file for write that failed to use rename
* Changed default file mode to mode  include S_IRGRP and S_IROTH

[CL 34262059 by henrik karlsson in ue5-main branch]
2024-06-10 20:26:41 -04:00
henrik karlsson
fdd2fd52c4 [UBA]
* Renamed system stats to kernel stats
* Changed all stats storing to use bitfield first to say which fields that are non-zero

[CL 34058881 by henrik karlsson in ue5-main branch]
2024-06-02 18:14:50 -04:00
henrik karlsson
233c2cddb2 [UBA]
* Added stats tracking for get file attributes to get more information on what is stalling on the farm
* Enabled storage and system stats for cache misses

[CL 34020773 by henrik karlsson in ue5-main branch]
2024-05-30 19:43:04 -04:00
henrik karlsson
f90752c220 [UBA]
* Added more stats to cache fetch to be able to figure out where time goes
* Changed code to use FileExists instead of GetFileInformation.. since we only need information that can be fetched with FileExists
* Removed usage of sprintf when creating cas filename

[CL 33979482 by henrik karlsson in ue5-main branch]
2024-05-29 13:31:50 -04:00
henrik karlsson
1273801622 [UBA]
* Added -expiration=<seconds> to UbaCacheService.. This controls when cache entries should be deleted. If an entry has been unused for more than <expiration> time it gets deleted. Defaults to two days.
* Added reason why cache entries are deleted
* Changed so cas database is always checked in startup.

[CL 33963630 by henrik karlsson in ue5-main branch]
2024-05-29 01:07:06 -04:00
henrik karlsson
ac8207c873 [Uba]
* Removed GetNowFileTime. GetSystemTimeAsFileTime should be used instead
* Fixed some minor bugs in uba cache server related to times
* Added obliterate command to cache server

[CL 33916906 by henrik karlsson in ue5-main branch]
2024-05-25 22:30:11 -04:00
henrik karlsson
cf462f5bfc [UBA]
* Added support for extracting import/export symbols from obj file and strip it from exports at the same time

[CL 33761137 by henrik karlsson in ue5-main branch]
2024-05-19 23:12:02 -04:00
henrik karlsson
7c61b9c51d [UBA]
* Improved MAKE_LONG_FILENAME to also take care of \.\ and \..\ properly

[CL 33144544 by henrik karlsson in ue5-main branch]
2024-04-22 13:52:46 -04:00
henrik karlsson
a5ab953437 [Uba]
* Fixed so code adding \\?\ also fix path separator type from / to \ if forward slashes exists

[CL 33119303 by henrik karlsson in ue5-main branch]
2024-04-19 18:00:16 -04:00
henrik karlsson
a4659181c4 [UBA]
* Fixed so absolute paths (paths with ':' as second character) are prefixed with \\?\ on windows. This to hopefully solve problems with systems that does not have long paths enabled but still end up with long paths

[CL 33109893 by henrik karlsson in ue5-main branch]
2024-04-19 14:03:20 -04:00
henrik karlsson
9b17fef999 [UBA]
* Added support for storing .obj files compressed by cl.exe and then decompressed in-memory by link.exe. Note this is not properly implemented and only work for cl.exe+link.exe combo

[CL 32625054 by henrik karlsson in ue5-main branch]
2024-03-29 17:49:27 -04:00
henrik karlsson
dd0af54454 [UBA]
* Fixed bug where pipes between processes did not work properly on wine
* Added unit tests for pipes (win only)
* Improved error message when failing to open file
* Fixed so .obj is seen as output files from ispc.exe
* Fixed up application path when creating child processes

[CL 32537147 by henrik karlsson in ue5-main branch]
2024-03-27 01:44:06 -04:00
henrik karlsson
b9ec730a06 [UBA]
* Changed so macos is using ftruncate instead of lseek+write.  This fixes failed unit test on some macs
* Removed locks in UbaNetworkBackendMemory to fix potential deadlock
* Added StringBuffer::GetFileName since it is a common usecase when doing debug printing
* Added missing close of file descriptor in error situation

[CL 32191183 by henrik karlsson in ue5-main branch]
2024-03-12 16:37:13 -04:00
henrik karlsson
5e6dc44ec3 [UBA]
* Added PrefetchVirtualMemory function
* Fixed bug where code writing callstacks checks end of module to figure out if address belongs to module
* Changed so FileAccessor::CreateMemoryWrite set always allow read because you can't create file mappings that are write-only
* Added option to scheduler to force all runs to run native
* Fixed so bottleneck class has enter/leave function
* Added ScopedReadLock::Enter
* Moved LogWriter to its own file
* Visualizer - Fixed so when hovering over timeline time under pointer is showing
* Visualizer - Fixed so hovering over work records shows name, start and duration
* Visualizer - Added context menu item that can show/hide text of process bars
* Improved assert message
* Changed so lock is not on stack but is leaked to prevent shutdown warnings in debug
* Added missing includes

[CL 31981637 by henrik karlsson in ue5-main branch]
2024-03-03 20:42:41 -05:00
henrik karlsson
0b12758059 [UBA]
* Changed so all ScopedReadLock and ScopedWriteLock are macros (SCOPED_READ_LOCK and SCOPED_WRITE_LOCK).. this to be able to add code around the lock to measure contention. Contention testing code can be enabled by setting UBA_TRACK_CONTENTION to 1

[CL 31795889 by henrik karlsson in ue5-main branch]
2024-02-26 01:14:42 -05:00
henrik karlsson
153bc16e50 [Uba]
* Added O_CLOEXEC to all open to prevent file descriptors from leaking to child processes. This seems to really fix the ETXTBUSY
* Added read hint for DecompressMemoryToMemory to be able to show better error messages

[CL 31636371 by henrik karlsson in ue5-main branch]
2024-02-19 20:43:29 -05:00
henrik karlsson
5673117e65 [UBA]
* Added code to handling uba::ReadFile trying to read file with less bytes than requested. It will try for 3 seconds and then fail with a message saying it read 0 bytes for 3 seconds
* Added more information to error messages

[CL 31461069 by henrik karlsson in ue5-main branch]
2024-02-14 01:37:37 -05:00
henrik karlsson
bbbcf50893 [UBA]
* Fixed so GetFullPathNameW for linux works for non-existing paths (we just want it to fix it up)

[CL 31368282 by henrik karlsson in ue5-main branch]
2024-02-10 12:51:07 -05:00
henrik karlsson
34b52deafb [UBA]
* Added hint to unmapfileview to be able to add useful information to asserts on fails
* Added error handling for vsnprintf returning error
* Fixed memory stomp in unit test
* Removed usage of %hs on non-windows since asan complains about it
* Changed UBA_USE_MIMALLOC to always be defined and be 0 or 1 instead of using ifdef
* Enabled mimalloc on linux

[CL 31355077 by henrik karlsson in ue5-main branch]
2024-02-09 16:12:12 -05:00
henrik karlsson
0eaf4b8111 [UBA]
* Fixed some error handling for posix
* Improved some logging
* Disabled some unit tests on linux because they always fail

[CL 30991796 by henrik karlsson in ue5-main branch]
2024-01-30 02:17:34 -05:00
henrik karlsson
ade0725e14 [UBA]
* Fixed so imagehlp.dll and dbghelp.dll are detoured when loaded. Detour ImageGetDigestStream and SymLoadModuleExW because both of them cause trouble on wine
* Fixed so reuse of processes also honor log files so a new log file is created when reuse happen
* Improved log file naming so host can set name and log files are sent back properly with the right name

[CL 30638193 by henrik karlsson in ue5-main branch]
2024-01-16 13:24:28 -05:00
henrik karlsson
460bdb7637 [UBA/Shader]
* Changed Scheduler_GetStats to separate active into activeLocal and activeRemote
* Added Storage_DeleteFile that can be used to delete cas entries if we know a file will never be used again (no point taking up space)
* Fixed so ubaagents spawned through horde has a idle timeout of 15 seconds (also made them quiet)
* Changed so UbaJobProcessor shutdown uba after 100 seconds instead of 10 seconds of idling without tasks
* Added so .uba.in and .uba.out is deleted from cas storage after used.. no point storing them there since they will never be used again
* Fixed so number of local SCW instances are decreased when more remote helpers are running
* Fixed race condition where tasks could be added when uba was shutting down causing tasks to be cancelled

[CL 30607957 by henrik karlsson in ue5-main branch]
2024-01-12 18:46:59 -05:00
henrik karlsson
9b1e22319d [UBA]
* Fixed so paths returned from SearchPathForFile are cleaned up (not having .. etc)
* Fixed so UbaAgent handle WSAPoll POLLERR as timeout so it retries when failing (this made the process exit on wine)
* Fixed so returned processes that had been reused returned the reused process to the scheduler
* Fixed so EnsureBinaryFile is using correct fileNameKeys
* Removed assert in detoured GetFullPathNameA since it is always calling GetFullPathNameW

[CL 30465533 by henrik karlsson in ue5-main branch]
2024-01-03 16:23:58 -05:00