Commit Graph

104 Commits

Author SHA1 Message Date
henrik karlsson
1bd7940eef [UBA]
* Fixed so bad file path entries are not saved to casdb file
* Added so allow proxy can be set from config files
* Added assert for if bad files have entered the cas entry directory on disk

[CL 34476627 by henrik karlsson in ue5-main branch]
2024-06-18 16:23:41 -04:00
henrik karlsson
81920d7856 [UBA]
* Fixed validation code in scheduler for process reuse
* Moved all code inside ServerSession::HandleMessage into one function per message to more easily be able to read callstacks
* Added more validation around missing file errors.. now server logs out lots of information too

[CL 34414815 by henrik karlsson in ue5-main branch]
2024-06-17 02:02:04 -04:00
henrik karlsson
0e04dc3843 [UBA]
* Added option to keep files in memory
* Added option to allow using output files (transferred via handle to session process)
* Added option to suppress logging

[CL 34393564 by henrik karlsson in ue5-main branch]
2024-06-15 00:56:16 -04:00
henrik karlsson
4ae98f4792 [UBA]
* Added support for GetLongPathName and RemoveDirectory for remote helpers

[CL 34393494 by henrik karlsson in ue5-main branch]
2024-06-15 00:37:17 -04:00
henrik karlsson
0a94bfe21e [UBA]
* Fixed so default constructed StringView can still be used to do EndsWith and Contains

[CL 34393465 by henrik karlsson in ue5-main branch]
2024-06-15 00:17:52 -04:00
henrik karlsson
7647b785f2 [UBA]
* Added support for strings and comments in config files
* Added tests
* Fixed for uba to work integrated in other programs (proper dll exports)
* Added process id to process info in visualizer

[CL 34364596 by henrik karlsson in ue5-main branch]
2024-06-14 02:35:27 -04:00
henrik karlsson
9333b3232e [UBA]
* Added support for having multiple tilde in same path
* Added IsAbsolutePath function

[CL 34364566 by henrik karlsson in ue5-main branch]
2024-06-14 02:30:57 -04:00
henrik karlsson
c947275a30 [UBA]
* Added uba::Parse to parse u64 from string

[CL 34262305 by henrik karlsson in ue5-main branch]
2024-06-10 20:35:11 -04:00
henrik karlsson
acd0701e3d [UBA]
* Added timer scope around compressed obj files preload

[CL 34189252 by henrik karlsson in ue5-main branch]
2024-06-07 00:04:54 -04:00
henrik karlsson
5c2d22ba69 [UBA]
* Reserve memory for elements that we know will be allocated in dir table and filemapping table

[CL 34132310 by henrik karlsson in ue5-main branch]
2024-06-05 13:58:58 -04:00
henrik karlsson
791502389d [UBA]
* Attempt at fixing problem with cache client seeing the wrong file when writing to cache. Problem is related to a race condition between directory crawler threads and writing files. We can end up in this scenario
1. Crawler query directory, file information gets stored in temporary state. and crawler starts traversing state
2. File is written and new date is set to file on disk, and file db entry is set to verified false.
3. Crawler hits the same file and now checks temporary state with file db entry and notice the lastwritten is the same, and set it back to verified. (but it is not the same)

[CL 34132081 by henrik karlsson in ue5-main branch]
2024-06-05 13:52:28 -04:00
henrik karlsson
fdd2fd52c4 [UBA]
* Renamed system stats to kernel stats
* Changed all stats storing to use bitfield first to say which fields that are non-zero

[CL 34058881 by henrik karlsson in ue5-main branch]
2024-06-02 18:14:50 -04:00
henrik karlsson
6baefebe6d [UBA]
* Added ToStringKeyLower that takes in char* and length

[CL 34058835 by henrik karlsson in ue5-main branch]
2024-06-02 17:59:16 -04:00
henrik karlsson
ef41a5b583 [UBA]
* Changed so posix targets use clock_gettime(CLOCK_MONOTONIC, x) instead of gettimeofday in order to make sure time is always showing increasing number (and I guess it is more accurate)

[CL 34048427 by henrik karlsson in ue5-main branch]
2024-05-31 18:08:19 -04:00
henrik karlsson
f071620893 [UBA]
* Bumped directory table mem size from 40mb to 48mb to be able to handle builds that includes tons and tons of targets

[CL 34048051 by henrik karlsson in ue5-main branch]
2024-05-31 17:54:30 -04:00
henrik karlsson
233c2cddb2 [UBA]
* Added stats tracking for get file attributes to get more information on what is stalling on the farm
* Enabled storage and system stats for cache misses

[CL 34020773 by henrik karlsson in ue5-main branch]
2024-05-30 19:43:04 -04:00
henrik karlsson
2e72e16d1d [UBA]
* Fixed bug in argument parser when reading rsp file that does not have a string termination character in the end

[CL 33960224 by henrik karlsson in ue5-main branch]
2024-05-28 20:01:03 -04:00
henrik karlsson
639a380cda [UBA]
* Fixed static analysis errors

[CL 33948663 by henrik karlsson in ue5-main branch]
2024-05-28 15:07:55 -04:00
henrik karlsson
502a9e44e2 [UBA]
* Added StringView type and replaced a ton of string+stringLen to StringView
* Fixed bug in obj file preloader.
* Fixed so clang targets can used compressed obj files
* Fixed bug in cache client when entry count is zero
* Added magic to compressed obj file header to be able to see if an obj file is compressed or not
* Moved ParseArguments to its own file and added so it can handle both char and wchar

[CL 33933853 by henrik karlsson in ue5-main branch]
2024-05-27 23:22:26 -04:00
henrik karlsson
fddb85fe3f [Uba]
* First round of optimizations to uba cache server maintenance code.

[CL 33900598 by henrik karlsson in ue5-main branch]
2024-05-24 14:04:37 -04:00
henrik karlsson
fc56ce5b68 [UBA]
* Fixed deadlock that could happen when lots of clients are connecting while maintenance is running
* Added UbaCli -cachecommand=<cmd> that can be used to query status of cache service
* Fixed code setting compressed byte on caskey to only use a bit
* Changed so a bit on the caskey can tell if a file is normalized or not
* Added stats for writing to cache to trace file
*

[CL 33880362 by henrik karlsson in ue5-main branch]
2024-05-23 19:16:15 -04:00
henrik karlsson
09ddadcee3 [UBA]
* Fixed bug in cache client where it could create a caskey for a file that should always go through ShouldNormalize that matches caskey of non-normalized file
* Added timers around traverse directory calls to be able to see how much time that is spent in traversing directories

[CL 33855227 by henrik karlsson in ue5-main branch]
2024-05-23 01:51:09 -04:00
henrik karlsson
a698ad4a9c [UBA]
* Added exclusions for exports in UbObjTool and added D3D12SDKVersion and friends to exclusion list
* Fixed so UbaObjTool works for linux (or builds at least)
* Added version to importexport files

[CL 33840177 by henrik karlsson in ue5-main branch]
2024-05-22 15:22:17 -04:00
henrik karlsson
da17a5644c [UBA]
* Added so ispc compiles also creates .exi files when compiling with merged modules.

[CL 33800223 by henrik karlsson in ue5-main branch]
2024-05-21 13:05:57 -04:00
henrik karlsson
e65e55d6e7 [UBA]
* Fixed so ubaobjtool works with compressed obj files
* Removed unused code

[CL 33787648 by henrik karlsson in ue5-main branch]
2024-05-21 01:28:52 -04:00