[ComputeFramework] Treat preprocessor issues as warnings if compile succeeded, otherwise errors, which is the standard rule.
[PCG] Adapt error summary to whatever line information is available in the error (if any).
#rb dan.elksnitis, Jeremy.Moore, Laura.Hermanns, wyatt.marvil
#rnx
[CL 35362997 by huw bowles in ue5-main branch]
- fix line number mismatches caused due to multiline macro arguments by keeping leading/trailing whitespaces in cases of multiple lines encountered in a single macro argument (single line macro args still have whitespace trimmed as this is needed for token pasting)
- avoid early out preventing parsing of "defined" builtin macro outside of #if conditions; this allows using defined in #define directives (among other places)
#rb Yuriy.ODonnell
[CL 32051438 by dan elksnitis in ue5-main branch]
* Moved identifier copy and macro bloom filter from maybe_expand_macro to test into copy_to_action_point / copy_to_action_point_macro_expansion. 13.1% of improvement.
* SSE implementation of scan_to_directive, 10x faster, 5.2%
* SSE implementation of identifier copy, 3x faster, 4.5%
* SSE ShaderConvertAndStripComments, 4x faster, 3.6%
* Fast inline string equality comparison, 4x faster, 1.5%
To make SSE implementations "safe" without needing special cases near the end of a buffer, it's necessary to ensure padding is present in the relevant buffers, anything that goes through a preprocess_string call. This includes the string arena allocator, temporary stbds arrays that hold strings, and file buffers passed in. The latter all pass through ShaderConvertAndStripComments, where we can add padding. (ShaderConvertAndStripComments itself has special cases for end of buffer). Code related to original 1 and 2 character macro filter removed, since I can't see a reason to enable it over the bloom filter.
I also attempted SSE optimization of copy_to_action_point and copy_line_without_comments, but improvement wasn't big enough to be worth the complexity (around 2% for the former, but massive code complexity, 0.5% for the latter). That's pretty much everything SSE friendly that's over 1% on a profile, although I think copy_argument can be made a lot faster, not primarily through SSE.
#jira UE-197212
#rnx
#rb yuriy.odonnell jason.nadro
[CL 28834324 by jason hoerner in ue5-main branch]
* Low overhead bloom filter for identifier hash lookups in maybe_expand_macro (3.1%)
* Optimized inline version of library calls (isspace, isalnum), and avoid strtoul for single digit numbers in evaluate_if (2.6%)
* Array reserve function inlining (1.7%)
* Fetch output size from preprocessor library instead of calling strlen (1.2%)
* Avoid FName -> string -> FName round trip conversion of key in FShaderCompilerEnvironment::Merge (0.3%)
* Use string concatentation instead of printf and inline storage in AddStbDefines (0.3%)
* Other overhead reduction (remove unused fast_dest feature, smaller stbds_array_header struct, savings from inlining) (1.5%)
#rnx
#rb dan.elksnitis jason.nadro
[CL 28537574 by jason hoerner in ue5-main branch]
* Inline array memory allocation added to low level preprocessor for output and various temporary buffers to reduce dynamic memory allocation and reallocation overhead. Saved 4.6%.
* FShaderPreprocessOutput::StripCode optimized to write to FString as TCHAR array, rather than using AppendChar (over 4x speedup). Saved 2.9%
* Shader source file cache now also stores stripped and ANSI converted source, to avoid need to convert and strip the source, plus allocating a copy is avoided. Saved 4.3%
* Uniform buffer structure declarations stored as ANSI converted source, avoiding convert and copy. Saved 4.9%
#rnx
#rb dan.elksnitis jason.nadro
[CL 28219741 by jason hoerner in ue5-main branch]
- modify versioning mechanisms to use UEMETADATA pragmas
- update commands which generate version.ush files to generate such pragmas instead of comments
- include the above pragma-driven version(s) in the input hash when preprocessed job cache is enabled (not needed for the disabled case since as before the entire contents of the version files are hashed)
- fix bug in stb_preprocessor that was causing it to stop expanding macros if any diagnostic was encountered (and rename the array field storing diagnostic messages to diagnostics, since it contains more than just errors)
#rb Yuriy.ODonnell
[CL 27515060 by dan elksnitis in ue5-main branch]
