- add handling of #pragma directives in macro definitions
- add handling of #pragma message to create diagnostic outputs
- add and update comments in preprocessor.h
- unifying memory allocation mechanisms between stb_ds, stb_alloc and preprocessor - allow configuring all functions performing memory allocation/freeing via macros which should be specified via a header whose include path is specified via the STB_CONFIG macro (this will be included in stb_common.h if defined)
- don't pass an stb_arena to the include resolution callback; this is unnecessary complexity (implementation can handle its own memory allocation as long as the resulting strings have the appropriate lifetime) and indeed the arena used was global (and so not threadsafe)
- expose function allowing external modification of error modes so we can disable the error case for preprocessor directives not being at the start of a line (and so handle the case where we #define a macro whose value is a #pragma, or any other string possibly containing a # that we want to pass unmodified to the output i.e. the case of #define COMPILER_DEFINE #define which we use to keep #defines in the preprocessed code for handling by the platform compilers)
- fix leak of pp diagnostic messages
- remove internal caches for include resolution; they are problematic since (a) they aren't threadsafe, and (b) the caching of resolved filenames based on the include path found in source is inherently problematic (can potentially cause issues with same-named files in different folders included via relative paths)
- strip all code related to explicit file loading and lists of include paths; this has no purpose for our use case. note this makes the callbacks for file load/unload and include resolve mandatory, comments in preprocessor.h updated to reflect this.
- fix a few other memory leaks
#preflight 638611a9fa053c489a54b2ec
#rb Yuriy.ODonnell
[CL 23305324 by dan elksnitis in ue5-main branch]
- add explicit extern "C" on preprocessor methods to allow linking from C++ code
- add custom callbacks for include path resolution and file loading/unloading
- add padding space following macro expansion ending with a > to work around DXC bug (not handling >> correctly if it occurs at the end of a templated type)
- adding explicit handling for HLSL infinity constant (1.#INF); the # character outside of a directive was previously putting the preprocessor into an error state
- change line directive output to use #line format rather than # (DXC is ok with the latter but minifier doesn't currently handle it), and also always include the filename (so when minifying don't have to be careful not to strip the line directive with the filename in it); this also matches mcpp behaviour
- adding const-correctness and casts where needed to avoid const casts/compile errors in calling C++ code and satisfy Mac/Intellisense compilers
- bugfix: parsing of '.' was immediately putting the scanning state machine into "number" mode resulting in skipped macro expansion in code of the form "var.MACRO.var2"/numbers; added extra state transitions and changing the default transition when encountering '.' to an explicit mode that can allow either transitioning to parsing identifiers or numbers depending on following characters.
- bugfix: macros with an empty parameter list were incorrectly terminating said list on expansion (adding extra rparens)
- bugfix: set file load mode as "callback" when callback used for loading, and check the mode in the free callbacks
- various minor changes to satisfy Clang code analysis
#preflight 637e3391e30d438849ddf902
#rb Yuriy.ODonnell
[CL 23249332 by dan elksnitis in ue5-main branch]
