Every call to shader_instruction_array_insert_at() means a possible
reallocation of all vsir instructions in the program. This means that all
previous pointers are potentially no longer valid.
We are currently using these potentially invalid pointers in some cases,
usually in the form of "ins->location". This commit fixes these.
I moved all pointer changes to right after the call to
shader_instruction_array_insert_at() to make this more evident.
A good part of the I/O normaliser job is to merge together signature
elements that are spanned by DCL_INDEX_RANGE instructions. The
current algorithm assumes that each index range touches exactly
one signature element for each index spanned by the range. The
assumption is used in shader_signature_merge() in the form of
expecting that, if the index range is N registers long, then,
once you find the first signature element of an index range, the
other elements that will have to be merged with it are exactly the
following N-1 according to the order given by
signature_element_register_compare() or
signature_element_mask_compare(), depending on the signature type.
This doesn't necessarily happen. For example, The Falconeer has a
few hull shaders in which this happens:
hs_fork_phase
dcl_hs_fork_phase_instance_count 13
dcl_input vForkInstanceId
dcl_output o4.z
dcl_output o5.z
dcl_output o6.z
dcl_output o7.z
dcl_output o12.z
dcl_output o13.z
dcl_output o14.z
dcl_output o15.z
dcl_output o16.z
dcl_output o17.z
dcl_output o18.z
dcl_output o19.z
dcl_output o20.z
dcl_temps 1
dcl_index_range o4.z 17
iadd r0.x, vForkInstanceId.x, l(4)
ult r0.y, vForkInstanceId.x, l(4)
movc r0.x, r0.y, vForkInstanceId.x, r0.x
mov o[r0.x + 4].z, l(0)
ret
Here the index range "skips" o8.z through o11.z, because those
registers only use mask .xy. The current algorithm fails on such
a shader.
Even depending on the signature element order doesn't look ideal.
I don't have a full counterexample for that, but it looks fragile,
especially given that the register allocation algorithm in FXC is
notoriously full of unexpected corner cases.
We solve both problems by slightly changing the architecture of
the normaliser: first we move computing the masks for the merge
signature element from signature_element_range_expand_mask(),
which is executed while merging signature, to
io_normaliser_add_index_range(), which is executed before merging
signatures. Then, while we are merging signatures, we can decide
for each single signature element whether it has to be retained
or not, and how it should be patched. The algorithm becomes
independent of the order, because each signature element can be
processed individually.
The assumptions the I/O normaliser makes on its input program are
rather intricated. In theory the VSIR validator checks should be
strong enough, but the validator isn't run by default anyway.
Whether the TPF parser validation is strong enough is not
completely clear to me, and considering that the I/O normaliser
could end up being used on different programs as well it's
probably better to revalidate locally just in case.
Commit 4717775abb removed the code turning
the corresponding DCL_INPUT instructions into NOPs, presumably because
vsir_program_remove_io_decls() now already removes them. That function
only removes the instructions though, not the declarations as such; we
now need to remove these from the "io_dcls" bitmap instead.
Otherwise ubsan reports runtime errors such as:
libs/vkd3d-shader/ir.c:4731:5: runtime error: null pointer passed as argument 1, which is declared to never be null
Otherwise when passing "-fsanitize=undefined" to the compiler, ubsan
reports such as:
libs/vkd3d-shader/ir.c:3794:5: runtime error: null pointer passed as argument 1, which is declared to never be null
