shader_signature_find_element_for_reg() is also used in the TPF parser,
where the program has not been validated yet, so it must not crash
on errors.
The I/O normaliser can instead assume that the shader is already
validated.
This fixes a crash with a shader used by The Falconeer. The bug is still
present, because the shader will be incorrectly rejected, but at least
the vkd3d-shader will fail gracefully.
We need to distinguish between the data type of a resource and the data
type of its components. These are usually the same except for 4.0
profiles where an array (or multi-dimensional array) of resources is
still considered a single resource, so it is possible for it to hold
more than one component.
In the latter case, we often need to access the type of a single
component (all components have the same type) instead of the type of the
whole array which often doesn't contain the required information, such
as sampler dimension.
This patch replaces the extern_resource.data_type field with the
extern_resource.component_type field, which points to the type of a
single component in the resource. Using it relieves many other code
paths from considering the possibility of the resource being an array.
This fixes runtime errors reported by UBSan, such as this:
vkd3d/libs/vkd3d-shader/tpf.c:6075:87: runtime error: load of value 7, which is not a valid value for type '_Bool'
when trying to compile shaders that contain UAV arrays on 4.0 profiles.
Before this commit, tpf.c accesses the
hlsl_type->e.resource.rasteriser_ordered
field, but on 4.0 and 4.1 profiles these code paths can also be reached
by UAV arrays which are HLSL_CLASS_ARRAY and this field is not supposed
to be accessed.
By coincidence, the value of hlsl_type->e.array.elements_count was being
read because these fields have the same offset in the hlsl_type.e union.
This causes a crash in the native compiler, but can only happen in
ps_5_0 were it is possible to declare structs that are both used in the
shader and contain strings.
struct
{
float a;
string b;
} apple = {1, "foobar"};
float4 main() : sv_target
{
return apple.a;
}
In our case, hlsl_type_get_component_offset() triggered an assertion
failure because it does not expect the string type. So this is replaced
by an hlsl_error().
