We need to distinguish between the data type of a resource and the data
type of its components. These are usually the same except for 4.0
profiles where an array (or multi-dimensional array) of resources is
still considered a single resource, so it is possible for it to hold
more than one component.
In the latter case, we often need to access the type of a single
component (all components have the same type) instead of the type of the
whole array which often doesn't contain the required information, such
as sampler dimension.
This patch replaces the extern_resource.data_type field with the
extern_resource.component_type field, which points to the type of a
single component in the resource. Using it relieves many other code
paths from considering the possibility of the resource being an array.
This fixes runtime errors reported by UBSan, such as this:
vkd3d/libs/vkd3d-shader/tpf.c:6075:87: runtime error: load of value 7, which is not a valid value for type '_Bool'
when trying to compile shaders that contain UAV arrays on 4.0 profiles.
Before this commit, tpf.c accesses the
hlsl_type->e.resource.rasteriser_ordered
field, but on 4.0 and 4.1 profiles these code paths can also be reached
by UAV arrays which are HLSL_CLASS_ARRAY and this field is not supposed
to be accessed.
By coincidence, the value of hlsl_type->e.array.elements_count was being
read because these fields have the same offset in the hlsl_type.e union.
This causes a crash in the native compiler, but can only happen in
ps_5_0 were it is possible to declare structs that are both used in the
shader and contain strings.
struct
{
float a;
string b;
} apple = {1, "foobar"};
float4 main() : sv_target
{
return apple.a;
}
In our case, hlsl_type_get_component_offset() triggered an assertion
failure because it does not expect the string type. So this is replaced
by an hlsl_error().
As the newly added documentation describes, this reroll serves two purposes:
* to allow shader parameters to be used for any target type (which allows using
parameters for things like Direct3D 8-9 alpha test),
* to allow the union in struct vkd3d_shader_parameter to contain types larger
than 32 bits (by specifying them indirectly through a pointer).
As the diffstat shows, HLSL_CLASS_OBJECT does not really have much in common.
Resource types (TEXTURE, SAMPLER, UAV) sometimes behave similarly to each other,
but do not generally behave similarly to effect-specific types (string, shader,
state, view). Most consumers of HLSL_CLASS_OBJECT subsequently check the base
type anyway.
Hence we want to replace HLSL_TYPE_* with individual classes for object types.
As a first step, change the last few places that only check HLSL_CLASS_OBJECT.
