Improving Hardware decode handling, especially on failure to get a decoded frame from the GPU, to safely fallback to software decode in those cases, so we don't end up with a black blank frame. This should be compatible with all hardware decoding, but I tested this only on VAAPI and AMD Radeon GPU on Ubuntu 24.04. Also updating the HW-ACCEL.md file with updated notes, and tried to add more detail about how decode and decode fallback works.

doc/HW-ACCEL.md

@@ -6,55 +6,187 @@ SPDX-License-Identifier: LGPL-3.0-or-later
 
 ## Hardware Acceleration
 
-OpenShot now has experimental support for hardware acceleration, which uses 1 (or more)
-graphics cards to offload some of the work for both decoding and encoding. This is
-very new and experimental (as of May 2019), but we look forward to "accelerating"
-our support for this in the future!
+Hardware acceleration in libopenshot allows FFmpeg to use platform-specific GPU
+APIs for video decode and encode when available. In practice, this means some of
+the work that would otherwise be done by the CPU can be offloaded to the GPU or
+to dedicated media blocks on the GPU.
 
-The following table summarizes our current level of support:
+This document focuses on what hardware acceleration in libopenshot does today,
+how it fits into the current processing pipeline, and what users and developers
+should expect from it.
 
-|                    |  Linux Decode   | Linux Encode   | Mac Decode |    Mac Encode  | Windows Decode | Windows Encode | Notes            |
-|--------------------|:---------------:|:--------------:|:----------:|:--------------:|:--------------:|:--------------:|------------------|
-| VA-API             |   ✔️        |    ✔️      |      -     |        -       |       -        |        -       | *Linux Only*     |
-| VDPAU              | ✔️ <sup>1</sup>  | ✅ <sup>2</sup> |      -    |        -       |       -        |        -       | *Linux Only*     |
-| CUDA (NVDEC/NVENC) | ❌ <sup>3</sup> |   ✔️ &nbsp;     |      -     |        -       |       -        |    ✔️ &nbsp;    | *Cross Platform* |
-| VideoToolBox       |           -     |         -      |  ✔️ &nbsp;  | ❌ <sup>4</sup> |      -        |       -         | *Mac Only*       |
-| DXVA2              |           -     |         -      |      -     |        -       | ❌ <sup>3</sup> |      -         | *Windows Only*   |
-| D3D11VA            |           -     |         -      |      -     |        -       | ❌ <sup>3</sup> |      -         | *Windows Only*   |
-| QSV                | ❌ <sup>3</sup> |   ❌ &nbsp;   |  ❌ &nbsp; |   ❌ &nbsp;    |   ❌ &nbsp;     |    ❌ &nbsp;   | *Cross Platform* |
+## Backend Overview
+
+The following table summarizes the historically supported hardware-acceleration
+backends in libopenshot. Actual behavior still depends on FFmpeg build options,
+driver availability, operating system support, and the runtime environment.
+
+|                    | Linux Decode | Linux Encode | macOS Decode | macOS Encode | Windows Decode | Windows Encode | Notes |
+|--------------------|:------------:|:------------:|:------------:|:------------:|:--------------:|:--------------:|-------|
+| VA-API             |      ✔️      |      ✔️      |      -       |      -       |       -        |       -        | Linux only |
+| VDPAU              |   ✔️ <sup>1</sup> | ✅ <sup>2</sup> |      -       |      -       |       -        |       -        | Linux only |
+| CUDA (NVDEC/NVENC) |   ❌ <sup>3</sup> |      ✔️      |      -       |      -       |       -        |      ✔️        | Backend availability depends on the FFmpeg build |
+| VideoToolbox       |      -       |      -       |      ✔️       | ❌ <sup>4</sup> |       -        |       -        | macOS only |
+| DXVA2              |      -       |      -       |      -       |      -       | ❌ <sup>3</sup> |       -        | Windows only |
+| D3D11VA            |      -       |      -       |      -       |      -       | ❌ <sup>3</sup> |       -        | Windows only |
+| QSV                |   ❌ <sup>3</sup> |      ❌      |      ❌       |      ❌       |       ❌        |       ❌        | Backend availability depends on the FFmpeg build |
 
 #### Notes
 
-1.  VDPAU for some reason needs a card number one higher than it really is
-2.  VDPAU is a decoder only
-3.  Green frames (pixel data not correctly tranferred back to system memory)
-4.  Crashes and burns
+1. VDPAU historically needed a card number one higher than expected.
+2. VDPAU is decode-only.
+3. Historically associated with failed transfers, corrupt frames, or unusable output on some setups.
+4. Historically unstable.
+
+This table should be read as a support map, not a guarantee that every backend
+is fully validated on every current OS/driver combination.
+
+## Why Hardware Acceleration Exists
+
+Hardware acceleration is useful for two main reasons:
+
+* It can reduce CPU load during decode and encode.
+* It can improve throughput for some media, especially on systems with strong
+  hardware video support.
+
+However, hardware acceleration is not automatically faster for every file or on
+every system. The real result depends on codec support, driver quality, stream
+format, pixel format, resolution, frame rate, and how much CPU-side work still
+needs to happen after decode.
+
+## What libopenshot Uses Hardware Acceleration For
+
+Today, hardware acceleration in libopenshot is primarily used for:
+
+* video decode
+* video encode
+
+It is not currently used to keep the entire edit/render pipeline on the GPU.
+Decoded frames usually still need to be copied back into system memory for
+colorspace conversion, scaling, caching, effect processing, compositing, and
+timeline rendering.
+
+That detail is important because it explains why hardware decode does not always
+produce a speedup.
+
+## Decode Flow in libopenshot
+
+The current decode flow looks roughly like this:
+
+1. A hardware decoder may be requested through `Settings::HARDWARE_DECODER`.
+2. FFmpeg opens the requested hardware decode path if the backend and driver
+   support it.
+3. The decoder produces a frame, either:
+   * directly as a software-readable frame, or
+   * as a hardware frame that must be transferred to system memory.
+4. libopenshot converts that frame into the CPU-side image representation used
+   by the rest of the pipeline.
+
+If hardware decode fails during startup decode or frame transfer, libopenshot
+falls back to software decode for that reader instead of returning corrupt,
+green, or black frames.
+
+## Fallback Behavior
+
+Hardware decode is best-effort, not all-or-nothing.
+
+If a hardware decoder is requested and one of the following happens early in the
+decode path:
+
+* repeated startup decode failures
+* failed hardware-frame transfer
+* invalid transferred frame data
+* failed software conversion of a transferred frame
+
+libopenshot reopens that reader in software decode mode and continues decoding.
+
+This behavior is intentionally conservative. The priority is correctness and
+stability:
+
+* valid frames are better than corrupt frames
+* software fallback is better than black or green output
+* a file that cannot be decoded by one hardware backend should still decode if
+  CPU decoding can handle it
+
+For diagnostics and UI checks, this means there is a difference between:
+
+* decode succeeded
+* hardware decode actually produced frames
+* hardware decode failed and software fallback was used
+
+`FFmpegReader::HardwareDecodeSuccessful()` exists to expose that distinction.
+
+## Performance Expectations
+
+Hardware decode is not guaranteed to be faster than software decode.
+
+In libopenshot's current pipeline, decoded frames are brought back to
+system memory immediately after decode. That introduces costs that can erase or
+outweigh the raw decode benefit:
+
+* hardware device setup overhead
+* frame transfer overhead between GPU and CPU memory
+* colorspace conversion and scaling after decode
+* caching and image wrapping in CPU memory
+* container/seek behavior and stream structure
+
+Because of that, hardware decode performance is workload-dependent.
+
+General guidance:
+
+* some files benefit from hardware decode
+* some files are effectively neutral
+* some files are slower with hardware decode
+* files with similar codec and resolution can still behave differently
+
+Hardware acceleration should be treated as a capability that may help, not as a
+guarantee of better performance.
+
+## Why Some Files Fail on Hardware Decode
+
+A file can fail on hardware decode for several reasons:
+
+* unsupported codec profile
+* unsupported chroma format or pixel format
+* unsupported bit depth or color range
+* driver/backend limitations
+* FFmpeg/backend integration quirks on a specific platform
+
+For example, consumer hardware decode paths often handle H.264 4:2:0 very well,
+but may not support H.264 4:2:2 decode reliably. In those cases, software decode
+may work perfectly while hardware decode fails.
 
 ## Supported FFmpeg Versions
 
-* HW accel is supported from FFmpeg version 3.4
-* HW accel was removed for nVidia drivers in Ubuntu for FFmpeg 4+
+* Hardware acceleration support requires FFmpeg versions new enough to expose the
+  relevant hardware APIs to libopenshot.
+* In practice, decode support in libopenshot relies on FFmpeg's modern send/receive
+  decode API and hardware-frame APIs.
+* Actual backend availability depends on how FFmpeg was compiled on the target system.
 
-**Notice:** The FFmpeg versions of Ubuntu and PPAs for Ubuntu show the
-same behaviour. FFmpeg 3 has working nVidia hardware acceleration while
-FFmpeg 4+ has no support for nVidia hardware acceleration
-included.
+Older historical note:
+
+* Some Ubuntu/FFmpeg/NVIDIA combinations behaved differently between FFmpeg 3.x
+  and FFmpeg 4.x, especially for NVIDIA decode support.
+
+Because backend support has changed over time, always validate against the
+actual FFmpeg build and driver stack in use.
 
 ## OpenShot Settings
 
-The following settings are use by libopenshot to enable, disable, and control
-the various hardware acceleration features.
+The following settings are used by libopenshot to enable, disable, and control
+hardware acceleration features.
 
-```{cpp}
+```cpp
 /// Use video codec for faster video decoding (if supported)
 int HARDWARE_DECODER = 0;
 
 /* 0 - No acceleration
    1 - Linux VA-API
-   2 - nVidia NVDEC
+   2 - NVIDIA NVDEC
    3 - Windows D3D9
    4 - Windows D3D11
-   5 - MacOS / VideoToolBox
+   5 - macOS / VideoToolbox
    6 - Linux VDPAU
    7 - Intel QSV */
 
@@ -70,73 +202,91 @@ int DE_LIMIT_HEIGHT_MAX = 1100;
 /// Maximum columns that hardware decode can handle
 int DE_LIMIT_WIDTH_MAX = 1950;
 
-/// Which GPU to use to decode (0 is the first, LINUX ONLY)
+/// Which GPU to use to decode (0 is the first, Linux only)
 int HW_DE_DEVICE_SET = 0;
 
-/// Which GPU to use to encode (0 is the first, LINUX ONLY)
+/// Which GPU to use to encode (0 is the first, Linux only)
 int HW_EN_DEVICE_SET = 0;
 ```
 
-## Libva / VA-API (Video Acceleration API)
+## Platform Notes
 
-The correct version of libva is needed (libva in Ubuntu 16.04 or libva2
-in Ubuntu 18.04) for the AppImage to work with hardware acceleration.
-An AppImage that works on both systems (supporting libva and libva2),
-might be possible when no libva is included in the AppImage.
+### Linux / VA-API
 
-*  vaapi is working for intel and AMD
-*  vaapi is working for decode only for nouveau
-*  nVidia driver is working for export only
+VA-API is one of the primary Linux hardware-decode paths used by libopenshot.
+On supported Intel and AMD systems it can work well, but not every file format,
+codec profile, or pixel format is supported by every driver.
 
-## AMD Graphics Cards (RadeonOpenCompute/ROCm)
+### Linux / VDPAU
 
-Decoding and encoding on the (AMD) GPU is possible with the default drivers.
-On systems where ROCm is installed and run a future use for GPU acceleration
-of effects could be implemented (contributions welcome).
+VDPAU support exists historically, but behavior can vary with driver and FFmpeg
+stack. Treat it as backend-dependent rather than universally reliable.
+
+### NVIDIA
+
+NVIDIA hardware encode support has historically been more reliable than decode
+support in libopenshot, depending on FFmpeg build and driver stack. Validate the
+actual runtime environment before assuming support.
+
+### macOS / VideoToolbox
+
+VideoToolbox support exists, but stability and feature coverage should be tested
+carefully on the target FFmpeg/macOS version.
+
+### Windows / DXVA2 / D3D11VA
+
+Windows decode backends are highly dependent on FFmpeg build options and device
+support. They should be treated as runtime-validated features, not assumptions.
 
 ## Multiple Graphics Cards
 
-If the computer has multiple graphics cards installed, you can choose which
-should be used by libopenshot. Also, you can optionally use one card for
-decoding and the other for encoding (if both cards support acceleration).
-This is currently only supported on Linux, due to the device name FFmpeg
-expects (i.e. **/dev/dri/render128**). Contributions welcome if anyone can
-determine what string format to pass for Windows and Mac.
+If the computer has multiple graphics cards installed, libopenshot can choose
+which device should be used for decode and encode. This is currently practical
+mainly on Linux, where FFmpeg expects device names such as `/dev/dri/renderD128`.
 
-## Help Us Improve Hardware Support
+Contributions are welcome for improving cross-platform device enumeration and
+selection.
 
-This information might be wrong, and we would love to continue improving
-our support for hardware acceleration in OpenShot. Please help us update
-this document if you find an error or discover new and/or useful information.
+## Testing and Validation
 
-**FFmpeg 4 + nVidia** The manual at:
-https://www.tal.org/tutorials/ffmpeg_nvidia_encode
-works pretty well. We could compile and install a version of FFmpeg 4.1.3
-on Mint 19.1 that supports the GPU on nVidia cards. A version of openshot
-with hardware support using these libraries could use the nVidia GPU.
+When validating hardware decode, check both:
 
-**BUG:** Hardware supported decoding still has some bugs (as you can see from
-the chart above). Also, the speed gains with decoding are not as great
-as with encoding. Currently, if hardware decoding fails, there is no
-fallback (you either get green frames or an "invalid file" error in OpenShot).
-This needs to be improved to successfully fall-back to software decoding.
+* correctness of the decoded output
+* whether hardware decode actually succeeded
 
-**Needed:**
-  * A way to get options and limits of the GPU, such as
- supported dimensions (width and height).
-  *  A way to list the actual Graphic Cards available to FFmpeg (for the
-  user to choose which card for decoding and encoding, as opposed
-  to "Graphics Card X")
+A frame that looks correct is not enough to prove that hardware acceleration
+worked, because software fallback may have rescued the decode.
 
-**Further improvement:** Right now the frame can be decoded on the GPU, but the
-frame is then copied to CPU memory for modifications. It is then copied back to
-GPU memory for encoding. Using the GPU for both decoding and modifications
-will make it possible to do away with these two copies. A possible solution would
-be to use Vulkan compute which would be available on Linux and Windows natively
-and on MacOS via MoltenVK.
+Recommended validation:
+
+* compare output against a software-decode baseline
+* track whether hardware decode actually produced frames
+* test both a known-good hardware sample and a known-failing fallback sample
+
+## Future Improvements
+
+The biggest architectural limitation today is that decoded frames are generally
+copied back to CPU memory for the rest of the pipeline.
+
+Longer-term improvements could include:
+
+* better hardware capability probing
+* better device enumeration for users
+* broader backend validation across platforms
+* keeping more of the pipeline on GPU memory
+* GPU-native effects/compositing paths
+
+Avoiding repeated GPU-to-CPU and CPU-to-GPU copies would make hardware
+acceleration much more effective for end-to-end editing and export workflows.
+
+## Help Improve This Document
+
+Hardware acceleration support changes with FFmpeg, drivers, operating systems,
+and GPU generations. If you find incorrect information or validate a backend on
+a newer stack, please update this document.
 
 ## Credit
 
-A big thanks to Peter M (https://github.com/eisneinechse) for all his work
-on integrating hardware acceleration into libopenshot! The community thanks
-you for this major contribution!
+A big thanks to Peter M (https://github.com/eisneinechse) for all his work on
+integrating hardware acceleration into libopenshot. The community thanks you for
+this major contribution.

src/FFmpegReader.cpp

@@ -268,7 +268,10 @@ void FFmpegReader::Open() {
 		// Initialize format context
 		pFormatCtx = NULL;
 		{
-			hw_de_on = (openshot::Settings::Instance()->HARDWARE_DECODER == 0 ? 0 : 1);
+			hw_de_on = (!force_sw_decode && openshot::Settings::Instance()->HARDWARE_DECODER != 0 ? 1 : 0);
+			hw_decode_failed = false;
+			hw_decode_error_count = 0;
+			hw_decode_succeeded = false;
 			ZmqLogger::Instance()->AppendDebugMethod("Decode hardware acceleration settings", "hw_de_on", hw_de_on, "HARDWARE_DECODER", openshot::Settings::Instance()->HARDWARE_DECODER);
 		}
 
@@ -1318,6 +1321,9 @@ std::shared_ptr<Frame> FFmpegReader::ReadStream(int64_t requested_frame) {
 			(info.has_video && !packet && !packet_status.video_eof)) {
 			// Process Video Packet
 			ProcessVideoPacket(requested_frame);
+			if (ReopenWithoutHardwareDecode(requested_frame)) {
+				continue;
+			}
 		}
 		// Audio packet
 		if ((info.has_audio && packet && packet->stream_index == audioStream) ||
@@ -1492,6 +1498,26 @@ int FFmpegReader::GetNextPacket() {
 // Get an AVFrame (if any)
 bool FFmpegReader::GetAVFrame() {
 	int frameFinished = 0;
+	auto note_hw_decode_failure = [&](int err, const char* stage) {
+#if USE_HW_ACCEL
+		if (!hw_de_on || !hw_de_supported || force_sw_decode) {
+			return;
+		}
+		if (err == AVERROR_INVALIDDATA && packet_status.video_decoded == 0) {
+			hw_decode_error_count++;
+			ZmqLogger::Instance()->AppendDebugMethod(
+				std::string("FFmpegReader::GetAVFrame (hardware decode failure candidate during ") + stage + ")",
+				"error_count", hw_decode_error_count,
+				"error", err);
+			if (hw_decode_error_count >= 3) {
+				hw_decode_failed = true;
+			}
+		}
+#else
+		(void) err;
+		(void) stage;
+#endif
+	};
 
 	// Decode video frame
 	AVFrame *next_frame = AV_ALLOCATE_FRAME();
@@ -1516,6 +1542,7 @@ bool FFmpegReader::GetAVFrame() {
 	#endif // USE_HW_ACCEL
 		if (send_packet_err < 0 && send_packet_err != AVERROR_EOF) {
 			ZmqLogger::Instance()->AppendDebugMethod("FFmpegReader::GetAVFrame (send packet: Not sent [" + av_err2string(send_packet_err) + "])", "send_packet_err", send_packet_err, "send_packet_pts", send_packet_pts);
+			note_hw_decode_failure(send_packet_err, "send_packet");
 			if (send_packet_err == AVERROR(EAGAIN)) {
 				hold_packet = true;
 				ZmqLogger::Instance()->AppendDebugMethod("FFmpegReader::GetAVFrame (send packet: AVERROR(EAGAIN): user must read output with avcodec_receive_frame()", "send_packet_pts", send_packet_pts);
@@ -1532,6 +1559,7 @@ bool FFmpegReader::GetAVFrame() {
 		// Even if the above avcodec_send_packet failed to send,
 		// we might still need to receive a packet.
 		int receive_frame_err = 0;
+		AVFrame *decoded_frame = next_frame;
 		AVFrame *next_frame2;
 #if USE_HW_ACCEL
 		if (hw_de_on && hw_de_supported) {
@@ -1548,6 +1576,7 @@ bool FFmpegReader::GetAVFrame() {
 
 			if (receive_frame_err != 0) {
 				ZmqLogger::Instance()->AppendDebugMethod("FFmpegReader::GetAVFrame (receive frame: frame not ready yet from decoder [\" + av_err2string(receive_frame_err) + \"])", "receive_frame_err", receive_frame_err, "send_packet_pts", send_packet_pts);
+				note_hw_decode_failure(receive_frame_err, "receive_frame");
 
 				if (receive_frame_err == AVERROR_EOF) {
 					ZmqLogger::Instance()->AppendDebugMethod(
@@ -1578,47 +1607,93 @@ bool FFmpegReader::GetAVFrame() {
 			if (hw_de_on && hw_de_supported) {
 				int err;
 				if (next_frame2->format == hw_de_av_pix_fmt) {
-					next_frame->format = AV_PIX_FMT_YUV420P;
-					if ((err = av_hwframe_transfer_data(next_frame,next_frame2,0)) < 0) {
-						ZmqLogger::Instance()->AppendDebugMethod("FFmpegReader::GetAVFrame (Failed to transfer data to output frame)", "hw_de_on", hw_de_on);
+					if ((err = av_hwframe_transfer_data(next_frame, next_frame2, 0)) < 0) {
+						ZmqLogger::Instance()->AppendDebugMethod(
+							"FFmpegReader::GetAVFrame (Failed to transfer data to output frame)",
+							"hw_de_on", hw_de_on,
+							"error", err);
+						note_hw_decode_failure(AVERROR_INVALIDDATA, "hwframe_transfer");
+						break;
 					}
-					if ((err = av_frame_copy_props(next_frame,next_frame2)) < 0) {
-						ZmqLogger::Instance()->AppendDebugMethod("FFmpegReader::GetAVFrame (Failed to copy props to output frame)", "hw_de_on", hw_de_on);
+					if ((err = av_frame_copy_props(next_frame, next_frame2)) < 0) {
+						ZmqLogger::Instance()->AppendDebugMethod(
+							"FFmpegReader::GetAVFrame (Failed to copy props to output frame)",
+							"hw_de_on", hw_de_on,
+							"error", err);
+						note_hw_decode_failure(AVERROR_INVALIDDATA, "hwframe_copy_props");
+						break;
 					}
+					if (next_frame->format == AV_PIX_FMT_NONE) {
+						next_frame->format = pCodecCtx->sw_pix_fmt;
+					}
+					if (next_frame->width <= 0) {
+						next_frame->width = next_frame2->width;
+					}
+					if (next_frame->height <= 0) {
+						next_frame->height = next_frame2->height;
+					}
+					decoded_frame = next_frame;
+				} else {
+					// Some hardware decoders can still return software-readable frames.
+					decoded_frame = next_frame2;
 				}
 			}
 			else
 #endif // USE_HW_ACCEL
 			{	// No hardware acceleration used -> no copy from GPU memory needed
-				next_frame = next_frame2;
+				decoded_frame = next_frame2;
+			}
+
+			if (!decoded_frame->data[0]) {
+				ZmqLogger::Instance()->AppendDebugMethod(
+					"FFmpegReader::GetAVFrame (Decoded frame missing image data)",
+					"format", decoded_frame->format,
+					"width", decoded_frame->width,
+					"height", decoded_frame->height);
+				note_hw_decode_failure(AVERROR_INVALIDDATA, "decoded_frame_empty");
+				break;
 			}
 
 			// TODO also handle possible further frames
 			// Use only the first frame like avcodec_decode_video2
 			frameFinished = 1;
+			hw_decode_error_count = 0;
+#if USE_HW_ACCEL
+			if (hw_de_on && hw_de_supported && !force_sw_decode) {
+				hw_decode_succeeded = true;
+			}
+#endif
 			packet_status.video_decoded++;
 
 			// Allocate image (align 32 for simd)
-			AVPixelFormat decoded_pix_fmt = (AVPixelFormat)(next_frame->format);
+			AVPixelFormat decoded_pix_fmt = (AVPixelFormat)(decoded_frame->format);
 			if (decoded_pix_fmt == AV_PIX_FMT_NONE)
 				decoded_pix_fmt = (AVPixelFormat)(pStream->codecpar->format);
 			if (AV_ALLOCATE_IMAGE(pFrame, decoded_pix_fmt, info.width, info.height) <= 0) {
 				throw OutOfMemory("Failed to allocate image buffer", path);
 			}
-			av_image_copy(pFrame->data, pFrame->linesize, (const uint8_t**)next_frame->data, next_frame->linesize,
+			av_image_copy(pFrame->data, pFrame->linesize, (const uint8_t**)decoded_frame->data, decoded_frame->linesize,
 										decoded_pix_fmt, info.width, info.height);
+			pFrame->format = decoded_pix_fmt;
+			pFrame->width = info.width;
+			pFrame->height = info.height;
+			pFrame->color_range = decoded_frame->color_range;
+			pFrame->colorspace = decoded_frame->colorspace;
+			pFrame->color_primaries = decoded_frame->color_primaries;
+			pFrame->color_trc = decoded_frame->color_trc;
+			pFrame->chroma_location = decoded_frame->chroma_location;
 
 			// Get display PTS from video frame, often different than packet->pts.
 			// Sending packets to the decoder (i.e. packet->pts) is async,
 			// and retrieving packets from the decoder (frame->pts) is async. In most decoders
 			// sending and retrieving are separated by multiple calls to this method.
-			if (next_frame->pts != AV_NOPTS_VALUE) {
+			if (decoded_frame->pts != AV_NOPTS_VALUE) {
 				// This is the current decoded frame (and should be the pts used) for
 				// processing this data
-				video_pts = next_frame->pts;
-			} else if (next_frame->pkt_dts != AV_NOPTS_VALUE) {
+				video_pts = decoded_frame->pts;
+			} else if (decoded_frame->pkt_dts != AV_NOPTS_VALUE) {
 				// Some videos only set this timestamp (fallback)
-				video_pts = next_frame->pkt_dts;
+				video_pts = decoded_frame->pkt_dts;
 			}
 
 			ZmqLogger::Instance()->AppendDebugMethod(
@@ -1628,7 +1703,7 @@ bool FFmpegReader::GetAVFrame() {
 			break;
 		}
 #if USE_HW_ACCEL
-		if (hw_de_on && hw_de_supported) {
+		if (hw_de_on && hw_de_supported && next_frame2 != next_frame) {
 			AV_FREE_FRAME(&next_frame2);
 		}
 	#endif // USE_HW_ACCEL
@@ -1655,6 +1730,41 @@ bool FFmpegReader::GetAVFrame() {
 	return frameFinished;
 }
 
+bool FFmpegReader::ReopenWithoutHardwareDecode(int64_t requested_frame) {
+#if USE_HW_ACCEL
+	if (!hw_decode_failed || force_sw_decode) {
+		return false;
+	}
+
+	ZmqLogger::Instance()->AppendDebugMethod(
+		"FFmpegReader::ReopenWithoutHardwareDecode (falling back to software decode)",
+		"requested_frame", requested_frame,
+		"video_packets_read", packet_status.video_read,
+		"video_packets_decoded", packet_status.video_decoded,
+		"hw_decode_error_count", hw_decode_error_count);
+
+	force_sw_decode = true;
+	hw_decode_failed = false;
+	hw_decode_error_count = 0;
+
+	Close();
+	Open();
+	Seek(requested_frame);
+	return true;
+#else
+	(void) requested_frame;
+	return false;
+#endif
+}
+
+bool FFmpegReader::HardwareDecodeSuccessful() const {
+#if USE_HW_ACCEL
+	return hw_decode_succeeded;
+#else
+	return false;
+#endif
+}
+
 // Check the current seek position and determine if we need to seek again
 bool FFmpegReader::CheckSeek() {
 	// Are we seeking for a specific frame?
@@ -1879,9 +1989,49 @@ void FFmpegReader::ProcessVideoPacket(int64_t requested_frame) {
 	sws_setColorspaceDetails(img_convert_ctx, src_coeff, src_full_range ? 1 : 0,
 							 dst_coeff, dst_full_range, 0, 1 << 16, 1 << 16);
 
+	if (!pFrame || !pFrame->data[0] || pFrame->linesize[0] <= 0) {
+#if USE_HW_ACCEL
+		if (hw_de_on && hw_de_supported && !force_sw_decode) {
+			hw_decode_failed = true;
+			ZmqLogger::Instance()->AppendDebugMethod(
+				"FFmpegReader::ProcessVideoPacket (Invalid source frame; forcing software fallback)",
+				"requested_frame", requested_frame,
+				"current_frame", current_frame,
+				"src_pix_fmt", src_pix_fmt,
+				"src_width", src_width,
+				"src_height", src_height);
+		}
+#endif
+		if (pFrame) {
+			RemoveAVFrame(pFrame);
+			pFrame = NULL;
+		}
+		return;
+	}
+
 	// Resize / Convert to RGB
-	sws_scale(img_convert_ctx, pFrame->data, pFrame->linesize, 0,
+	const int scaled_lines = sws_scale(img_convert_ctx, pFrame->data, pFrame->linesize, 0,
 			  original_height, pFrameRGB->data, pFrameRGB->linesize);
+	if (scaled_lines <= 0) {
+#if USE_HW_ACCEL
+		if (hw_de_on && hw_de_supported && !force_sw_decode) {
+			hw_decode_failed = true;
+			ZmqLogger::Instance()->AppendDebugMethod(
+				"FFmpegReader::ProcessVideoPacket (sws_scale failed; forcing software fallback)",
+				"requested_frame", requested_frame,
+				"current_frame", current_frame,
+				"scaled_lines", scaled_lines,
+				"src_pix_fmt", src_pix_fmt,
+				"src_width", src_width,
+				"src_height", src_height);
+		}
+#endif
+		free(buffer);
+		AV_RESET_FRAME(pFrameRGB);
+		RemoveAVFrame(pFrame);
+		pFrame = NULL;
+		return;
+	}
 
 	// Create or get the existing frame object
 	std::shared_ptr<Frame> f = CreateFrame(current_frame);

src/FFmpegReader.h

@@ -165,6 +165,10 @@ namespace openshot {
 		AVFrame *pFrameRGB_cached = nullptr;          ///< Temporary frame used for video conversion
 
 		int hw_de_supported = 0;	// Is set by FFmpegReader
+		bool force_sw_decode = false;
+		bool hw_decode_failed = false;
+		int hw_decode_error_count = 0;
+		bool hw_decode_succeeded = false;
 #if USE_HW_ACCEL
 		AVPixelFormat hw_de_av_pix_fmt = AV_PIX_FMT_NONE;
 		AVHWDeviceType hw_de_av_device_type = AV_HWDEVICE_TYPE_NONE;
@@ -198,6 +202,9 @@ namespace openshot {
 		/// Get an AVFrame (if any)
 		bool GetAVFrame();
 
+		/// Reopen the current reader with software decode after hardware decode fails
+		bool ReopenWithoutHardwareDecode(int64_t requested_frame);
+
 		/// Get the next packet (if any)
 		int GetNextPacket();
 
@@ -285,6 +292,9 @@ namespace openshot {
 		/// Determine if reader is open or closed
 		bool IsOpen() override { return is_open; };
 
+		/// Return true if hardware decode was requested and successfully produced at least one frame
+		bool HardwareDecodeSuccessful() const override;
+
 		/// Return the type name of the class
 		std::string Name() override { return "FFmpegReader"; };
 

src/ReaderBase.h

@@ -113,6 +113,10 @@ namespace openshot
 		/// Determine if reader is open or closed
 		virtual bool IsOpen() = 0;
 
+		/// Return true if hardware decode successfully produced at least one frame.
+		/// Readers without hardware decode support should return false.
+		virtual bool HardwareDecodeSuccessful() const { return false; }
+
 		/// Return the type name of the class
 		virtual std::string Name() = 0;
 

tests/FFmpegReader.cpp

@@ -14,6 +14,7 @@
 #include <memory>
 #include <set>
 #include <algorithm>
+#include <array>
 #include <cstdio>
 #include <cstdlib>
 #include <ctime>
@@ -29,6 +30,62 @@
 
 using namespace openshot;
 
+namespace {
+
+double SampleAverageLuma(const std::shared_ptr<Frame>& frame, int sample_grid = 4) {
+	const int width = frame->GetWidth();
+	const int height = frame->GetHeight();
+	if (width <= 0 || height <= 0) {
+		return 0.0;
+	}
+
+	int64_t luma_sum = 0;
+	int64_t sample_count = 0;
+	for (int y = 0; y < sample_grid; ++y) {
+		const int row = std::min(height - 1, (y * height) / sample_grid);
+		const unsigned char* pixels = frame->GetPixels(row);
+		for (int x = 0; x < sample_grid; ++x) {
+			const int col = std::min(width - 1, (x * width) / sample_grid);
+			const int pixel_index = col * 4;
+			luma_sum += (pixels[pixel_index] + pixels[pixel_index + 1] + pixels[pixel_index + 2]) / 3;
+			++sample_count;
+		}
+	}
+
+	return sample_count > 0
+		? static_cast<double>(luma_sum) / static_cast<double>(sample_count)
+		: 0.0;
+}
+
+struct HardwareDecoderSettingsGuard {
+	int decoder = 0;
+	int device = 0;
+
+	HardwareDecoderSettingsGuard()
+		: decoder(Settings::Instance()->HARDWARE_DECODER),
+		  device(Settings::Instance()->HW_DE_DEVICE_SET) {}
+
+	~HardwareDecoderSettingsGuard() {
+		Settings::Instance()->HARDWARE_DECODER = decoder;
+		Settings::Instance()->HW_DE_DEVICE_SET = device;
+	}
+};
+
+struct TemporaryFileGuard {
+	std::string path;
+
+	explicit TemporaryFileGuard(std::string temp_path)
+		: path(std::move(temp_path)) {}
+
+	~TemporaryFileGuard() {
+		if (!path.empty()) {
+			std::remove(path.c_str());
+		}
+	}
+};
+
+}
+
 TEST_CASE( "Invalid_Path", "[libopenshot][ffmpegreader]" )
 {
 	// Check invalid path and error details
@@ -471,6 +528,134 @@ TEST_CASE( "Attached_Picture_Audio_Does_Not_Stall_Early_Frames", "[libopenshot][
 	std::remove(fixture_path.str().c_str());
 }
 
+TEST_CASE( "HardwareDecodeSuccessful_IsFalse_WhenHardwareDecodeIsDisabled", "[libopenshot][ffmpegreader][hardware]" )
+{
+	HardwareDecoderSettingsGuard guard;
+	Settings::Instance()->HARDWARE_DECODER = 0;
+	Settings::Instance()->HW_DE_DEVICE_SET = 0;
+
+	std::stringstream path;
+	path << TEST_MEDIA_PATH << "sintel_trailer-720p.mp4";
+	FFmpegReader r(path.str(), DurationStrategy::VideoPreferred);
+	r.Open();
+
+	REQUIRE(r.info.has_video);
+	auto frame = r.GetFrame(1);
+	REQUIRE(frame->has_image_data);
+	CHECK_FALSE(r.HardwareDecodeSuccessful());
+
+	r.Close();
+}
+
+TEST_CASE( "VAAPI_H264_420_Reports_HardwareDecodeSuccess", "[libopenshot][ffmpegreader][hardware]" )
+{
+#if !defined(__linux__) || !USE_HW_ACCEL
+	WARN("Skipping hardware decode success test: requires Linux build with hardware decode support");
+	return;
+#else
+	if (std::system("ffmpeg -hide_banner -version >/dev/null 2>&1") != 0) {
+		WARN("Skipping hardware decode success test: ffmpeg executable not available");
+		return;
+	}
+	if (std::system("ffmpeg -hide_banner -hwaccels 2>/dev/null | grep -q '\\<vaapi\\>'") != 0) {
+		WARN("Skipping hardware decode success test: ffmpeg does not report VAAPI support");
+		return;
+	}
+	if (std::system("sh -c 'test -e /dev/dri/renderD128 -o -e /dev/dri/renderD129 -o -e /dev/dri/renderD130' >/dev/null 2>&1") != 0) {
+		WARN("Skipping hardware decode success test: no render node available under /dev/dri");
+		return;
+	}
+
+	std::srand(static_cast<unsigned int>(std::time(nullptr)));
+	std::stringstream fixture_path;
+	fixture_path << "libopenshot-vaapi-420-test-" << std::rand() << ".mp4";
+	TemporaryFileGuard fixture_cleanup(fixture_path.str());
+
+	std::stringstream command;
+	command << "ffmpeg -y -hide_banner -loglevel error "
+	        << "-f lavfi -i \"testsrc2=size=128x72:rate=30\" "
+	        << "-t 1 "
+	        << "-c:v libx264 "
+	        << "-pix_fmt yuv420p "
+	        << "-profile:v high "
+	        << "\"" << fixture_path.str() << "\"";
+	const int command_result = std::system(command.str().c_str());
+	REQUIRE(command_result == 0);
+
+	HardwareDecoderSettingsGuard hw_guard;
+	Settings::Instance()->HARDWARE_DECODER = 1;
+	Settings::Instance()->HW_DE_DEVICE_SET = 0;
+
+	FFmpegReader r(fixture_path.str(), DurationStrategy::VideoPreferred);
+	r.Open();
+
+	REQUIRE(r.info.has_video);
+	auto frame = r.GetFrame(1);
+	REQUIRE(frame->has_image_data);
+	CHECK(r.HardwareDecodeSuccessful());
+
+	r.Close();
+#endif
+}
+
+TEST_CASE( "VAAPI_H264_422_Does_Not_Return_Black_Frames", "[libopenshot][ffmpegreader][hardware]" )
+{
+#if !defined(__linux__) || !USE_HW_ACCEL
+	WARN("Skipping VAAPI regression test: requires Linux build with hardware decode support");
+	return;
+#else
+	if (std::system("ffmpeg -hide_banner -version >/dev/null 2>&1") != 0) {
+		WARN("Skipping VAAPI regression test: ffmpeg executable not available");
+		return;
+	}
+	if (std::system("ffmpeg -hide_banner -hwaccels 2>/dev/null | grep -q '\\<vaapi\\>'") != 0) {
+		WARN("Skipping VAAPI regression test: ffmpeg does not report VAAPI support");
+		return;
+	}
+	if (std::system("sh -c 'test -e /dev/dri/renderD128 -o -e /dev/dri/renderD129 -o -e /dev/dri/renderD130' >/dev/null 2>&1") != 0) {
+		WARN("Skipping VAAPI regression test: no render node available under /dev/dri");
+		return;
+	}
+
+	std::srand(static_cast<unsigned int>(std::time(nullptr)));
+	std::stringstream fixture_path;
+	fixture_path << "libopenshot-vaapi-422-test-" << std::rand() << ".mp4";
+	TemporaryFileGuard fixture_cleanup(fixture_path.str());
+
+	std::stringstream command;
+	command << "ffmpeg -y -hide_banner -loglevel error "
+	        << "-f lavfi -i \"testsrc2=size=128x72:rate=30\" "
+	        << "-t 1 "
+	        << "-c:v libx264 "
+	        << "-pix_fmt yuvj422p "
+	        << "-profile:v high422 "
+	        << "-color_range pc "
+	        << "\"" << fixture_path.str() << "\"";
+	const int command_result = std::system(command.str().c_str());
+	REQUIRE(command_result == 0);
+
+	HardwareDecoderSettingsGuard guard;
+	Settings::Instance()->HARDWARE_DECODER = 1;
+	Settings::Instance()->HW_DE_DEVICE_SET = 0;
+
+	FFmpegReader r(fixture_path.str(), DurationStrategy::VideoPreferred);
+	r.Open();
+
+	REQUIRE(r.info.has_video);
+	REQUIRE(r.info.video_length >= 3);
+
+	const std::array<int64_t, 3> frames_to_check = {1, r.info.video_length / 2, r.info.video_length};
+	for (const int64_t frame_number : frames_to_check) {
+		auto frame = r.GetFrame(frame_number);
+		REQUIRE(frame->has_image_data);
+		INFO("frame=" << frame_number << ", avg_luma=" << SampleAverageLuma(frame));
+		CHECK(SampleAverageLuma(frame) > 8.0);
+	}
+
+	r.Close();
+#endif
+}
+
 TEST_CASE( "verify parent Timeline", "[libopenshot][ffmpegreader]" )
 {
 	// Create a reader