Commit Graph

2187 Commits

Author SHA1 Message Date
Jonathan Thomas da35707966 Merge branch 'object-detection-v2' into object-mask-effect 2026-05-20 21:42:07 -05:00
Jonathan Thomas 5dc3724bba Fixing Codacy nitpicks
- Reordered the bounds check before indexing predictedBoxes in sort.cpp
- Marked SortTracker constructor explicit
- Removed TrackingBox::classScores from the public result struct and moved it into a private DetectionBox helper in sort.cpp
- Replaced the raw prototype scan loop with std::find_if
2026-05-20 21:41:45 -05:00
Jonathan Thomas 9f228c923b Improve Object Mask prompts and conservative GPU selection
- Supports multiple positive rect prompts instead of only the first rect.
- Preserves negative rects as ranking filters instead of collapsing them to center points.
- Ranks EfficientSAM candidates with soft penalties for negative point hits and negative rect overlap.
- Unions separate EfficientSAM passes for multiple positive rects so multiple target objects can remain selected.
- Makes GPU_AUTO conservative:
    - CUDA if available
    - otherwise CPU
    - OpenCL only when explicitly requested with GPU_OPENCL
- Adds simple requested/selected DNN device logging for Object Detection, EfficientSAM, and Cutie.
2026-05-20 18:45:57 -05:00
Jonathan Thomas 395f66253c Improve Object Mask model compatibility
- Removing XMem as a backend for Object Mask effect (replacing with Cutie model)
- Detect Cutie model dimensions from selected model filenames instead of relying on a fixed propagation size.
- Support multiple Cutie quality tiers with matching encode/decode/readout dimensions.
- Add fallback mask threshold behavior for EfficientSAM when the default threshold produces an empty mask.
- Reduce Object Mask preview/export failures with larger Cutie model variants.
2026-05-20 14:22:28 -05:00
Jonathan Thomas 8bd98a592b Replace ObjectMask EdgeSAM seed masks with EfficientSAM ONNX
- Use a single EfficientSAM ONNX model for ObjectMask seed mask generation
- Add EfficientSAM prompt preprocessing and mask candidate selection
- Keep seed-frame output as the raw EfficientSAM mask while still seeding XMem
- Add ClipProcessingJobs::PreviewObjectMask for single-frame interactive previews
- Accept EfficientSAM model JSON keys while preserving legacy encoder_model aliases
- Remove ObjectMask protobuf shutdown call that could destabilize Python teardown
- Add ObjectMask ONNX validation coverage
2026-05-18 19:18:56 -05:00
Jonathan Thomas 1f0b1b9698 Add Object Mask effect with EdgeSAM and XMem propagation
- Add ObjectMask effect for rendering and exposing generated object masks
- Add EdgeSAM preprocessing path for prompt-based seed and reseed masks
- Add XMem ONNX propagation between seed frames for single-object tracking
- Support positive/negative points, positive rect prompts, and prompt keyframes
- Store generated masks in protobuf data for playback and downstream effects
- Add mask color/alpha plus stroke color/alpha/width effect controls
- Register ObjectMask with effect discovery and clip preprocessing jobs
- Add focused ObjectMask effect tests
2026-05-18 16:28:13 -05:00
Jonathan Thomas ec370404d9 Add YOLO segmentation mask support to Object Detection
- Support YOLO segmentation ONNX outputs, persist sparse RLE masks in object-detection protobuf data, and render configurable mask overlays in the Object Detection effect.
- Also normalize tracked object classes at save time, smooth short mask gaps, and expose Draw Mask / Mask Color / Mask Alpha only when mask data is available.
2026-05-15 17:51:16 -05:00
Jonathan Thomas 9fbe600244 Updated compatibility for CVObjectDetection to support:
- YOLOv5: [1, num_boxes, 5 + classes]
- YOLO26 no-end2end export: [1, attributes, candidates], e.g. [1,116,8400]
-Also kept the YOLOv5-seg safety fix: when a classes file exists, mask coefficients are not treated as extra classes.
2026-05-15 09:15:09 -05:00
Jonathan Thomas f7586af1be Fallback to CPU backend for OpenCV when CUDA backend not available. 2026-05-13 23:20:31 -05:00
Jonathan Thomas 47d1784b1c Validate YOLO ONNX models in libopenshot
- Add libopenshot-side ONNX model validation for object detection
- Return user-displayable error text instead of throwing through callers
- Share ONNX load/error handling between validation and object detection processing
- Fail early for YOLOv5 on OpenCV older than 4.3
- Expose validation through ClipProcessingJobs for Python bindings
2026-05-13 14:30:40 -05:00
Jonathan Thomas 4f6cb7c50f Updating gitlab builder for Linux to use new /opt installed OpenCV 4.5.5 folder, and return error message on < 4.3.0 OpenCV versions for Object Detection effect. 2026-05-13 00:00:29 -05:00
Jonathan Thomas 14c63f3714 Fix margin-bounded effects to avoid invalid image access
- Clamp and validate Blur, Bars, and Pixelate margin-derived regions
- Add regression coverage for collapsed/out-of-range margin cases
- Verify focused Bars, Blur, and Pixelate tests pass
2026-05-12 15:15:15 -05:00
Jonathan Thomas 4f289bc442 Fix saturation stride and parent effect updates
- Respect QImage::bytesPerLine() in saturation and mask blending
- Preserve child effect IDs when inheriting parent effect changes
- Propagate parent updates correctly through nested child effects
- Add regression tests for padded image stride and parent effect inheritance
2026-05-12 14:22:06 -05:00
Jonathan Thomas eae93e1ebf Protecting crash when reading ONNX model with wrong format (Object Detection effect) 2026-05-10 22:26:18 -05:00
Jonathan Thomas be99ec02c6 Migrate object detection to YOLOv5 ONNX
- Replace YOLOv3 Darknet loading with YOLOv5 ONNX model loading and validation.
- Parse YOLOv5 outputs with top class candidates for smoother labels.
- Improve SORT tracking with class-score smoothing and stricter geometry gates.
- Prevent active tracks from hopping to adjacent objects or tiny nested detections.
- Keep object ids stable through protobuf load/save and add an object-detection debug example.
- Add class-based default colors and matching border/background defaults.
- Add All Objects pseudo-selection support for tracked-object properties.
- Honor tracked-object corner radius when using tracker/object detector masks.
- Add regression tests for tracking stability, all-object styling, masks, and keyframes.
2026-05-10 21:48:24 -05:00
Jonathan Thomas d274163314 Improve tracked object rendering controls
- Remove unused tracked-object rotation property from JSON, properties, and box values
- Add per-object Draw Text support alongside Draw Box
- Use deterministic class colors for Object Detector instead of random seeded colors
- Scale tracked-object stroke widths based on preview raster size
- Keep Object Detector text rendering gated by both global and per-object text settings
2026-05-10 14:34:38 -05:00
Jonathan Thomas 0751153d5a Merge pull request #1074 from OpenShot/tracked-mask-source
Use TrackedObjectMask as "Mask: Source" for any Effect
2026-05-09 20:40:18 -05:00
Jonathan Thomas 07e60a8a60 Improve Caption margins and default visibility
- Add Bottom margin support to Caption JSON, properties, rendering, and tests
- Rename Caption margin property labels to "Margin: Top/Left/Bottom/Right"
- Clip Caption rendering to the configured margin rectangle
- Default Caption fade in/out to 0.0 so newly inserted captions render immediately
- Avoid divide-by-zero fade calculations when Caption fade durations are disabled
2026-05-09 16:27:10 -05:00
Jonathan Thomas ec71b7335c Added top/bottom/left/right margins to Blur effect, similar to Pixelate one (for consistency). Renamed the user-facing JsonProperty labels across all effects with similar keyframes to:
- Margin: Left
- Margin: Top
- Margin: Right
- Margin: Bottom
2026-05-09 14:50:36 -05:00
Jonathan Thomas a6e7d85c93 Adding the ability to use TrackedObjectMask as "Mask: Source" for any effect (i.e. blur/pixilate only the tracked area)
- Add tracked effect IDs as mask sources
- Generate bbox masks from Tracker/Object Detection
- Preserve original frame when no bbox is visible
- Add focused mask-source tests
2026-05-09 11:27:49 -05:00
Jonathan Thomas 22ec091a06 Merge pull request #1073 from OpenShot/source-rotation
Apply reader-side orientation metadata (i.e. rotate pixels before Clip)
2026-05-09 10:55:48 -05:00
Jonathan Thomas 0081cda5ee Apply reader-side orientation metadata and stabilize tracker stroke rendering
- Move phone/video rotation metadata handling into readers with an internal clip compatibility mode for legacy projects. FFmpegReader now reports oriented dimensions, applies orientation to decoded frames, and preserves old JSON behavior through reader_orientation_mode.

- Also adjust tracker/object-detection stroke width for preview raster scaling so tracked boxes remain visible during clip scale animations.
2026-05-08 20:58:53 -05:00
Jonathan Thomas 6011e73716 Merge pull request #1071 from OpenShot/small-svg-thumbnails
Fix SVG thumbnail rasterization with max decode size (fix fuzzy thumbnails for small SVG files)
2026-05-08 17:34:06 -05:00
Jonathan Thomas 226f686074 Fix SVG thumbnail rasterization with max decode size (SVG upscaling for tiny sizes)
-Treat SetMaxDecodeSize as the rasterization target for parentless SVG readers, instead of only as a downscale ceiling. This lets thumbnail generation render tiny SVG files at the requested decode size, avoiding blurry thumbnails caused by first rasterizing at the SVG document size.
-Raster images and SVGs with a parent clip/timeline keep their existing sizing behavior.
2026-05-08 14:22:33 -05:00
Jonathan Thomas 6ccb526cb9 Fixing GetMaxFrame() to fix tiny float overshoot, causing occasional black frames at the end of an export. For example, it might have concluded that we had 505.00001 frames, and this would then be converted later to 506 with ceil(). 2026-05-08 13:52:28 -05:00