mirror of
https://github.com/uutils/sed.git
synced 2026-06-10 16:14:15 -07:00
fix: rewrite GNU testsuite harness to run upstream test scripts directly
The previous harness tried to extract sed commands from GNU test scripts via regex pattern matching, which produced false negatives (comparing against empty expected output) and false positives. This led to inflated test counts and unreliable pass/fail signals. The new approach: - Provides a lightweight shim for the gnulib test framework (init.sh) with implementations of compare_, returns_, skip_, framework_failure_, and all require_* functions - Executes each .sh test script from the GNU testsuite directly, injecting our Rust sed binary via PATH - Uses a clean srcdir with symlinks to real test data files - Adds per-test timeout (10s) to catch infinite loops, with SIGTERM isolation so timeout signals don't kill the parent script - Properly propagates exit codes (0=pass, 77=skip, 99=framework failure) Results are now consistent with CI tracking (~12% pass rate) with clear PASS/FAIL/SKIP/timeout categorization. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
committed by
Sylvestre Ledru
co-authored by
Claude Opus 4.6
parent
81d98b9049
commit
16f11456b3
@@ -75,8 +75,8 @@ jobs:
|
||||
run: |
|
||||
## Run GNU sed testsuite using our script
|
||||
cd 'sed'
|
||||
# Set GNU testsuite directory
|
||||
export GNU_TESTSUITE_DIR="../gnu.sed/testsuite"
|
||||
# Set GNU sed source directory (testsuite is at gnu.sed/testsuite/)
|
||||
export GNU_SED_DIR="../gnu.sed"
|
||||
# Run tests with JSON output
|
||||
./util/run-gnu-testsuite.sh --json-output "${{ env.TEST_FULL_SUMMARY_FILE }}" || true
|
||||
|
||||
|
||||
@@ -47,7 +47,8 @@ The binary is named `sed` in `target/release/sed`.
|
||||
|
||||
### GNU sed Compatibility Testing
|
||||
|
||||
Test compatibility against GNU sed using the comprehensive testsuite (47+ tests, ~10% pass rate):
|
||||
Test compatibility against GNU sed by running the upstream testsuite shell scripts
|
||||
with a lightweight gnulib test-framework shim:
|
||||
|
||||
```bash
|
||||
# Clone GNU sed testsuite (one time setup)
|
||||
@@ -56,11 +57,16 @@ git clone https://github.com/mirror/sed.git ../gnu.sed
|
||||
# Run compatibility tests
|
||||
./util/run-gnu-testsuite.sh
|
||||
|
||||
# Verbose mode shows failure details
|
||||
./util/run-gnu-testsuite.sh -v
|
||||
|
||||
# Generate JSON results for CI
|
||||
./util/run-gnu-testsuite.sh --json-output results.json
|
||||
```
|
||||
|
||||
The testsuite extracts test cases from the GNU sed repository and tests them against expected outputs.
|
||||
The harness executes each `.sh` test from the GNU sed testsuite directly, injecting
|
||||
our Rust sed binary via `PATH` and providing shim implementations of the gnulib test
|
||||
framework functions (`compare_`, `returns_`, `skip_`, etc.).
|
||||
|
||||
### Unit Tests
|
||||
|
||||
|
||||
+274
-315
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user