26 Commits
Author SHA1 Message Date
oech3andGitHub e8b93e94cb uufuzz: simplify (#59) 2026-06-10 12:43:17 +02:00
oech3andGitHub 0326b43cb2 uufuzz: simplify pipe() code (#58) 2026-06-09 23:21:59 +02:00
WondrandGitHub e925ff4ac8 grep: honor GNU buffer anchors (#56) 2026-06-05 17:33:23 +02:00
Leonard Hecker f2f86ef572 grep: Fix slow_path_binary_handling test 2026-06-05 17:04:50 +02:00
Leonard Hecker 265a562c26 grep: Simplify session_finalize 2026-06-05 17:04:29 +02:00
WondrandGitHub bc61aece73 grep: cover zero-width EOF only-matching behavior (#44) 2026-06-05 16:52:30 +02:00
WondrandGitHub 55a7b0180b grep: suppress EOF output in quiet mode (#41) 2026-06-05 16:48:36 +02:00
WondrandGitHub d556159952 grep: let dot match newline in null-data mode (#45) 2026-06-05 16:25:04 +02:00
WondrandGitHub 7c79cb4e3a grep: keep invalid UTF-8 text under -I (#49) 2026-06-05 16:06:49 +02:00
Sylvestre Ledru abcdd7e84d docs: add playground section to README with URL example 2026-06-05 13:34:16 +02:00
Sylvestre LedruandGitHub d28bf769a1 Merge pull request #7 from uutils/add-differential-fuzzer
fuzz: add differential fuzzer against GNU grep
2026-06-05 08:29:30 +02:00
Sylvestre LedruandGitHub b4980df814 Merge pull request #16 from uutils/literal-fast-path
perf: buffer-at-a-time search for literal patterns
2026-06-05 08:28:04 +02:00
Sylvestre LedruandGitHub e9825e3503 Merge pull request #12 from uutils/grep-initial-tab-empty-line
grep: don't emit the -T alignment tab on empty lines
2026-06-05 08:27:43 +02:00
Sylvestre LedruandGitHub f5d5f6c063 Merge pull request #40 from koopatroopa787/issue-34-perl-single-pattern
fix: reject multiple patterns when -P/--perl-regexp is used
2026-06-05 08:25:18 +02:00
Sylvestre LedruandGitHub 337b7c704b Merge pull request #52 from wondr-wclabs/codex/empty-match-word-line
grep: select zero-width matches under -w and -x
2026-06-05 08:23:03 +02:00
Wondr ad7595ffe6 grep: select zero-width matches under -w and -x 2026-06-05 04:39:22 +01:00
Kanishk Sachan e3d80f59e2 fix: reject multiple patterns when -P/--perl-regexp is used
GNU grep's PCRE backend supports only a single pattern. Supplying
multiple patterns via repeated -e flags, or a pattern string that
contains a literal newline, must exit 2 with the message:

    the -P option only supports a single pattern

Add the validation immediately after patterns are collected, before
regex-mode selection. Add a test covering:
  - two separate -e flags with -P
  - a newline-embedded pattern string with -P
  - single -e with -P still works normally

Closes #34
2026-06-05 01:25:22 +01:00
Sylvestre Ledru 56d774f576 test: cover slow-path modes that literal tests no longer reach
The buffer-at-a-time fast path now serves the literal patterns that the
existing -l/-L/-q and binary tests used, leaving the line-at-a-time
engine's equivalents uncovered. Add bracket-class (non-literal) tests
for -l/-L/-q and binary handling (notice, -a text, without-match bail,
and the finalize-time notice), plus a fast-path test for a NUL that is
only discovered after a line was already printed.

No dead code was found: the remaining uncovered lines are writer I/O
error-propagation arms and pre-existing filesystem error handlers.
2026-06-04 22:24:27 +02:00
Sylvestre Ledru 28186e9ec3 perf: buffer-at-a-time search for literal patterns
Literal searches were ~50-70x slower than GNU grep because every line
paid per-line costs (terminator scan, NUL scan, dispatch) even when a
buffer held no match. Add a buffer-at-a-time driver that scans whole
chunks with a substring searcher and only locates line boundaries
around the matches it finds; a chunk with no match costs a single
vectorized sweep and no per-line work.

The driver activates only for plain ASCII literal patterns (case
sensitive, no metacharacters) in the simpler output modes: -c, -l, -L,
-q, and plain line printing with -n/-b/filename/-m. Anything needing
match positions, context, inversion, color, or special binary handling
falls back to the unchanged line-at-a-time path. Output stays
byte-identical to that path, including binary/invalid-UTF-8 behavior.

- line_buffer: read_chunk() yields the largest span of complete lines.
- matcher: expose per-pattern memmem searchers when every pattern is a
  plain literal (plain_literal()).
- searcher: eligible_for_fast_path(), fast_locate(), fast_print().

All scanning rides on the memchr crate (SIMD memchr/memrchr/memmem).
Unit tests for read_chunk and plain_literal; integration tests for
prefixes, -m, and multi-chunk line-number correctness.

Benchmarks (31 MB corpus) vs prior release:
  -F (no match):  232ms -> 15ms  (15.9x; now faster than GNU)
  -c literal:     229ms -> 15ms  (15.2x)
  plain print:    248ms -> 18ms  (13.5x)
Regex and -i paths are unchanged (still the line-at-a-time engine).
2026-06-04 22:24:27 +02:00
rifatxandGitHub f4798cb6d0 fix -l and -L to be mutually exclusive so that last one wins (#30) 2026-06-04 22:14:12 +02:00
Sylvestre LedruandGitHub 4e6823a8b3 Add installation section to README (#29) 2026-06-03 17:56:23 +02:00
Sylvestre LedruandGitHub b0700b1d78 Merge pull request #21 from oech3/pub
GnuTests: publish binary from main
2026-06-02 21:04:50 +02:00
oech3 bc416c6d8b GnuTests: publish binary from main 2026-06-03 00:05:24 +09:00
Sylvestre Ledru 9598faf708 fuzz: add differential fuzzer against GNU grep
Add a cargo-fuzz harness (fuzz_grep) that runs uu_grep and GNU grep on
the same generated args/input and panics on any divergence, using the
vendored uufuzz crate (adapted from uutils/coreutils to depend on
crates.io uucore rather than a path).

A CI workflow (.github/workflows/fuzzing.yml) builds the uufuzz
examples, builds the fuzzer, and runs it for 60s. fuzz_grep is marked
should_pass: false / continue-on-error since it currently surfaces real
GNU-compatibility gaps (e.g. uu_grep rejects a repeated -m, GNU accepts).
2026-06-02 10:13:13 +02:00
Sylvestre LedruandGitHub c614a57a05 Merge pull request #17 from uutils/e2e-bench-only
E2e bench only
2026-05-31 11:17:38 +02:00
Sylvestre Ledru a7b15320af grep: don't emit the -T alignment tab on empty lines
With -T, grep pads the prefix with a tab so line content lands on a tab
stop. GNU omits that tab when the line has no content: an empty line
prints just its prefix (a whitespace-only line still gets the tab).
uu_grep always wrote the tab, so empty matched lines gained a spurious
trailing tab. Gate the tab on non-empty content. Fixes the GNU testsuite
'initial-tab' test.
2026-05-30 19:13:52 +02:00
22 changed files with 2648 additions and 24 deletions
+18 -1
View File
@@ -9,6 +9,9 @@ on:
branches:
- '*'
permissions:
contents: write # Publish grep instead of discarding
# End the current execution if there is a new changeset in the PR.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -47,7 +50,21 @@ jobs:
shell: bash
run: |
cd 'grep'
cargo build --release
cargo build --release --config=profile.release.strip=true
tar -C target/release -cf - grep | zstd -19 -o ../grep-x86_64-unknown-linux-gnu.tar.zst
- name: Publish latest commit
uses: softprops/action-gh-release@v3
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
with:
tag_name: latest-commit
body: |
commit: ${{ github.sha }}
draft: false
prerelease: true
files: |
grep-x86_64-unknown-linux-gnu.tar.zst
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Run GNU grep testsuite
shell: bash
+162
View File
@@ -0,0 +1,162 @@
name: Fuzzing
# spell-checker:ignore (people) taiki-e
# spell-checker:ignore (misc) fuzzer uufuzz
env:
CARGO_INCREMENTAL: "0"
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
permissions:
contents: read # to fetch code (actions/checkout)
# End the current execution if there is a new changeset in the PR.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
jobs:
uufuzz-examples:
name: Build and test uufuzz examples
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Build uufuzz library
run: |
cd fuzz/uufuzz
cargo build --release
- name: Run uufuzz tests
run: |
cd fuzz/uufuzz
cargo test --lib
- name: Build and run uufuzz examples
run: |
cd fuzz/uufuzz
echo "Building all examples..."
cargo build --examples --release
# Run all examples except integration_testing (which has FD issues in CI)
for example in examples/*.rs; do
example_name=$(basename "$example" .rs)
if [ "$example_name" != "integration_testing" ]; then
cargo run --example "$example_name" --release
fi
done
fuzz-build:
name: Build the fuzzers
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Install `cargo-fuzz`
uses: taiki-e/install-action@v2
with:
tool: cargo-fuzz
- name: Emulate a nightly toolchain
run: |
echo "RUSTC_BOOTSTRAP=1" >> "${GITHUB_ENV}"
- name: Run `cargo-fuzz build`
# Force the correct target
# https://github.com/rust-fuzz/cargo-fuzz/issues/398
run: cargo fuzz build --target $(rustc --print host-tuple)
fuzz-run:
needs: fuzz-build
name: Fuzz
runs-on: ubuntu-latest
timeout-minutes: 5
env:
RUN_FOR: 60
strategy:
fail-fast: false
matrix:
test-target:
# fuzz_grep is a differential fuzzer against GNU grep; it currently
# surfaces compatibility differences, so it is not expected to pass yet.
- { name: fuzz_grep, should_pass: false }
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Install `cargo-fuzz`
uses: taiki-e/install-action@v2
with:
tool: cargo-fuzz
- name: Emulate a nightly toolchain
run: |
echo "RUSTC_BOOTSTRAP=1" >> "${GITHUB_ENV}"
- name: Run ${{ matrix.test-target.name }} for ${{ env.RUN_FOR }} seconds
id: run_fuzzer
shell: bash
continue-on-error: ${{ !matrix.test-target.should_pass }}
run: |
mkdir -p fuzz/stats
STATS_FILE="fuzz/stats/${{ matrix.test-target.name }}.txt"
# Force the correct target
# https://github.com/rust-fuzz/cargo-fuzz/issues/398
cargo fuzz run --target $(rustc --print host-tuple) ${{ matrix.test-target.name }} -- -max_total_time=${{ env.RUN_FOR }} -timeout=${{ env.RUN_FOR }} -detect_leaks=0 -print_final_stats=1 2>&1 | tee "$STATS_FILE"
# Save should_pass value for later inspection
echo "${{ matrix.test-target.should_pass }}" > "fuzz/stats/${{ matrix.test-target.name }}.should_pass"
# Print stats to job output for immediate visibility
echo "----------------------------------------"
echo "FUZZING STATISTICS FOR ${{ matrix.test-target.name }}"
echo "----------------------------------------"
echo "Runs: $(grep -q "stat::number_of_executed_units" "$STATS_FILE" && grep "stat::number_of_executed_units" "$STATS_FILE" | awk '{print $2}' || echo "unknown")"
echo "Execution Rate: $(grep -q "stat::average_exec_per_sec" "$STATS_FILE" && grep "stat::average_exec_per_sec" "$STATS_FILE" | awk '{print $2}' || echo "unknown") execs/sec"
echo "New Units: $(grep -q "stat::new_units_added" "$STATS_FILE" && grep "stat::new_units_added" "$STATS_FILE" | awk '{print $2}' || echo "unknown")"
echo "Expected: ${{ matrix.test-target.should_pass }}"
if grep -q "SUMMARY: " "$STATS_FILE"; then
echo "Status: $(grep "SUMMARY: " "$STATS_FILE" | head -1)"
else
echo "Status: Completed"
fi
echo "----------------------------------------"
# Add summary to GitHub step summary
echo "### Fuzzing Results for ${{ matrix.test-target.name }}" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "| Metric | Value |" >> "$GITHUB_STEP_SUMMARY"
echo "|--------|-------|" >> "$GITHUB_STEP_SUMMARY"
if grep -q "stat::number_of_executed_units" "$STATS_FILE"; then
echo "| Runs | $(grep "stat::number_of_executed_units" "$STATS_FILE" | awk '{print $2}') |" >> "$GITHUB_STEP_SUMMARY"
fi
if grep -q "stat::average_exec_per_sec" "$STATS_FILE"; then
echo "| Execution Rate | $(grep "stat::average_exec_per_sec" "$STATS_FILE" | awk '{print $2}') execs/sec |" >> "$GITHUB_STEP_SUMMARY"
fi
if grep -q "stat::new_units_added" "$STATS_FILE"; then
echo "| New Units | $(grep "stat::new_units_added" "$STATS_FILE" | awk '{print $2}') |" >> "$GITHUB_STEP_SUMMARY"
fi
echo "| Should pass | ${{ matrix.test-target.should_pass }} |" >> "$GITHUB_STEP_SUMMARY"
if grep -q "SUMMARY: " "$STATS_FILE"; then
echo "| Status | $(grep "SUMMARY: " "$STATS_FILE" | head -1) |" >> "$GITHUB_STEP_SUMMARY"
else
echo "| Status | Completed |" >> "$GITHUB_STEP_SUMMARY"
fi
echo "" >> "$GITHUB_STEP_SUMMARY"
- name: Upload Stats
if: always()
uses: actions/upload-artifact@v4
with:
name: fuzz-stats-${{ matrix.test-target.name }}
path: |
fuzz/stats/${{ matrix.test-target.name }}.txt
fuzz/stats/${{ matrix.test-target.name }}.should_pass
retention-days: 5
+17
View File
@@ -11,6 +11,23 @@
A Rust implementation of [GNU Grep](https://www.gnu.org/software/grep/).
This project is an initial release and may contain bugs.
## Install
```shell
cargo install uu_grep
```
## 🚀 Try it online
You can try `grep` directly in your browser on the [uutils playground](https://uutils.github.io/playground/).
Arguments (and a full command) can be passed through the URL via the `cmd` query parameter, for example:
```shell
printf '🚀 rocket\n🛰️ satellite\n🌙 moon\n⭐ star\n' | grep 🌙
```
[Run it in the playground](https://uutils.github.io/playground/?cmd=printf%20%27%F0%9F%9A%80%20rocket%5Cn%F0%9F%9B%B0%EF%B8%8F%20satellite%5Cn%F0%9F%8C%99%20moon%5Cn%E2%AD%90%20star%5Cn%27%20%7C%20grep%20%F0%9F%8C%99)
## Building
Download Rust at: https://rustup.rs/
+2
View File
@@ -0,0 +1,2 @@
[build]
rustflags = ["--cfg", "fuzzing"]
+4
View File
@@ -0,0 +1,4 @@
target
corpus
artifacts
Cargo.lock
+33
View File
@@ -0,0 +1,33 @@
[package]
name = "uu_grep-fuzz"
version = "0.0.0"
description = "uutils ~ 'grep' fuzzers"
repository = "https://github.com/microsoft/uutils-grep/tree/main/fuzz/"
edition = "2024"
rust-version = "1.88.0"
license = "MIT"
publish = false
[package.metadata]
cargo-fuzz = true
# Prevent this from interfering with the parent workspace
[workspace]
members = ["."]
# Enable debug symbols in release builds for readable backtraces
# when fuzzing discovers crashes.
[profile.release]
debug = true
[dependencies]
libfuzzer-sys = "0.4.7"
rand = { version = "0.10.1", features = ["std_rng"] }
uufuzz = { path = "uufuzz" }
uu_grep = { path = ".." }
[[bin]]
name = "fuzz_grep"
path = "fuzz_targets/fuzz_grep.rs"
test = false
doc = false
+189
View File
@@ -0,0 +1,189 @@
// This file is part of the uutils grep package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
// spell-checker:ignore uumain seedable
#![no_main]
use libfuzzer_sys::fuzz_target;
use uu_grep::uumain;
use rand::prelude::IndexedRandom;
use rand::rngs::StdRng;
use rand::{RngExt, SeedableRng};
use std::ffi::OsString;
use uufuzz::{CommandResult, compare_result, generate_and_run_uumain, run_gnu_cmd};
static CMD_PATH: &str = "grep";
/// Derive a 32-byte RNG seed from the libFuzzer input so that every run is a
/// pure function of `data`. This is what makes crash artifacts reproducible:
/// the same bytes always generate the same pattern/args/input.
fn seed_from_data(data: &[u8]) -> StdRng {
let mut seed = [0u8; 32];
for (i, b) in data.iter().enumerate() {
seed[i % 32] ^= b;
}
StdRng::from_seed(seed)
}
/// Random string mixing valid UTF-8 (incl. multi-byte) and the occasional
/// invalid byte. Driven by the caller's seeded RNG so output is deterministic.
fn gen_random_string(rng: &mut StdRng, max_length: usize) -> String {
let valid_utf8: Vec<char> =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789🔩🪛🪓⚙️🔗🧰"
.chars()
.collect();
let invalid_utf8 = [0xC3u8, 0x28];
let mut result = String::new();
for _ in 0..rng.random_range(0..=max_length) {
if rng.random_bool(0.9) {
result.push(*valid_utf8.choose(rng).unwrap());
} else if let Some(c) = char::from_u32(*invalid_utf8.choose(rng).unwrap() as u32) {
result.push(c);
}
}
result
}
/// Generate a (mostly) meaningful set of grep flags, occasionally throwing in
/// garbage to exercise error handling.
fn generate_grep_args(rng: &mut StdRng) -> Vec<OsString> {
let arg_count = rng.random_range(0..=5);
let mut args = Vec::new();
for _ in 0..arg_count {
// Small chance of an invalid argument.
if rng.random_bool(0.1) {
let len = rng.random_range(1..=10);
args.push(OsString::from(gen_random_string(rng, len)));
continue;
}
match rng.random_range(0..=15) {
0 => args.push(OsString::from("-i")), // ignore case
1 => args.push(OsString::from("-v")), // invert match
2 => args.push(OsString::from("-c")), // count
3 => args.push(OsString::from("-n")), // line number
4 => args.push(OsString::from("-o")), // only matching
5 => args.push(OsString::from("-w")), // word boundaries
6 => args.push(OsString::from("-x")), // whole line match
7 => args.push(OsString::from("-F")), // fixed strings
8 => args.push(OsString::from("-E")), // extended regexp
9 => args.push(OsString::from("-G")), // basic regexp
10 => args.push(OsString::from("--null-data")),
11 => args.push(OsString::from("--byte-offset")),
12 => {
// max-count
args.push(OsString::from("-m"));
args.push(OsString::from(rng.random_range(0..=5).to_string()));
}
13 => {
// after-context
args.push(OsString::from("-A"));
args.push(OsString::from(rng.random_range(0..=3).to_string()));
}
14 => {
// before-context
args.push(OsString::from("-B"));
args.push(OsString::from(rng.random_range(0..=3).to_string()));
}
15 => args.push(OsString::from("-s")), // suppress error messages
_ => (),
}
}
args
}
/// Build a pattern. Sometimes a literal token, sometimes a small regex made of
/// random characters and metacharacters.
fn generate_pattern(rng: &mut StdRng) -> String {
match rng.random_range(0..=3) {
0 => {
let len = rng.random_range(1..=5);
gen_random_string(rng, len)
}
1 => {
// A small alternation / anchored regex.
let la = rng.random_range(1..=3);
let a = gen_random_string(rng, la);
let lb = rng.random_range(1..=3);
let b = gen_random_string(rng, lb);
format!("{a}|{b}")
}
2 => {
let lb = rng.random_range(1..=3);
let base = gen_random_string(rng, lb);
let meta = ["*", "+", "?", ".", "^", "$", ".*", "[a-z]", "\\w"];
let m = meta[rng.random_range(0..meta.len())];
format!("{base}{m}")
}
_ => {
// Pick one of a few hand-written patterns that exercise common paths.
let canned = ["a", "^", "$", ".", ".*", "[0-9]+", "\\b", "()"];
canned[rng.random_range(0..canned.len())].to_string()
}
}
}
/// Generate input text with a mix of short and long lines.
fn generate_input(rng: &mut StdRng, count: usize) -> String {
let mut lines = Vec::new();
for _ in 0..count {
if rng.random_bool(0.1) {
let len = rng.random_range(200..=500);
lines.push(gen_random_string(rng, len));
} else {
let len = rng.random_range(0..=20);
lines.push(gen_random_string(rng, len));
}
}
lines.join("\n")
}
fuzz_target!(|data: &[u8]| {
let mut rng = seed_from_data(data);
let pattern = generate_pattern(&mut rng);
// Pass the pattern through `-e` so it is never mistaken for a flag, then
// append the (possibly invalid) extra arguments.
let mut args = vec![
OsString::from("grep"),
OsString::from("-e"),
OsString::from(&pattern),
];
args.extend(generate_grep_args(&mut rng));
let input = generate_input(&mut rng, 10);
let rust_result = generate_and_run_uumain(&args, uumain, Some(&input));
let gnu_result = match run_gnu_cmd(CMD_PATH, &args[1..], false, Some(&input)) {
Ok(result) => result,
Err(error_result) => {
eprintln!("Failed to run GNU command:");
eprintln!("Stderr: {}", error_result.stderr);
eprintln!("Exit Code: {}", error_result.exit_code);
CommandResult {
stdout: String::new(),
stderr: error_result.stderr,
exit_code: error_result.exit_code,
}
}
};
compare_result(
"grep",
&format!("{:?}", &args[1..]),
Some(&input),
&rust_result,
&gnu_result,
false, // Set to true if you want to fail on stderr diff
);
});
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "uufuzz"
description = "uutils ~ 'core' uutils fuzzing library"
repository = "https://github.com/uutils/coreutils/tree/main/fuzz/uufuzz"
version = "0.8.0"
edition = "2024"
rust-version = "1.88.0"
license = "MIT"
[dependencies]
console = "0.16.0"
rand = { version = "0.10.1", features = ["std_rng"] }
similar = "3.0.0"
uucore = { version = "0.8.0", features = ["parser"] }
tempfile = "3.15.0"
rustix = { version = "1.1.4", features = ["stdio", "pipe"] }
+137
View File
@@ -0,0 +1,137 @@
# uufuzz
A Rust library for **differential fuzzing** of command-line utilities. Originally designed for testing uutils coreutils against GNU coreutils, but can be used to compare any two implementations of command-line tools.
Differential fuzzing is a testing technique that compares the behavior of two implementations of the same functionality using randomly generated inputs. This helps identify bugs, inconsistencies, and security vulnerabilities by finding cases where implementations diverge unexpectedly.
## Features
- **Command Execution**: Run and capture output from both Rust and reference implementations
- **Result Comparison**: Detailed comparison of stdout, stderr, and exit codes with diff output
- **Input Generation**: Utilities for generating random strings, files, and test inputs
- **GNU Compatibility**: Built-in support for detecting and running GNU coreutils
- **Pretty Output**: Colorized and formatted test result display
## Usage
Add to your `Cargo.toml`:
```toml
[dependencies]
uufuzz = "0.1.0"
```
### Basic Example
```rust
use std::ffi::OsString;
use uufuzz::{generate_and_run_uumain, run_gnu_cmd, compare_result};
// Your utility's main function
fn my_echo_main(args: std::vec::IntoIter<OsString>) -> i32 {
// Implementation here
0
}
// Test against GNU implementation
let args = vec![OsString::from("echo"), OsString::from("hello")];
// Run your implementation
let rust_result = generate_and_run_uumain(&args, my_echo_main, None);
// Run GNU implementation
let gnu_result = run_gnu_cmd("echo", &args[1..], false, None).unwrap();
// Compare results
compare_result("echo", "hello", None, &rust_result, &gnu_result, true);
```
### With Pipe Input
```rust
let pipe_input = "test data";
let rust_result = generate_and_run_uumain(&args, my_cat_main, Some(pipe_input));
let gnu_result = run_gnu_cmd("cat", &args[1..], false, Some(pipe_input)).unwrap();
compare_result("cat", "", Some(pipe_input), &rust_result, &gnu_result, true);
```
### Random Input Generation
```rust
use uufuzz::{generate_random_string, generate_random_file};
// Generate random string up to 50 characters
let random_input = generate_random_string(50);
// Generate random temporary file
let file_path = generate_random_file().expect("Failed to create file");
```
## Use Cases
### Fuzzing Testing
Perfect for libFuzzer-based differential fuzzing:
```rust
#![no_main]
use libfuzzer_sys::fuzz_target;
use uufuzz::*;
fuzz_target!(|_data: &[u8]| {
let args = generate_test_args();
let rust_result = generate_and_run_uumain(&args, my_utility_main, None);
let gnu_result = run_gnu_cmd("utility", &args[1..], false, None).unwrap();
compare_result("utility", &format!("{:?}", args), None, &rust_result, &gnu_result, true);
});
```
### Integration Testing
Use in regular test suites to verify compatibility:
```rust
#[test]
fn test_basic_functionality() {
let args = vec![OsString::from("sort"), OsString::from("-n")];
let input = "3\n1\n2\n";
let rust_result = generate_and_run_uumain(&args, sort_main, Some(input));
let gnu_result = run_gnu_cmd("sort", &args[1..], false, Some(input)).unwrap();
assert_eq!(rust_result.stdout, gnu_result.stdout);
assert_eq!(rust_result.exit_code, gnu_result.exit_code);
}
```
## Environment Variables
- `LC_ALL=C` - Automatically set when running GNU commands for consistent behavior
## Platform Support
- **Linux**: Full support with GNU coreutils
- **macOS**: Works with GNU coreutils via Homebrew (`brew install coreutils`)
- **Windows**: Limited support (depends on available reference implementations)
## Examples
The library includes several working examples in the `examples/` directory:
### Running Examples
```bash
# Basic differential comparison
cargo run --example basic_echo
# Pipe input handling
cargo run --example pipe_input
# Simple integration testing (recommended approach)
cargo run --example simple_integration
# Complex integration testing (demonstrates file descriptor handling issues)
cargo run --example integration_testing
```
## License
Licensed under the MIT License, same as uutils coreutils.
+61
View File
@@ -0,0 +1,61 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
use std::ffi::OsString;
use uufuzz::{compare_result, generate_and_run_uumain, run_gnu_cmd};
// Mock echo implementation for demonstration
fn mock_echo_main(args: std::vec::IntoIter<OsString>) -> i32 {
let args: Vec<OsString> = args.collect();
// Skip the program name (first argument)
for (i, arg) in args.iter().skip(1).enumerate() {
if i > 0 {
print!(" ");
}
print!("{}", arg.to_string_lossy());
}
println!();
0
}
fn main() {
println!("=== Basic uufuzz Example ===");
// Test against GNU implementation
let args = vec![
OsString::from("echo"),
OsString::from("hello"),
OsString::from("world"),
];
println!("Running mock echo implementation...");
let rust_result = generate_and_run_uumain(&args, mock_echo_main, None);
println!("Running GNU echo...");
match run_gnu_cmd("echo", &args[1..], false, None) {
Ok(gnu_result) => {
println!("Comparing results...");
compare_result(
"echo",
"hello world",
None,
&rust_result,
&gnu_result,
false,
);
}
Err(error_result) => {
println!("Failed to run GNU echo: {}", error_result.stderr);
println!("This is expected if GNU coreutils is not installed");
// Show what our implementation produced
println!("\nOur implementation result:");
println!("Stdout: '{}'", rust_result.stdout);
println!("Stderr: '{}'", rust_result.stderr);
println!("Exit code: {}", rust_result.exit_code);
}
}
}
+163
View File
@@ -0,0 +1,163 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
use rand::RngExt;
use std::ffi::OsString;
use uufuzz::{generate_and_run_uumain, generate_random_string, run_gnu_cmd};
// Mock echo implementation with some bugs for demonstration
fn mock_buggy_echo_main(args: std::vec::IntoIter<OsString>) -> i32 {
let args: Vec<OsString> = args.collect();
let mut should_add_newline = true;
let mut enable_escapes = false;
let mut start_index = 1;
// Parse arguments (simplified)
for arg in args.iter().skip(1) {
let arg_str = arg.to_string_lossy();
if arg_str == "-n" {
should_add_newline = false;
start_index += 1;
} else if arg_str == "-e" {
enable_escapes = true;
start_index += 1;
} else {
break;
}
}
// Print arguments
for (i, arg) in args.iter().skip(start_index).enumerate() {
if i > 0 {
print!(" ");
}
let arg_str = arg.to_string_lossy();
if enable_escapes {
// Simulate a bug: incomplete escape sequence handling
let processed = arg_str.replace("\\n", "\n").replace("\\t", "\t");
print!("{}", processed);
} else {
print!("{}", arg_str);
}
}
if should_add_newline {
println!();
}
0
}
// Generate test arguments for echo command
fn generate_echo_args() -> Vec<OsString> {
let mut rng = rand::rng();
let mut args = vec![OsString::from("echo")];
// Randomly add flags
if rng.random_bool(0.3) {
// 30% chance
args.push(OsString::from("-n"));
}
if rng.random_bool(0.2) {
// 20% chance
args.push(OsString::from("-e"));
}
// Add 1-3 random string arguments
let num_args = rng.random_range(1..=3);
for _ in 0..num_args {
let arg = generate_random_string(rng.random_range(1..=15));
args.push(OsString::from(arg));
}
args
}
fn main() {
println!("=== Fuzzing Simulation uufuzz Example ===");
println!("This simulates how libFuzzer would test our echo implementation");
println!("against GNU echo with random inputs.\n");
let num_tests = 10;
let mut passed = 0;
let mut failed = 0;
for i in 1..=num_tests {
println!("--- Fuzz Test {} ---", i);
let args = generate_echo_args();
println!(
"Testing with args: {:?}",
args.iter().map(|s| s.to_string_lossy()).collect::<Vec<_>>()
);
// Run our implementation
let rust_result = generate_and_run_uumain(&args, mock_buggy_echo_main, None);
// Run GNU implementation
match run_gnu_cmd("echo", &args[1..], false, None) {
Ok(gnu_result) => {
// Check if results match
let stdout_match = rust_result.stdout.trim() == gnu_result.stdout.trim();
let exit_code_match = rust_result.exit_code == gnu_result.exit_code;
if stdout_match && exit_code_match {
println!("✓ PASS: Implementations match");
passed += 1;
} else {
println!("✗ FAIL: Implementations differ");
failed += 1;
// Show the difference in a controlled way (not panicking like compare_result)
if !stdout_match {
println!(" Stdout difference:");
println!(
" Ours: '{}'",
rust_result.stdout.trim().replace('\n', "\\n")
);
println!(
" GNU: '{}'",
gnu_result.stdout.trim().replace('\n', "\\n")
);
}
if !exit_code_match {
println!(
" Exit code difference: {} vs {}",
rust_result.exit_code, gnu_result.exit_code
);
}
}
}
Err(error_result) => {
println!("⚠ GNU echo not available: {}", error_result.stderr);
println!(" Our result: '{}'", rust_result.stdout.trim());
// Don't count this as pass or fail
continue;
}
}
println!();
}
println!("=== Fuzzing Results ===");
println!("Total tests: {}", num_tests);
println!("Passed: {}", passed);
println!("Failed: {}", failed);
if failed > 0 {
println!(
"\n⚠ Found {} discrepancies! In real fuzzing, these would be investigated.",
failed
);
println!("This demonstrates how differential fuzzing can find bugs in implementations.");
} else {
println!("\n✓ All tests passed! The implementations appear compatible.");
}
println!("\nIn a real libfuzzer setup, this would run thousands of iterations");
println!("automatically with more sophisticated input generation.");
}
+236
View File
@@ -0,0 +1,236 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
use std::ffi::OsString;
use uufuzz::{generate_and_run_uumain, run_gnu_cmd};
// Mock sort implementation for demonstration
fn mock_sort_main(args: std::vec::IntoIter<OsString>) -> i32 {
use std::io::{self, Read};
let args: Vec<OsString> = args.collect();
let mut numeric_sort = false;
let mut reverse_sort = false;
// Parse arguments
for arg in args.iter().skip(1) {
let arg_str = arg.to_string_lossy();
match arg_str.as_ref() {
"-n" | "--numeric-sort" => numeric_sort = true,
"-r" | "--reverse" => reverse_sort = true,
_ => {}
}
}
// Read from stdin
let mut input = String::new();
match io::stdin().read_to_string(&mut input) {
Ok(_) => {
let mut lines: Vec<&str> = input.lines().collect();
if numeric_sort {
// Sort numerically
lines.sort_by(|a, b| {
let a_num: f64 = a.trim().parse().unwrap_or(0.0);
let b_num: f64 = b.trim().parse().unwrap_or(0.0);
a_num.partial_cmp(&b_num).unwrap()
});
} else {
// Sort lexically
lines.sort();
}
if reverse_sort {
lines.reverse();
}
for line in lines {
println!("{}", line);
}
0
}
Err(_) => {
eprintln!("Error reading from stdin");
1
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_basic_sort_functionality() {
println!("Testing basic sort functionality...");
let args = vec![OsString::from("sort")];
let input = "zebra\napple\nbanana\n";
let rust_result = generate_and_run_uumain(&args, mock_sort_main, Some(input));
match run_gnu_cmd("sort", &args[1..], false, Some(input)) {
Ok(gnu_result) => {
// In test environment, stdout might not be captured properly
// Just verify the function runs without errors and exit codes match
assert_eq!(
rust_result.exit_code, gnu_result.exit_code,
"Exit codes should match"
);
println!("✓ Basic sort test passed (exit codes match)");
}
Err(_) => {
// GNU sort not available, just check our implementation runs
assert_eq!(
rust_result.exit_code, 0,
"Our sort should exit successfully"
);
println!("✓ Basic sort test passed (GNU sort not available)");
}
}
}
#[test]
fn test_numeric_sort() {
println!("Testing numeric sort...");
let args = vec![OsString::from("sort"), OsString::from("-n")];
let input = "10\n2\n1\n20\n";
let rust_result = generate_and_run_uumain(&args, mock_sort_main, Some(input));
match run_gnu_cmd("sort", &args[1..], false, Some(input)) {
Ok(gnu_result) => {
assert_eq!(
rust_result.exit_code, gnu_result.exit_code,
"Exit codes should match"
);
println!("✓ Numeric sort test passed (exit codes match)");
}
Err(_) => {
// GNU sort not available, just check our implementation runs
assert_eq!(
rust_result.exit_code, 0,
"Our numeric sort should exit successfully"
);
println!("✓ Numeric sort test passed (GNU sort not available)");
}
}
}
#[test]
fn test_reverse_sort() {
println!("Testing reverse sort...");
let args = vec![OsString::from("sort"), OsString::from("-r")];
let input = "apple\nbanana\nzebra\n";
let rust_result = generate_and_run_uumain(&args, mock_sort_main, Some(input));
match run_gnu_cmd("sort", &args[1..], false, Some(input)) {
Ok(gnu_result) => {
assert_eq!(
rust_result.exit_code, gnu_result.exit_code,
"Exit codes should match"
);
println!("✓ Reverse sort test passed (exit codes match)");
}
Err(_) => {
// GNU sort not available, just check our implementation runs
assert_eq!(
rust_result.exit_code, 0,
"Our reverse sort should exit successfully"
);
println!("✓ Reverse sort test passed (GNU sort not available)");
}
}
}
#[test]
fn test_empty_input() {
println!("Testing empty input...");
let args = vec![OsString::from("sort")];
let input = "";
let rust_result = generate_and_run_uumain(&args, mock_sort_main, Some(input));
match run_gnu_cmd("sort", &args[1..], false, Some(input)) {
Ok(gnu_result) => {
assert_eq!(
rust_result.exit_code, gnu_result.exit_code,
"Exit codes should match"
);
println!("✓ Empty input test passed (exit codes match)");
}
Err(_) => {
// GNU sort not available, just check our implementation runs
assert_eq!(
rust_result.exit_code, 0,
"Should exit successfully with empty input"
);
println!("✓ Empty input test passed (GNU sort not available)");
}
}
}
}
fn main() {
println!("=== Integration Testing uufuzz Example ===");
println!("This demonstrates how to use uufuzz in regular test suites");
println!("to verify compatibility with reference implementations.\n");
println!("Run 'cargo test --example integration_testing' to execute the tests.");
println!("Or run individual tests below for demonstration:\n");
// Demonstrate the tests manually
let test_cases = [
(
"Basic lexical sort",
vec![OsString::from("sort")],
"zebra\napple\nbanana\n",
),
(
"Numeric sort",
vec![OsString::from("sort"), OsString::from("-n")],
"10\n2\n1\n20\n",
),
(
"Reverse sort",
vec![OsString::from("sort"), OsString::from("-r")],
"apple\nbanana\nzebra\n",
),
("Empty input", vec![OsString::from("sort")], ""),
];
for (test_name, args, input) in test_cases {
println!("--- {} ---", test_name);
println!(
"Args: {:?}",
args.iter().map(|s| s.to_string_lossy()).collect::<Vec<_>>()
);
println!("Input: {:?}", input.replace('\n', "\\n"));
let rust_result = generate_and_run_uumain(&args, mock_sort_main, Some(input));
println!("Our output: {:?}", rust_result.stdout.replace('\n', "\\n"));
println!("Exit code: {}", rust_result.exit_code);
match run_gnu_cmd("sort", &args[1..], false, Some(input)) {
Ok(gnu_result) => {
println!("GNU output: {:?}", gnu_result.stdout.replace('\n', "\\n"));
if rust_result.stdout == gnu_result.stdout
&& rust_result.exit_code == gnu_result.exit_code
{
println!("✓ Outputs match!");
} else {
println!("✗ Outputs differ!");
}
}
Err(_) => {
println!("GNU sort not available for comparison");
}
}
println!();
}
println!("=== Example completed ===");
println!("In a real test suite, assertions would ensure compatibility.");
}
+68
View File
@@ -0,0 +1,68 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
use std::ffi::OsString;
use std::io::{self, Read};
use uufuzz::{compare_result, generate_and_run_uumain, run_gnu_cmd};
// Mock cat implementation for demonstration
fn mock_cat_main(args: std::vec::IntoIter<OsString>) -> i32 {
let _args: Vec<OsString> = args.collect();
// Read from stdin and write to stdout
let mut input = String::new();
match io::stdin().read_to_string(&mut input) {
Ok(_) => {
print!("{}", input);
0
}
Err(_) => {
eprintln!("Error reading from stdin");
1
}
}
}
fn main() {
println!("=== Pipe Input uufuzz Example ===");
let args = vec![OsString::from("cat")];
let pipe_input = "Hello from pipe!\nThis is line 2.\nAnd line 3.";
println!("Running mock cat implementation with pipe input...");
let rust_result = generate_and_run_uumain(&args, mock_cat_main, Some(pipe_input));
println!("Running GNU cat with pipe input...");
match run_gnu_cmd("cat", &args[1..], false, Some(pipe_input)) {
Ok(gnu_result) => {
println!("Comparing results...");
compare_result(
"cat",
"",
Some(pipe_input),
&rust_result,
&gnu_result,
false,
);
}
Err(error_result) => {
println!("Failed to run GNU cat: {}", error_result.stderr);
println!("This is expected if GNU coreutils is not installed");
// Show what our implementation produced
println!("\nOur implementation result:");
println!("Stdout: '{}'", rust_result.stdout);
println!("Stderr: '{}'", rust_result.stderr);
println!("Exit code: {}", rust_result.exit_code);
// Verify our mock implementation works
if rust_result.stdout.trim() == pipe_input.trim() {
println!("✓ Our mock cat implementation correctly echoed the pipe input");
} else {
println!("✗ Our mock cat implementation failed to echo the pipe input correctly");
}
}
}
}
+105
View File
@@ -0,0 +1,105 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
use std::ffi::OsString;
use uufuzz::{CommandResult, run_gnu_cmd};
fn main() {
println!("=== Simple Integration Testing uufuzz Example ===");
println!("This demonstrates how to use uufuzz to compare against GNU tools");
println!("without the complex file descriptor manipulation.\n");
// Test cases that work well with external command comparison
let test_cases = [
(
"echo test",
"echo",
vec![OsString::from("hello"), OsString::from("world")],
None,
),
(
"echo with flag",
"echo",
vec![OsString::from("-n"), OsString::from("no-newline")],
None,
),
(
"cat with input",
"cat",
vec![],
Some("Hello from cat!\nLine 2\n"),
),
("sort basic", "sort", vec![], Some("zebra\napple\nbanana\n")),
(
"sort numeric",
"sort",
vec![OsString::from("-n")],
Some("10\n2\n1\n20\n"),
),
];
for (test_name, cmd, args, input) in test_cases {
println!("--- {} ---", test_name);
// Run GNU command
match run_gnu_cmd(cmd, &args, false, input) {
Ok(gnu_result) => {
println!("✓ GNU {} succeeded", cmd);
println!(
" Stdout: {:?}",
gnu_result.stdout.trim().replace('\n', "\\n")
);
println!(" Exit code: {}", gnu_result.exit_code);
// This demonstrates how you would compare results
// In real usage, you'd run your implementation and compare:
// let my_result = run_my_implementation(&args, input);
// assert_eq!(my_result.stdout, gnu_result.stdout);
// assert_eq!(my_result.exit_code, gnu_result.exit_code);
}
Err(error_result) => {
println!(
"⚠ GNU {} failed or not available: {}",
cmd, error_result.stderr
);
println!(" This is normal if GNU coreutils isn't installed");
}
}
println!();
}
println!("=== Practical Example: Compare two echo implementations ===");
// Simple echo comparison
let args = vec![OsString::from("hello"), OsString::from("world")];
match run_gnu_cmd("echo", &args, false, None) {
Ok(gnu_result) => {
println!("GNU echo result: {:?}", gnu_result.stdout.trim());
// Simulate our own echo implementation result
let our_result = CommandResult {
stdout: "hello world\n".to_string(),
stderr: String::new(),
exit_code: 0,
};
if our_result.stdout.trim() == gnu_result.stdout.trim()
&& our_result.exit_code == gnu_result.exit_code
{
println!("✓ Our echo matches GNU echo!");
} else {
println!("✗ Our echo differs from GNU echo");
println!(" Our result: {:?}", our_result.stdout.trim());
println!(" GNU result: {:?}", gnu_result.stdout.trim());
}
}
Err(_) => {
println!("Cannot compare - GNU echo not available");
}
}
println!("\n=== Example completed ===");
println!("This approach is simpler and more reliable for integration testing.");
}
+469
View File
@@ -0,0 +1,469 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
use console::Style;
use pretty_print::{
print_diff, print_end_with_status, print_or_empty, print_section, print_with_style,
};
use rand::RngExt;
use rand::prelude::IndexedRandom;
use rustix::io::dup;
use rustix::io::read;
use rustix::stdio::{dup2_stderr, dup2_stdin, dup2_stdout};
use std::env::temp_dir;
use std::ffi::OsString;
use std::fs::File;
use std::io::{Seek, SeekFrom, Write, pipe};
use std::process::{Command, Stdio};
use std::sync::atomic::Ordering;
use std::sync::{Once, atomic::AtomicBool};
use std::{io, thread};
pub mod pretty_print;
/// Represents the result of running a command, including its standard output,
/// standard error, and exit code.
#[derive(Debug)]
pub struct CommandResult {
/// The standard output (stdout) of the command as a string.
pub stdout: String,
/// The standard error (stderr) of the command as a string.
pub stderr: String,
/// The exit code of the command.
pub exit_code: i32,
}
static CHECK_GNU: Once = Once::new();
static IS_GNU: AtomicBool = AtomicBool::new(false);
pub fn is_gnu_cmd(cmd_path: &str) -> Result<(), std::io::Error> {
CHECK_GNU.call_once(|| {
let version_output = Command::new(cmd_path).arg("--version").output().unwrap();
println!("version_output {version_output:#?}");
let version_str = String::from_utf8_lossy(&version_output.stdout).to_string();
if version_str.contains("GNU coreutils") {
IS_GNU.store(true, Ordering::Relaxed);
}
});
if IS_GNU.load(Ordering::Relaxed) {
Ok(())
} else {
panic!("Not the GNU implementation");
}
}
pub fn generate_and_run_uumain<F>(
args: &[OsString],
uumain_function: F,
pipe_input: Option<&str>,
) -> CommandResult
where
F: FnOnce(std::vec::IntoIter<OsString>) -> i32 + Send + 'static,
{
// Duplicate the stdout and stderr file descriptors to restore later
let original_stdout_fd_owned =
dup(std::io::stdout()).expect("Failed to duplicate STDOUT_FILENO");
let original_stderr_fd_owned =
dup(std::io::stderr()).expect("Failed to duplicate STDERR_FILENO");
println!("Running test {:?}", &args[0..]);
let (read_pipe_stdout, write_pipe_stdout) = pipe().expect("Failed to create pipes");
let (read_pipe_stderr, write_pipe_stderr) = pipe().expect("Failed to create pipes");
// Redirect stdout and stderr to their respective pipes
dup2_stdout(&write_pipe_stdout).expect("Failed to redirect STDOUT_FILENO");
dup2_stderr(&write_pipe_stderr).expect("Failed to redirect STDERR_FILENO");
// Handle stdin redirection if needed
let original_stdin_fd_owned = if let Some(input_str) = pipe_input {
// we have pipe input
let mut input_file = tempfile::tempfile().unwrap();
write!(input_file, "{input_str}").unwrap();
input_file.seek(SeekFrom::Start(0)).unwrap();
// Redirect stdin to read from the in-memory file
let stdin_fd = dup(std::io::stdin()).expect("Failed to duplicate STDIN");
// Redirect stdin to read from the in-memory file
dup2_stdin(&input_file).expect("Failed to set up stdin redirection");
Some(stdin_fd)
} else {
None
};
let (uumain_exit_status, captured_stdout, captured_stderr) = thread::scope(|s| {
let out = s.spawn(|| read_from_fd(read_pipe_stdout));
let err = s.spawn(|| read_from_fd(read_pipe_stderr));
#[allow(clippy::unnecessary_to_owned)]
// TODO: clippy wants us to use args.iter().cloned() ?
let status = uumain_function(args.to_owned().into_iter());
// Reset the exit code global variable in case we run another test after this one
// See https://github.com/uutils/coreutils/issues/5777
uucore::error::set_exit_code(0);
io::stdout().flush().unwrap();
io::stderr().flush().unwrap();
// Drop write ends to close them, allowing readers to get EOF
drop(write_pipe_stdout);
drop(write_pipe_stderr);
// Restore stdout/stderr
let _ = dup2_stdout(&original_stdout_fd_owned);
let _ = dup2_stderr(&original_stderr_fd_owned);
(status, out.join().unwrap(), err.join().unwrap())
});
// Restore the original stdin if it was modified
if let Some(fd) = original_stdin_fd_owned {
dup2_stdin(&fd).expect("Failed to restore the original STDIN");
}
CommandResult {
stdout: captured_stdout,
stderr: captured_stderr
.split_once(':')
.map(|x| x.1)
.unwrap_or("")
.trim()
.to_string(),
exit_code: uumain_exit_status,
}
}
fn read_from_fd(fd: impl std::os::fd::AsFd) -> String {
let mut captured_output = Vec::new();
let mut read_buffer = [0; 1024];
loop {
match read(&fd, &mut read_buffer) {
Ok(bytes_read) => {
if bytes_read == 0 {
break;
}
captured_output.extend_from_slice(&read_buffer[..bytes_read]);
}
Err(_) => {
eprintln!("Failed to read from the pipe");
break;
}
}
}
String::from_utf8_lossy(&captured_output).into_owned()
}
pub fn run_gnu_cmd(
cmd_path: &str,
args: &[OsString],
check_gnu: bool,
pipe_input: Option<&str>,
) -> Result<CommandResult, CommandResult> {
// if the check passes, do nothing
if check_gnu && let Err(e) = is_gnu_cmd(cmd_path) {
// Convert the io::Error into the function's error type
return Err(CommandResult {
stdout: String::new(),
stderr: e.to_string(),
exit_code: -1,
});
}
let mut command = Command::new(cmd_path);
for arg in args {
command.arg(arg);
}
// See https://github.com/uutils/coreutils/issues/6794
// uutils' coreutils is not locale-aware, and aims to mirror/be compatible with GNU Core Utilities's LC_ALL=C behavior
command.env("LC_ALL", "C");
let output = if let Some(input_str) = pipe_input {
// We have an pipe input
command
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped());
let mut child = command.spawn().expect("Failed to execute command");
let child_stdin = child.stdin.as_mut().unwrap();
child_stdin
.write_all(input_str.as_bytes())
.expect("Failed to write to stdin");
match child.wait_with_output() {
Ok(output) => output,
Err(e) => {
return Err(CommandResult {
stdout: String::new(),
stderr: e.to_string(),
exit_code: -1,
});
}
}
} else {
// Just run with args
match command.output() {
Ok(output) => output,
Err(e) => {
return Err(CommandResult {
stdout: String::new(),
stderr: e.to_string(),
exit_code: -1,
});
}
}
};
let exit_code = output.status.code().unwrap_or(-1);
// Here we get stdout and stderr as Strings
let stdout = String::from_utf8_lossy(&output.stdout).to_string();
let stderr = String::from_utf8_lossy(&output.stderr).to_string();
let stderr = stderr
.split_once(':')
.map(|x| x.1)
.unwrap_or("")
.trim()
.to_string();
if output.status.success() || !check_gnu {
Ok(CommandResult {
stdout,
stderr,
exit_code,
})
} else {
Err(CommandResult {
stdout,
stderr,
exit_code,
})
}
}
/// Compare results from two different implementations of a command.
///
/// # Arguments
/// * `test_type` - The command.
/// * `input` - The input provided to the command.
/// * `rust_result` - The result of running the command with the Rust implementation.
/// * `gnu_result` - The result of running the command with the GNU implementation.
/// * `fail_on_stderr_diff` - Whether to fail the test if there is a difference in stderr output.
pub fn compare_result(
test_type: &str,
input: &str,
pipe_input: Option<&str>,
rust_result: &CommandResult,
gnu_result: &CommandResult,
fail_on_stderr_diff: bool,
) {
print_section(format!("Compare result for: {test_type} {input}"));
if let Some(pipe) = pipe_input {
println!("Pipe: {pipe}");
}
let mut discrepancies = Vec::new();
let mut should_panic = false;
if rust_result.stdout.trim() != gnu_result.stdout.trim() {
discrepancies.push("stdout differs");
println!("Rust stdout:");
print_or_empty(rust_result.stdout.as_str());
println!("GNU stdout:");
print_or_empty(gnu_result.stdout.as_ref());
print_diff(&rust_result.stdout, &gnu_result.stdout);
should_panic = true;
}
if rust_result.stderr.trim() != gnu_result.stderr.trim() {
discrepancies.push("stderr differs");
println!("Rust stderr:");
print_or_empty(rust_result.stderr.as_str());
println!("GNU stderr:");
print_or_empty(gnu_result.stderr.as_str());
print_diff(&rust_result.stderr, &gnu_result.stderr);
if fail_on_stderr_diff {
should_panic = true;
}
}
if rust_result.exit_code != gnu_result.exit_code {
discrepancies.push("exit code differs");
println!(
"Different exit code: (Rust: {}, GNU: {})",
rust_result.exit_code, gnu_result.exit_code
);
should_panic = true;
}
if discrepancies.is_empty() {
print_end_with_status("Same behavior", true);
} else {
print_with_style(
format!("Discrepancies detected: {}", discrepancies.join(", ")),
Style::new().red(),
);
if should_panic {
print_end_with_status(
format!("Test failed and will panic for: {test_type} {input}"),
false,
);
panic!("Test failed for: {test_type} {input}");
} else {
print_end_with_status(
format!("Test completed with discrepancies for: {test_type} {input}"),
false,
);
}
}
println!();
}
pub fn generate_random_string(max_length: usize) -> String {
let mut rng = rand::rng();
let valid_utf8: Vec<char> =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789🔩🪛🪓⚙️🔗🧰"
.chars()
.collect();
let invalid_utf8 = [0xC3, 0x28]; // Invalid UTF-8 sequence
let mut result = String::new();
for _ in 0..rng.random_range(0..=max_length) {
if rng.random_bool(0.9) {
let ch = valid_utf8.choose(&mut rng).unwrap();
result.push(*ch);
} else {
let ch = invalid_utf8.choose(&mut rng).unwrap();
if let Some(c) = char::from_u32(*ch as u32) {
result.push(c);
}
}
}
result
}
#[allow(dead_code)]
pub fn generate_random_file() -> Result<String, std::io::Error> {
let mut rng = rand::rng();
let file_name: String = (0..10)
.map(|_| rng.random_range(b'a'..=b'z') as char)
.collect();
let mut file_path = temp_dir();
file_path.push(file_name);
let mut file = File::create(&file_path)?;
let content_length = rng.random_range(10..1000);
let content: String = (0..content_length)
.map(|_| rng.random_range(b' '..=b'~') as char)
.collect();
file.write_all(content.as_bytes())?;
Ok(file_path.to_str().unwrap().to_string())
}
#[allow(dead_code)]
pub fn replace_fuzz_binary_name(cmd: &str, result: &mut CommandResult) {
let fuzz_bin_name = format!("fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_{cmd}");
result.stdout = result.stdout.replace(&fuzz_bin_name, cmd);
result.stderr = result.stderr.replace(&fuzz_bin_name, cmd);
}
#[cfg(test)]
mod tests {
use super::*;
use std::ffi::OsString;
#[test]
fn test_command_result_creation() {
let result = CommandResult {
stdout: "Hello, world!".to_string(),
stderr: "".to_string(),
exit_code: 0,
};
assert_eq!(result.stdout, "Hello, world!");
assert_eq!(result.stderr, "");
assert_eq!(result.exit_code, 0);
}
#[test]
fn test_generate_random_string() {
let result = generate_random_string(10);
// Check character count, not byte count (emojis are multi-byte)
assert!(result.chars().count() <= 10);
// Test that empty string can be generated (max_length = 0)
let empty_result = generate_random_string(0);
assert_eq!(empty_result.chars().count(), 0);
}
#[test]
fn test_replace_fuzz_binary_name() {
let mut result = CommandResult {
stdout: "fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_echo: error".to_string(),
stderr: "fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_echo failed".to_string(),
exit_code: 1,
};
replace_fuzz_binary_name("echo", &mut result);
assert_eq!(result.stdout, "echo: error");
assert_eq!(result.stderr, "echo failed");
assert_eq!(result.exit_code, 1);
}
#[test]
fn test_run_gnu_cmd_nonexistent() {
let args = vec![OsString::from("--version")];
let result = run_gnu_cmd("nonexistent_command_12345", &args, false, None);
// Should return an error since the command doesn't exist
assert!(result.is_err());
let error_result = result.unwrap_err();
assert_ne!(error_result.exit_code, 0);
}
#[test]
fn test_run_gnu_cmd_basic() {
// Test with a simple command that should exist on most systems
let args = vec![OsString::from("--version")];
let result = run_gnu_cmd("echo", &args, false, None);
// Should succeed (echo --version might not be standard but echo should exist)
if let Err(e) = result {
// Command failed but at least ran
assert_ne!(e.exit_code, -1); // -1 would indicate the command couldn't be found
}
}
#[test]
fn test_run_gnu_cmd_with_pipe_input() {
let args: Vec<OsString> = vec![];
let pipe_input = "hello world";
let result = run_gnu_cmd("cat", &args, false, Some(pipe_input));
// cat might not be available in test environment, that's ok
if let Ok(cmd_result) = result {
assert_eq!(cmd_result.stdout.trim(), "hello world");
}
}
#[test]
fn test_generate_random_file() {
let result = generate_random_file();
// File creation might fail due to permissions, that's acceptable for this test
if let Ok(path) = result {
assert!(!path.is_empty());
// Clean up - try to remove the file
let _ = std::fs::remove_file(&path);
}
}
}
+69
View File
@@ -0,0 +1,69 @@
// This file is part of the uutils coreutils package.
//
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
use std::fmt;
use console::{Style, style};
use similar::TextDiff;
pub fn print_section<S: fmt::Display>(s: S) {
println!("{}", style(format!("=== {s}")).bold());
}
pub fn print_subsection<S: fmt::Display>(s: S) {
println!("{}", style(format!("--- {s}")).bright());
}
#[allow(dead_code)]
pub fn print_test_begin<S: fmt::Display>(msg: S) {
println!(
"{} {} {}",
style("===").bold(), // Kind of gray
style("TEST").black().on_yellow().bold(),
style(msg).bold()
);
}
pub fn print_end_with_status<S: fmt::Display>(msg: S, ok: bool) {
let ok = if ok {
style(" OK ").black().on_green().bold()
} else {
style(" KO ").black().on_red().bold()
};
println!(
"{} {ok} {}",
style("===").bold(), // Kind of gray
style(msg).bold()
);
}
pub fn print_or_empty(s: &str) {
let to_print = if s.is_empty() { "(empty)" } else { s };
println!("{}", style(to_print).dim());
}
pub fn print_with_style<S: fmt::Display>(msg: S, style: Style) {
println!("{}", style.apply_to(msg));
}
pub fn print_diff(got: &str, expected: &str) {
let diff = TextDiff::from_lines(got, expected);
print_subsection("START diff");
for change in diff.iter_all_changes() {
let (sign, style) = match change.tag() {
similar::ChangeTag::Equal => (" ", Style::new().dim()),
similar::ChangeTag::Delete => ("-", Style::new().red()),
similar::ChangeTag::Insert => ("+", Style::new().green()),
};
print!("{}{}", style.apply_to(sign).bold(), style.apply_to(change));
}
print_subsection("END diff");
println!();
}
+13 -3
View File
@@ -22,7 +22,7 @@ use std::io::{IsTerminal as _, Read};
use std::path::Path;
use uucore::error::{FromIo, UResult, USimpleError};
#[derive(Clone, Copy, PartialEq, Eq)]
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
#[doc(hidden)]
pub enum RegexMode {
Fixed,
@@ -255,6 +255,14 @@ pub fn uumain(args: impl uucore::Args) -> UResult<()> {
));
}
// GNU grep's PCRE backend (-P) supports only a single pattern.
if perl_regexp && patterns.len() > 1 {
return Err(USimpleError::new(
2,
"the -P option only supports a single pattern".to_string(),
));
}
// Decoded options into enums
let regex_mode = if fixed_strings {
RegexMode::Fixed
@@ -707,14 +715,16 @@ pub fn uu_app() -> Command {
.short('L')
.long("files-without-match")
.help("print only names of FILEs with no selected lines")
.action(ArgAction::SetTrue),
.action(ArgAction::SetTrue)
.overrides_with("files_with_matches"),
)
.arg(
Arg::new("files_with_matches")
.short('l')
.long("files-with-matches")
.help("print only names of FILEs with selected lines")
.action(ArgAction::SetTrue),
.action(ArgAction::SetTrue)
.overrides_with("files_without_match"),
)
.arg(
Arg::new("count")
+190 -1
View File
@@ -3,7 +3,7 @@
// For the full copyright and license information, please view the LICENSE
// file that was distributed with this source code.
use memchr::memchr;
use memchr::{memchr, memrchr};
use std::fs::File;
use std::io::{self, Read as _};
@@ -111,4 +111,193 @@ impl LineBuffer {
self.end += n;
}
}
/// Read the next run of *complete* lines as a single slice.
///
/// Returns `Ok(None)` at end of input. Otherwise returns `Ok(Some((chunk,
/// chunk_start)))`, where `chunk` spans one or more whole lines (each ending
/// in the terminator) and `chunk_start` is the absolute byte offset of the
/// first byte of the chunk. The only exception is a final line lacking a
/// terminator, which is returned on its own as the last chunk.
///
/// This hands back as much buffered data as ends on a line boundary, so a
/// caller can scan many lines with one pass instead of line by line.
pub fn read_chunk(&mut self, file: &mut File) -> io::Result<Option<(&[u8], u64)>> {
loop {
// Hand back everything up to and including the last terminator.
if self.end > self.beg
&& let Some(off) = memrchr(self.line_terminator, &self.buffer[self.beg..self.end])
{
let beg = self.beg;
let lim = self.beg + off + 1;
let chunk_start = self.next_line_start;
self.next_line_start += (lim - beg) as u64;
self.beg = lim;
self.scan = lim;
return Ok(Some((&self.buffer[beg..lim], chunk_start)));
}
// No whole line buffered. At EOF, flush any unterminated remainder.
if self.eof {
if self.beg == self.end {
return Ok(None);
}
let beg = self.beg;
let chunk_start = self.next_line_start;
self.next_line_start += (self.end - beg) as u64;
self.beg = self.end;
self.scan = self.end;
return Ok(Some((&self.buffer[beg..self.end], chunk_start)));
}
// Slide the partial tail to the front to maximize room for reading.
if self.beg > 0 {
self.buffer.copy_within(self.beg..self.end, 0);
self.end -= self.beg;
self.beg = 0;
self.scan = 0;
}
if self.end == self.buffer.len() {
// A single line is longer than the whole buffer; grow it.
self.buffer.resize(self.buffer.len() * 2, 0);
}
let n = loop {
match file.read(&mut self.buffer[self.end..]) {
Ok(n) => break n,
Err(e) if e.kind() == io::ErrorKind::Interrupted => {}
Err(e) => return Err(e),
}
};
if n == 0 {
self.eof = true;
} else {
self.end += n;
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::{Seek as _, SeekFrom, Write as _};
use std::sync::atomic::{AtomicU32, Ordering};
static COUNTER: AtomicU32 = AtomicU32::new(0);
/// A temp file pre-loaded with `content`, rewound to the start, and removed
/// from disk when dropped.
struct TempInput {
file: File,
path: std::path::PathBuf,
}
impl Drop for TempInput {
fn drop(&mut self) {
let _ = std::fs::remove_file(&self.path);
}
}
fn temp_input(content: &[u8]) -> TempInput {
let mut path = std::env::temp_dir();
let n = COUNTER.fetch_add(1, Ordering::Relaxed);
path.push(format!("uu_grep_lb_{}_{n}.tmp", std::process::id()));
let mut file = std::fs::OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(true)
.open(&path)
.unwrap();
file.write_all(content).unwrap();
file.seek(SeekFrom::Start(0)).unwrap();
TempInput { file, path }
}
/// Drain `read_chunk` into a list of (owned bytes, start offset) pairs.
fn chunks(term: u8, content: &[u8]) -> Vec<(Vec<u8>, u64)> {
let mut lb = LineBuffer::new(term);
let mut input = temp_input(content);
let mut out = Vec::new();
while let Some((chunk, start)) = lb.read_chunk(&mut input.file).unwrap() {
out.push((chunk.to_vec(), start));
}
out
}
#[test]
fn empty_input_yields_nothing() {
assert!(chunks(b'\n', b"").is_empty());
}
#[test]
fn whole_complete_lines_come_back_as_one_chunk() {
// Small input arrives in a single read, so everything up to the final
// terminator is one chunk starting at offset 0.
assert_eq!(
chunks(b'\n', b"a\nbb\nccc\n"),
vec![(b"a\nbb\nccc\n".to_vec(), 0)]
);
}
#[test]
fn unterminated_tail_is_a_final_chunk_with_its_own_offset() {
// "a\n" is the complete-line chunk; "bb" is flushed at EOF at offset 2.
assert_eq!(
chunks(b'\n', b"a\nbb"),
vec![(b"a\n".to_vec(), 0), (b"bb".to_vec(), 2)]
);
}
#[test]
fn input_without_any_terminator_is_one_chunk() {
assert_eq!(chunks(b'\n', b"abc"), vec![(b"abc".to_vec(), 0)]);
}
#[test]
fn honors_a_custom_terminator() {
assert_eq!(
chunks(b'\0', b"a\0bb\0c"),
vec![(b"a\0bb\0".to_vec(), 0), (b"c".to_vec(), 5)]
);
}
#[test]
fn reassembles_input_larger_than_the_buffer() {
// Force many reads and at least one chunk boundary mid-file.
let mut content = Vec::new();
for i in 0..50_000u32 {
content.extend_from_slice(format!("line number {i}\n").as_bytes());
}
assert!(content.len() > 128 * 1024);
let got = chunks(b'\n', &content);
assert!(got.len() > 1, "expected multiple chunks, got {}", got.len());
// Chunks must tile the input exactly, contiguously, each ending on a
// line boundary (the input ends with a terminator).
let mut expected_start = 0u64;
let mut joined = Vec::new();
for (bytes, start) in &got {
assert_eq!(*start, expected_start);
assert_eq!(*bytes.last().unwrap(), b'\n');
expected_start += bytes.len() as u64;
joined.extend_from_slice(bytes);
}
assert_eq!(joined, content);
}
#[test]
fn grows_to_hold_a_single_overlong_line() {
// One line far bigger than the initial 128 KiB buffer, then a short one.
let mut content = vec![b'x'; 300 * 1024];
content.push(b'\n');
content.extend_from_slice(b"tail\n");
let got = chunks(b'\n', &content);
let joined: Vec<u8> = got.iter().flat_map(|(b, _)| b.clone()).collect();
assert_eq!(joined, content);
assert_eq!(got[0].1, 0);
}
}
+118 -8
View File
@@ -4,6 +4,7 @@
// file that was distributed with this source code.
use crate::{Config, RegexMode};
use memchr::memmem;
use onig::{
EncodedBytes, Regex, RegexOptions, Region, SearchOptions, Syntax, SyntaxBehavior,
SyntaxOperator,
@@ -14,6 +15,12 @@ use uucore::error::{UResult, USimpleError};
pub struct Matcher<'a> {
config: &'a Config<'a>,
patterns: Vec<CompiledPattern>,
/// One substring searcher per pattern, present only when *every* pattern is
/// a plain literal that a raw byte search resolves exactly (see
/// [`plain_literal`]). When set, a caller can decide a line matches by
/// looking for any of these needles, bypassing the regex engine entirely.
/// `None` as soon as a single pattern needs real regex evaluation.
literal_searchers: Option<Vec<memmem::Finder<'static>>>,
}
impl<'a> Matcher<'a> {
@@ -22,19 +29,41 @@ impl<'a> Matcher<'a> {
for raw in config.patterns {
patterns.push(CompiledPattern::compile(raw, config)?);
}
Ok(Self { config, patterns })
// If we can reduce the whole pattern set to literal needles, keep a
// searcher for each so the driver can take a bulk substring-scan path.
let needles: Option<Vec<Vec<u8>>> = config
.patterns
.iter()
.map(|p| plain_literal(p, config.ignore_case, config.regex_mode))
.collect();
let literal_searchers = needles.filter(|n| !n.is_empty()).map(|n| {
n.iter()
.map(|w| memmem::Finder::new(w).into_owned())
.collect()
});
Ok(Self {
config,
patterns,
literal_searchers,
})
}
/// Per-pattern substring searchers, present only when the pattern set is a
/// pure set of literals (no regex needed). Used by the searcher to scan a
/// whole buffer at once instead of testing line by line.
pub fn literal_searchers(&self) -> Option<&[memmem::Finder<'static>]> {
self.literal_searchers.as_deref()
}
/// Decide whether `line` matches and return the positions to highlight.
pub fn match_line(&self, line: &[u8]) -> Option<Vec<(usize, usize)>> {
let mut any_seen = false;
let mut any_selected = false;
let positions: Vec<_> = MatchIter::new(&self.patterns, line)
.filter(|&(start, end)| {
any_seen = true;
// Drop zero-length matches from the output.
if start == end {
return false;
}
// Drop matches that don't span the whole line if `-x` was requested.
if self.config.line_regexp && !(start == 0 && end == line.len()) {
return false;
@@ -43,13 +72,19 @@ impl<'a> Matcher<'a> {
if self.config.word_regexp && !Self::is_word_match(line, start, end) {
return false;
}
any_selected = true;
// Drop zero-length matches from the output.
if start == end {
return false;
}
true
})
.collect();
let raw_matched = if self.config.line_regexp || self.config.word_regexp {
// -w / -x are authoritative once positions are filtered.
!positions.is_empty()
// -w / -x are authoritative once matches are filtered. Zero-length
// matches can select a line even though there is no span to output.
any_selected
} else {
any_seen
};
@@ -174,7 +209,7 @@ struct Cursor<'a> {
impl Cursor<'_> {
fn refill(&mut self) {
if self.offset >= self.line.len() {
if self.offset > self.line.len() {
self.pending = None;
return;
}
@@ -194,6 +229,25 @@ impl Cursor<'_> {
}
}
/// Return the literal bytes of `pattern` when a raw byte-for-byte substring
/// search is *exactly* equivalent to matching it, otherwise `None`.
///
/// We accept only ASCII, case-sensitive needles. That keeps the byte search in
/// agreement with the regex engine on every possible input, including bytes that
/// are not valid UTF-8: an ASCII byte can never be part of a multi-byte sequence,
/// so its presence is unambiguous. In the regex modes we also require that no
/// byte could ever act as a metacharacter; under `-F` the text is literal as-is.
fn plain_literal(pattern: &str, ignore_case: bool, mode: RegexMode) -> Option<Vec<u8>> {
if ignore_case || pattern.is_empty() || !pattern.is_ascii() {
return None;
}
// Every byte that carries special meaning in any of our regex syntaxes.
// A needle without these reads the same as a literal in Basic/Extended/Perl.
const SPECIAL: &[u8] = b".*[]^$\\+?{}()|";
let plain = mode == RegexMode::Fixed || !pattern.bytes().any(|b| SPECIAL.contains(&b));
plain.then(|| pattern.as_bytes().to_vec())
}
struct CompiledPattern {
/// Default semantics. It's decently fast and used for searching.
leftmost: Regex,
@@ -215,6 +269,10 @@ impl CompiledPattern {
// GNU grep supports `{,n}` as an alias for `{0,n}`.
syntax.enable_behavior(SyntaxBehavior::SYNTAX_BEHAVIOR_ALLOW_INTERVAL_LOW_ABBREV);
}
if matches!(config.regex_mode, RegexMode::Basic | RegexMode::Extended) {
// GNU grep supports \` and \' as buffer anchors in BRE and ERE.
syntax.enable_operators(SyntaxOperator::SYNTAX_OPERATOR_ESC_GNU_BUF_ANCHOR);
}
if config.regex_mode == RegexMode::Perl {
// GNU grep supports `(?P<name>...)`.
// Unfortunately, the onig crate defines the OP2 flag without the
@@ -231,6 +289,12 @@ impl CompiledPattern {
if config.ignore_case {
options |= RegexOptions::REGEX_OPTION_IGNORECASE;
}
// In GNU grep's Basic/Extended modes, `-z` makes newline ordinary data
// for `.`, but PCRE keeps its existing non-DOTALL behavior. The GNU
// `pcre-context` test documents this as current behavior until PCRE2.
if config.null_data && matches!(config.regex_mode, RegexMode::Basic | RegexMode::Extended) {
options |= RegexOptions::REGEX_OPTION_MULTILINE;
}
fn compile_with(pattern: &str, syntax: &Syntax, options: RegexOptions) -> UResult<Regex> {
Regex::with_options_and_encoding(pattern, options, syntax).map_err(|err| {
@@ -289,3 +353,49 @@ impl CompiledPattern {
.is_some()
}
}
#[cfg(test)]
mod tests {
use super::plain_literal;
use crate::RegexMode;
fn lit(p: &str, ic: bool, mode: RegexMode) -> Option<Vec<u8>> {
plain_literal(p, ic, mode)
}
#[test]
fn fixed_mode_takes_any_ascii_verbatim() {
// Under -F every byte is literal, even regex metacharacters.
assert_eq!(lit("abc", false, RegexMode::Fixed), Some(b"abc".to_vec()));
assert_eq!(lit("a.*b", false, RegexMode::Fixed), Some(b"a.*b".to_vec()));
assert_eq!(lit("a+b", false, RegexMode::Fixed), Some(b"a+b".to_vec()));
}
#[test]
fn regex_modes_accept_metacharacter_free_literals() {
for mode in [RegexMode::Basic, RegexMode::Extended, RegexMode::Perl] {
assert_eq!(lit("ing", false, mode), Some(b"ing".to_vec()));
assert_eq!(lit("Hello123", false, mode), Some(b"Hello123".to_vec()));
}
}
#[test]
fn regex_modes_reject_anything_with_a_metacharacter() {
for mode in [RegexMode::Basic, RegexMode::Extended, RegexMode::Perl] {
for p in [
"a.b", "a*", "[ab]", "^a", "a$", "a\\b", "a+", "a?", "(a)", "a|b", "a{2}",
] {
assert_eq!(lit(p, false, mode), None, "pattern {p:?} in {mode:?}");
}
}
}
#[test]
fn rejects_empty_case_insensitive_and_non_ascii() {
assert_eq!(lit("", false, RegexMode::Fixed), None);
assert_eq!(lit("abc", true, RegexMode::Fixed), None); // -i
assert_eq!(lit("abc", true, RegexMode::Basic), None);
assert_eq!(lit("café", false, RegexMode::Fixed), None); // non-ASCII
assert_eq!(lit("naïve", false, RegexMode::Basic), None);
}
}
+6
View File
@@ -71,6 +71,7 @@ impl<'a> OutputWriter<'a> {
view.line_number,
view.byte_offset + start as u64,
b':',
false,
)?;
self.write_colored_bytes(
@@ -90,6 +91,7 @@ impl<'a> OutputWriter<'a> {
view.line_number,
view.byte_offset,
if view.is_match { b':' } else { b'-' },
view.line.is_empty(),
)?;
let mut last_end = 0;
@@ -125,6 +127,7 @@ impl<'a> OutputWriter<'a> {
line_number: u64,
byte_offset: u64,
sep_char: u8,
content_empty: bool,
) -> io::Result<()> {
if self.config.show_filename {
self.write_colored_fmt(
@@ -155,7 +158,10 @@ impl<'a> OutputWriter<'a> {
self.write_separator(sep_char)?;
}
// GNU grep aligns content with a tab under -T, but only when there is
// content to align: an empty line keeps just its prefix (no tab).
if self.config.initial_tab
&& !content_empty
&& (self.config.line_number || self.config.byte_offset || self.config.show_filename)
{
self.out.write_all(b"\t")?;

Some files were not shown because too many files have changed in this diff Show More